The Transformers library relies on PIL (Pillow) for image preprocessing, which can become a major bottleneck during inference, especially with compiled models where the preprocessing time can dominate the overall inference time.
In the examples above, the RT-DETR preprocessing necessitates only to resize the image, while the DETR one involves resize+normalize.
However, there are other existing libraries that can do image preprocessing much more efficiently, and using the available hardware to its full capacity. OptimVision uses such library to get much better results compared to Transformers.
More specifically, OptimVision uses torchvision v2 preprocessing functions, and the following pipeline (each steps will be explained in more details later in the document)
Here are some benchmark and comparisons that led to these choices:
We test here different methods to load images as tensor on GPU. These metrics were averaged over 1000 iterations.
While torchvision.io.read_image is the fastest, we also note that loading PIL images and converting them to tensors on GPU is barely slower, which is nice as most users are probably used to passing PIL images to processors.
On this last point, it is also important to note this following quirk of Pytorch:
When transferring tensors to GPU and changing their dtypes, it seems as if using “tensor”.to(’cuda’, dtype=dtype) is much slower than “tensor”.to(’cuda’).to(dtype=dtype). Thus we only use “tensor”.to(’cuda’).to(dtype=dtype) in these comparisons.
Some important things to note in this comparison: All the methods operating on floats (including Albumentations and OpenCV) outputs slightly different results than the PIL resize method currently used in Transformers. The differences are exacerbated on GPU.
For that reason (and until we can assess the impact of these differences), we use torchvision v2 resize operating on uint8 tensors using GPU.