Improving the speed

This section explains how to improve the speed (frame rate).

Enable CPU acceleration

CPU acceleration is enabled using the backend json configuration entry.

  • On desktops (x86-64), OpenVINO is always faster than ONNX and consumes less memory. OpenVINO uses QINT8 models while ONNX uses FLOAT32.

  • On desktops (x86_64), OpenVINO is always faster than TfLite. Both use QINT8 models.

  • On mobile (ARM64), TfLite is always faster than ONNX and consumes less memory. TfLite uses QINT8 models while ONNX uses FLOAT32.

OpenVINO is supported on x86_64 only. TfLite and ONNX are supported on both x86_64 and ARM64.

Enable GPGPU acceleration

GPGPU acceleration is enabled using the backend json configuration entry.

  • NVIDIA GPUs: GPGPU acceleration is done using TensorRT and ONNX[CUDA]. TensorRT is faster.

  • Intel GPUs: GPGPU acceleration is done using OpenVINO.

  • Mobile (ARM64) GPUs: GPGPU acceleration is done using TfLite.

Enable NPU acceleration

NPU acceleration is enabled using the backend json configuration entry.

Check this section for more information on supported NPUs.

Parallel mode

Activate the parallel mode as explained in the previous sections. Please note that this won’t change the accuracy while your application will run up to #4 times faster than the sequential mode.

Memory alignment

Make sure to provide memory aligned data to the SDK. On ARM the preferred alignment is 16-bytes while on x86 it’s 32-bytes. If the input data is an image and the width isn’t aligned to the preferred alignment size, then it should be strided.

Please check the memory management section for more information.

Landscape mode

When the device is on portrait mode, then the image is rotated 90 or 270 degree (or any modulo 90 degree). On landscape mode it’s rotated 0 or 180 degree (or any modulo 180 degree). On some devices the image could also be horizontally/vertically mirrored in addition to being rotated. Our deep leaning model can natively handle rotations up to 45 degree but not 90, 180 or 270. There is a pre-processing operation to rotate the image back to 0 degree and remove the mirroring effect but such operation is time consuming on mobile devices. We recommend using the device on landscape mode to avoid the pre-processing operation.

Disabling License Plate Country Identification (LPCI)

Only enable this feature if you need it.

It’s useless to enable this function if you’re expecting the plates to be from a single country. This feature is useful in European Union (multiple countries), USA (multiple states), Canada (multiple provinces), UAE (multiple emirates)… but not in Russia, Korea, Tunisia or Mauritania.

Please check the configuration section on how to enable/disable LPCI.

Disabling Vehicle Color Recognition (VCR)

Only enable this feature if you need it.

Off course it doesn’t make sense to enable this feature if your camera is returning grayscale (8-bit) images. The result would be unpredictable even if you’re just trying to classify the colors as being black or white.

Please check the configuration section on how to enable/disable VCR.

Disabling Vehicle Make Model Recognition (VMMR)

Only enable this feature if you need it.

Vehicle Make Model Recognition (VMMR) feature adds latency to the pipeline. Make sure you really need it before enabling it.

Please check the configuration section on how to enable/disable VMMR.

Disabling Vehicle Body Style Recognition (VBSR)

Only enable this feature if you need it.

Vehicle Body Style Recognition (VBSR) feature adds latency to the pipeline. Make sure you really need it before enabling it.

Please check the configuration section on how to enable/disable VBSR.

Planar formats

Both the detector and recognizer expect a RGB_888 image as input but most likely your camera doesn’t support such format. Your camera will probably output YUV frames. If you can choose, then prefer the planar formats (e.g YUV420P) instead of the semi-planar ones (e.g. YUV420SP a.k.a NV21 or NV12). The issue with semi-planar formats is that we’ve to deinterleave the UV plane which takes some extra time.

Reducing camera frame rate and resolution

The CPU is a shared resource and all background tasks are fighting each other for their share of the resources. Requesting the camera to provide high resolution images at high frame rate means it’ll take a big share. It’s useless to have any frame rate above 25fps or any resolution above 720p (1280x720) unless you’re monitoring a very large zone and in such case we recommend using multiple cameras.

Change the platform

Change platform json configuration entry from desktop to mobile to use smaller detection weights. This has no effect on any other feature.