Parallel processing

Parallel processing refers to both mutithreading and batching.

Intra versus Inter parallel processing

We support 2 types of parallel processing: intra and inter.

Intra-processing

The elements processed in parallel are in the same image. For example, when multiple license plates or cars are in the same image they’re processed in parallel. Please also note that we heavily use SIMD (Single instruction, multiple data) which is another form of intra parallel processing. The supported extensions are SSE, AVX and NEON. We highly recommend using a CPU with support for AVX2.

Inter-processing

The elements processed in parallel are in different images. This mode will be activated if you provide a callback function when initializing the SDK.

Multithreading

The number of CPU threads is controlled using configuration entry num_threads. Multithreading works in both inter and intra parallel processing.

Batching

The batching mode is controlled using configuration entry batching_mode. Batching makes sense only if you have a GPU or NPU, otherwise (CPU) it’s multi-threading.