Benchmark

It’s easy to assert that our implementation is the fastest you can find without backing our claim with numbers and source code freely available to everyone to check.

More information about the benchmark application could be found here and you can checkout the source code from Github.

UltimateALPR versus OpenALPR on Android

We’ve found #3 OpenALPR (for Android) repositories on Github:

  1. https://github.com/SandroMachado/openalpr-android [708 stars]

  2. https://github.com/RobertSasak/react-native-openalpr [338 stars]

  3. https://github.com/sujaybhowmick/OpenAlprDroidApp [102 stars]

We’ve decided to go with the one with most stars on Github which is [1]. We’re using recognizeWithCountryRegionNConfig(country=”us”, region=””, topN = 10).

Rules:
  • We’re using Samsung Galaxy S10+ (Snapdragon 855)

  • For every implementation we’re running the recognition function within a loop for #1000 times.

  • The positive rate defines the percentage of images with a plate. For example, 20% positives means we will have #800 negative images (no plate) and #200 positives (with a plate) out of the #1000 total images. This percentage is important as it allows timing both the detector and recognizer.

  • All positive images contain a single plate.

  • Both implementations are initialized outside the loop.

0% positives

20% positives

50% positives

70% positives

100% positives

ultimateALPR

21344 ms
46.85 fps

25815 ms
38.73 fps

29712 ms
33.65 fps

33352 ms
29.98 fps

37825 ms
26.43 fps

OpenALPR

715800 ms
1.39 fps

758300 ms
1.31 fps

819500 ms
1.22 fps

849100 ms
1.17 fps

899900 ms
1.11 fps

One important note from the above table is that the detector in OpenALPR is very slow and 80% of the time is spent trying to detect the license plates. This could be problematic as most of the time there is no plate on the video stream (negative images) from a camera filming a street/road and in such situations an application must run as fast as possible (above the camera maximum frame rate) to avoid dropping frames and loosing positive frames. Also, the detection part should burn as less as possible CPU cycles which means more energy efficient.

The above table shows that ultimateALPR is up to 33 times faster than OpenALPR.

To be fair to OpenALPR:

  1. The API only allows providing a file path which means for every loop they are reading and decoding the input while ultimateALPR accepts raw bytes.

  2. There is no ARM64 binaries provided and the app is loading the ARMv7 versions.

Again, our benchmark application is open source and doesn’t require registration or license key to try. You can try to make the same test on your own device and please don’t hesitate to share your numbers or any feedback if you think we missed something.

Raspberry Pi 4 (Raspbian Buster OS)

The Github repository contains Raspberry Pi benchamark application to evaluate the performance pi version 3 and later.

More information on how to build and use the application could be found at https://github.com/DoubangoTelecom/ultimateALPR-SDK/blob/master/samples/c++/benchmark/README.md.

Please note that even if Raspberry Pi 4 has a 64-bit CPU Raspbian OS uses a 32-bit kernel which means we’re loosing many SIMD optimizations.

0% positives

20% positives

50% positives

70% positives

100% positives

Raspberry Pi

81890 ms
12.21 fps

89770 ms
11.13 fps

115190 ms
8.68 fps

122950 ms
8.13 fps

141460 ms
7.06 fps

On Android devices we have noticed that parallel processing can speedup the pipeline by up to 120% on some devices while on Raspberry Pi 4 the gain is marginal.