Skip to content

Some cvtColor perfomance tests fails with accuracy issue in OpenCV

Some of OpenCV performance tests have sanity check (some sparce result check against the values saved in XML). OpenCV: 4.x (PR: https://github.com/opencv/opencv/pull/25443) KleidiCV: main (df7b9991) Device: Pixel 8.

Steps to build: cmake -DCMAKE_TOOLCHAIN_FILE=/home/alexander/Android2/ndk/26.2.11394342/build/cmake/android.toolchain.cmake -DANDROID_SDK=/home/alexander/Android2/ -DANDROID_ABI=arm64-v8a -DWITH_KLEIDICV=ON -DKLEIDICV_SOURCE_PATH=/home/alexander/Projects/OpenCV/kleidicv/ ../opencv-master

List of failed cases:

  • Size_CvtMode_cvtColor8u.cvtColor8u/8, where GetParam() = (127x61, COLOR_BGR2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/9, where GetParam() = (127x61, COLOR_BGR2Luv)
  • Size_CvtMode_cvtColor8u.cvtColor8u/12, where GetParam() = (127x61, COLOR_BGR2XYZ)
  • Size_CvtMode_cvtColor8u.cvtColor8u/13, where GetParam() = (127x61, COLOR_BGR2YCrCb)
  • Size_CvtMode_cvtColor8u.cvtColor8u/34, where GetParam() = (127x61, CX_BGRA2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/35, where GetParam() = (127x61, CX_BGRA2Luv)
  • Size_CvtMode_cvtColor8u.cvtColor8u/37, where GetParam() = (127x61, CX_BGRA2YCrCb)
  • Size_CvtMode_cvtColor8u.cvtColor8u/67, where GetParam() = (127x61, COLOR_LBGR2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/68, where GetParam() = (127x61, COLOR_LBGR2Luv)
  • Size_CvtMode_cvtColor8u.cvtColor8u/69, where GetParam() = (127x61, COLOR_LRGB2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/71, where GetParam() = (127x61, CX_LBGRA2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/73, where GetParam() = (127x61, CX_LRGBA2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/90, where GetParam() = (127x61, COLOR_RGB2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/91, where GetParam() = (127x61, COLOR_RGB2Luv)
  • Size_CvtMode_cvtColor8u.cvtColor8u/92, where GetParam() = (127x61, COLOR_RGB2XYZ)
  • Size_CvtMode_cvtColor8u.cvtColor8u/93, where GetParam() = (127x61, COLOR_RGB2YCrCb)
  • Size_CvtMode_cvtColor8u.cvtColor8u/103, where GetParam() = (127x61, CX_RGBA2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/104, where GetParam() = (127x61, CX_RGBA2Luv)
  • Size_CvtMode_cvtColor8u.cvtColor8u/106, where GetParam() = (127x61, CX_RGBA2YCrCb)
  • Size_CvtMode_cvtColor8u.cvtColor8u/210, where GetParam() = (640x480, COLOR_RGB2Lab)
  • Size_CvtMode_cvtColor8u.cvtColor8u/211, where GetParam() = (640x480, COLOR_RGB2Luv)
  • Size_CvtMode_cvtColor8u.cvtColor8u/307, where GetParam() = (1920x1080, COLOR_LBGR2Lab)
  • Size_CvtMode2_cvtColorYUV420.cvtColorYUV420/112, where GetParam() = (130x60, COLOR_BGR2YUV_UYVY)
  • Size_CvtMode2_cvtColorYUV420.cvtColorYUV420/116, where GetParam() = (130x60, COLOR_BGR2YUV_YUY2)
  • Size_CvtMode2_cvtColorYUV420.cvtColorYUV420/118, where GetParam() = (130x60, COLOR_BGR2YUV_YVYU)

Fail example:

[ RUN      ] Size_CvtMode_cvtColor8u.cvtColor8u/307, where GetParam() = (1920x1080, COLOR_LBGR2Lab)
/home/alexander/Projects/OpenCV/opencv-master/modules/ts/src/ts_perf.cpp:370: Failure
The difference between expect_min and actual_min is 2, which exceeds eps, where
expect_min evaluates to 7,
actual_min evaluates to 9, and
eps evaluates to 1.
Argument "dst" has unexpected minimal value

params    = (1920x1080, COLOR_LBGR2Lab)
termination reason:  reached maximum number of iterations
bytesIn   =    6220800
bytesOut  =    6220800
samples   =          1
outliers  =          0
frequency = 1000000000
min       =    5339974 = 5.34ms
median    =    5339974 = 5.34ms
gmean     =    5339974 = 5.34ms
gstddev   = 0.00000000 = 0.00ms for 97% dispersion interval
mean      =    5339974 = 5.34ms
stddev    =          0 = 0.00ms
[  FAILED  ] Size_CvtMode_cvtColor8u.cvtColor8u/307, where GetParam() = (1920x1080, COLOR_LBGR2Lab) (33 ms)
Edited by Alexander Smorkalov