Skip to content

Some resize performance tests fail in OpenCV

Some of OpenCV performance tests have sanity check (some sparce result check against the values saved in XML). OpenCV: 4.x (PR: https://github.com/opencv/opencv/pull/25443) KleidiCV: main (df7b9991) Device: Pixel 8.

Steps to build: cmake -DCMAKE_TOOLCHAIN_FILE=/home/alexander/Android2/ndk/26.2.11394342/build/cmake/android.toolchain.cmake -DANDROID_SDK=/home/alexander/Android2/ -DANDROID_ABI=arm64-v8a -DWITH_KLEIDICV=ON -DKLEIDICV_SOURCE_PATH=/home/alexander/Projects/OpenCV/kleidicv/ ../opencv-master

List of failed cases:

  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/4, where GetParam() = (8UC3, 640x480, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/5, where GetParam() = (8UC3, 960x540, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/9, where GetParam() = (8UC4, 960x540, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/10, where GetParam() = (8UC4, 1280x720, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/16, where GetParam() = (16UC3, 640x480, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/17, where GetParam() = (16UC3, 960x540, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/18, where GetParam() = (16UC3, 1280x720, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/19, where GetParam() = (16UC3, 1920x1080, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/20, where GetParam() = (16UC4, 640x480, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/21, where GetParam() = (16UC4, 960x540, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/22, where GetParam() = (16UC4, 1280x720, 2)
  • MatInfo_Size_Scale_ResizeAreaFast.ResizeAreaFast/23, where GetParam() = (16UC4, 1920x1080, 2)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/15, where GetParam() = (8UC3, 640x480, 0.1)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/16, where GetParam() = (8UC3, 640x480, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/18, where GetParam() = (8UC3, 960x540, 0.1)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/19, where GetParam() = (8UC3, 960x540, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/20, where GetParam() = (8UC3, 960x540, 0.81)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/22, where GetParam() = (8UC3, 1280x720, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/23, where GetParam() = (8UC3, 1280x720, 0.81)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/25, where GetParam() = (8UC3, 1920x1080, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/27, where GetParam() = (8UC3, 3840x2160, 0.1)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/28, where GetParam() = (8UC3, 3840x2160, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/29, where GetParam() = (8UC3, 3840x2160, 0.81)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/30, where GetParam() = (8UC4, 640x480, 0.1)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/31, where GetParam() = (8UC4, 640x480, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/32, where GetParam() = (8UC4, 640x480, 0.81)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/33, where GetParam() = (8UC4, 960x540, 0.1)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/34, where GetParam() = (8UC4, 960x540, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/35, where GetParam() = (8UC4, 960x540, 0.81)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/36, where GetParam() = (8UC4, 1280x720, 0.1)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/37, where GetParam() = (8UC4, 1280x720, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/39, where GetParam() = (8UC4, 1920x1080, 0.1)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/40, where GetParam() = (8UC4, 1920x1080, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/43, where GetParam() = (8UC4, 3840x2160, 0.25)
  • MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/44, where GetParam() = (8UC4, 3840x2160, 0.81)

Fail example:

./opencv_perf_imgproc --perf_min_samples=1 --perf_force_samples=1 --gtest_filter=*Resize* 
...
[ RUN      ] MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/43, where GetParam() = (8UC4, 3840x2160, 0.25)
/home/alexander/Projects/OpenCV/opencv-master/modules/ts/src/ts_perf.cpp:370: Failure
The difference between expect_min and actual_min is 7, which exceeds eps, where
expect_min evaluates to 40,
actual_min evaluates to 47, and
eps evaluates to 1.
Argument "dst" has unexpected minimal value

params    = (8UC4, 3840x2160, 0.25)
termination reason:  reached maximum number of iterations
bytesIn   =   33177600
bytesOut  =    2073600
samples   =          1
outliers  =          0
frequency = 1000000000
min       =    1951335 = 1.95ms
median    =    1951335 = 1.95ms
gmean     =    1951335 = 1.95ms
gstddev   = 0.00000000 = 0.00ms for 97% dispersion interval
mean      =    1951335 = 1.95ms
stddev    =          0 = 0.00ms
[  FAILED  ] MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/43, where GetParam() = (8UC4, 3840x2160, 0.25) (27 ms)
[ RUN      ] MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/44, where GetParam() = (8UC4, 3840x2160, 0.81)
/home/alexander/Projects/OpenCV/opencv-master/modules/ts/src/ts_perf.cpp:370: Failure
The difference between expect_min and actual_min is 2, which exceeds eps, where
expect_min evaluates to 1,
actual_min evaluates to 3, and
eps evaluates to 1.
Argument "dst" has unexpected minimal value

params    = (8UC4, 3840x2160, 0.81)
termination reason:  reached maximum number of iterations
bytesIn   =   33177600
bytesOut  =   21770000
samples   =          1
outliers  =          0
frequency = 1000000000
min       =   50389689 = 50.39ms
median    =   50389689 = 50.39ms
gmean     =   50389689 = 50.39ms
gstddev   = 0.00000000 = 0.00ms for 97% dispersion interval
mean      =   50389689 = 50.39ms
stddev    =          0 = 0.00ms
[  FAILED  ] MatInfo_Size_Scale_Area_ResizeArea.ResizeArea/44, where GetParam() = (8UC4, 3840x2160, 0.81) (81 ms)