Skip to content

Size_MatType_OutMatDepth_integral performance test fials in OpenCV

Some of OpenCV performance tests have sanity check (some sparce result check against the values saved in XML). OpenCV: 4.x (PR: https://github.com/opencv/opencv/pull/25443) KleidiCV: main (df7b9991) Device: Pixel 8.

Steps to build: cmake -DCMAKE_TOOLCHAIN_FILE=/home/alexander/Android2/ndk/26.2.11394342/build/cmake/android.toolchain.cmake -DANDROID_SDK=/home/alexander/Android2/ -DANDROID_ABI=arm64-v8a -DWITH_KLEIDICV=ON -DKLEIDICV_SOURCE_PATH=/home/alexander/Projects/OpenCV/kleidicv/ ../opencv-master

List of failed cases:

  • Size_MatType_OutMatDepth_integral.integral/3, where GetParam() = (640x480, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/6, where GetParam() = (640x480, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/9, where GetParam() = (640x480, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/15, where GetParam() = (1280x720, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/18, where GetParam() = (1280x720, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/21, where GetParam() = (1280x720, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/27, where GetParam() = (1920x1080, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/30, where GetParam() = (1920x1080, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/33, where GetParam() = (1920x1080, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/39, where GetParam() = (127x61, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/42, where GetParam() = (127x61, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral.integral/45, where GetParam() = (127x61, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/3, where GetParam() = (640x480, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/6, where GetParam() = (640x480, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/9, where GetParam() = (640x480, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/15, where GetParam() = (1280x720, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/18, where GetParam() = (1280x720, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/21, where GetParam() = (1280x720, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/27, where GetParam() = (1920x1080, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/30, where GetParam() = (1920x1080, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/33, where GetParam() = (1920x1080, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/39, where GetParam() = (127x61, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/42, where GetParam() = (127x61, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum.integral_sqsum/45, where GetParam() = (127x61, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/3, where GetParam() = (640x480, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/6, where GetParam() = (640x480, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/9, where GetParam() = (640x480, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/15, where GetParam() = (1280x720, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/18, where GetParam() = (1280x720, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/21, where GetParam() = (1280x720, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/27, where GetParam() = (1920x1080, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/30, where GetParam() = (1920x1080, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/33, where GetParam() = (1920x1080, 8UC4, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/39, where GetParam() = (127x61, 8UC2, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/42, where GetParam() = (127x61, 8UC3, CV_32S)
  • Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/45, where GetParam() = (127x61, 8UC4, CV_32S)

Fail example:

./opencv_perf_imgproc --perf_min_samples=1 --perf_force_samples=1 --gtest_filter=*OutMatDepth_integral* 
...
[ RUN      ] Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/45, where GetParam() = (127x61, 8UC4, CV_32S)
/home/alexander/Projects/OpenCV/opencv-master/modules/ts/src/ts_perf.cpp:372: Failure
The difference between expect_max and actual_max is 748867, which exceeds eps, where
expect_max evaluates to 990791,
actual_max evaluates to 241924, and
eps evaluates to 9.9999999999999995e-07.
Argument "sum" has unexpected maximal value

params    = (127x61, 8UC4, CV_32S)
termination reason:  reached maximum number of iterations
bytesIn   =      30988
bytesOut  =      92964
samples   =          1
outliers  =          0
frequency = 1000000000
min       =     106364 = 0.11ms
median    =     106364 = 0.11ms
gmean     =     106364 = 0.11ms
gstddev   = 0.00000000 = 0.00ms for 97% dispersion interval
mean      =     106364 = 0.11ms
stddev    =          0 = 0.00ms
[  FAILED  ] Size_MatType_OutMatDepth_integral_sqsum_tilted.integral_sqsum_tilted/45, where GetParam() = (127x61, 8UC4, CV_32S) (1 ms)