Skip to content

MLBEDSW-11373: Calc scales for FCs with PC quant with high precision

Johan Gunnarsson requested to merge MLBEDSW-11373 into main

TFLite FULLY CONNECTED with per-channel quantization should use double precision when converting to explicit scales. This differs slightly compared to TFLite FULLY CONNECTED with global quantization which should use single precision when converting to explicit scales.

Signed-off-by: Johan Gunnarsson johan.gunnarsson@arm.com Change-Id: I483e753115436a16f4ad5e06e8f2b96b86885a9e

Merge request reports