I’m exploring the behavior of operations of TFlite for custom hardware. I quantized a pretrained VGG16 (from model zoo) into int8. The scale and zero point of input and output tensors are equal for each maxpool op. Since quantization is a monotonically increasing function, I believe the output of maxpool op (int8) should be the 2×2 maxpool of the input (int8). equivalent to following numpy code:
max_out_custom = max_in.reshape(1,112,2,112,2,64).max(axis=2).max(axis=3)
But it is not so, and I cant find a pattern. Any help will be appreciated.
Colab with example code: https://colab.research.google.com/drive/1410SH8uEE5IX0Iuvv27SwTtpCl2XM5T5?usp=sharing
submitted by /u/uncle-iroh-11
[visit reddit] [comments]