int8_embedl_tensorrt
- timm/efficientformerv2_s1.snap_dist_in1k
- INT8
- L4
- Labels
- Image
- Classification
- Source
- timm/efficientformerv2_s1.snap_dist_in1k (pretrained weights)
- Toolchain
- Embedl Deploy
- Backend
- TensorRT 11.2.1.2
- Calibration
- ImageNet, 5,000 images
- Validation
- ImageNet validation set, 50,000 images
INT8 build of the timm image classifier efficientformerv2_s1.snap_dist_in1k for the NVIDIA L4. Quantized with Embedl Deploy and compiled with TensorRT 11.2.1.2.
Methodology
- Latency: trtexec on the NVIDIA L4, batch 1, 201 iterations after warm-up, during the build: GPU compute time per inference, mean and p99, and throughput.
- Accuracy: the INT8 engine on the full ImageNet validation set (50,000 images), top-1 and top-5. timm publishes 79.684% top-1 for the FP32 weights (https://github.com/huggingface/pytorch-image-models/blob/v1.0.30/results/results-imagenet.csv); the INT8 engine is -0.33 points from it.
- Quantization: Embedl Deploy, activations INT8, weights INT8 per-channel; calibrated on ImageNet (5,000 images). Exported as ONNX opset 20, compiled with TensorRT 11.2.1.2, built on NVIDIA L4 (sm_89).
- Build: config efficientformerv2_s1/snap_dist_in1k-timm_int8_embedl_tensorrt_l4, on an NVIDIA L4.