fundamentalvision/BEVFormer on Jetson AGX Orin

BEVFormer-small (R101-DCN backbone) detects 3D objects in the ten nuScenes classes from six surround-view cameras.

BEVFormer is by Zhiqi Li et al. (arXiv:2203.17270), Apache-2.0, at fundamentalvision/BEVFormer; the TensorRT export and plugins are from DerryHub/BEVFormer_tensorrt, Apache-2.0.

Methodology

Latency: trtexec GPU compute time of the TensorRT 8.5.1.7 INT8 engine on the Jetson AGX Orin, batch 1, one nuScenes sample (six cameras); mean, p99 and throughput from the same run.

Accuracy: NDS and mAP on the 6,019-sample nuScenes validation set, evaluated in PyTorch FP16 with INT8 fake quantization on an NVIDIA L4 with BEVFormer_tensorrt’s bevformer_small_trt_p_q config, not on the device engine. The BEVFormer model zoo’s numbers are shown beside it.

Build: exported to ONNX with DerryHub/BEVFormer_tensorrt and compiled on the device with trtexec --fp16 --int8 and its plugin library, which the runtime image carries.

Baseline TensorRT: the FP16 build of the same model on TensorRT 8.5.1.7 with the BEVFormer_tensorrt plugins, same device, same trtexec measurement.

Files

Artifacts

What ships in the model bundle.

File	Kind	Size	sha256
TensorRT engine engine/bevformer_small_trt_p_ptq_max_cp_trtexec.engine	Engine	99.5 MiB	16ba44bb915a
trtexec build log logs/trtexec_build.log	Log	12.1 MiB	2a07369a0848
HTTP server for the engine (serve.sh --port 8000) engine/serve.sh	script	2 KiB	c59eaebcab93
Model settings bevformer.json	Config	396 B	34acbb4f71d8
Latency (trtexec) benchmark/metrics.json	Results	4 KiB	af13646e3f9b
Accuracy (PyTorch, on the host) benchmark/accuracy.json	Results	344 B	9fa0da9d0262
nuScenes sample (engine inputs, .npz) inference/nuscenes_sample.npz	sample	86.7 MiB	9e352c8dc5b1
Detections for the sample inference/detections.json	Results	109 KiB	1ce300bbd726
BEVFormer_tensorrt licence LICENSE-BEVFormer_tensorrt	license	11 KiB	c71d239df917

Variants

Every build of fundamentalvision/BEVFormer on AGX Orin.

Variant	Precision	Speed	Runtime
Baseline TensorRT fp16	FP16	295.20 ms Latency: GPU compute, mean	TensorRT 8.5.1.7 with BEVFormer_tensorrt plugins bevformer.orin.runtime.arm64.202610130739ec55
Embedl-optimized This page int8	INT8	301.49 ms Latency: GPU compute, mean	TensorRT 8.5.1.7 with BEVFormer_tensorrt plugins and Embedl optimizations bevformer.orin.runtime.arm64.202610130739ec55