Embedl-optimized · TensorRT
- huggingface
- INT8
- Orin NX 16GB
- Labels
- Image
- Text
- Segmentation
- Build
- Embedl-optimized
- Precision
- INT8
- Runtime
- TensorRT 10.16.2.10 with Embedl optimizations
- Input size
- 924×924 image, text prompt up to 32 tokens
- Output
- Masks, boxes and scores
Meta's SAM 3 finds every object a short text prompt names in a photo, such as "yellow school bus", and returns a mask, box and score for each one.
Methodology
- Latency: batch 1, one photo and prompt. GPU compute: the engine alone between CUDA events, copies excluded, mean over 20 s after a 5 s warm-up. Median latency: the served request end to end (tokenizing, resizing to 924×924, the engine, masks to PNG at the photo's size), median of 10 on one 640×480 COCO photo.
- Accuracy: COCO mask mAP (IoU 0.50 to 0.95), with mAP@50 and mAP@75, on the first 300 val2017 images with objects, each category present prompted by its name, through the served model's own pre- and post-processing. Only categories present are prompted, so the numbers compare builds and are not SAM 3's published results.
- Memory: peak resident memory of the server, engine loaded and one request answered (on Jetson, GPU memory is system memory).
- Size: the TensorRT engine.
- Baseline FP32: the unquantized SAM 3 in FP32 in onnxruntime, on the same images and prompts, for accuracy; its ONNX weights, for size. There is no FP16 baseline yet: TensorRT's FP16 build of the unquantized model finds nothing (mask mAP 0.0) because its values overflow FP16, and a valid one needs mixed precision. Latency is not compared until then.