Skip to content
Log in

Qwen/Qwen3-VL-4B-Instruct

ModelOpt (W4A16) · TensorRT Edge-LLM
Qwen/Qwen3-VL-4B-Instruct architecture graph
Qwen/Qwen3-VL-4B-Instruct architecture graph
  • huggingface
  • AGX Orin
Labels
  • Image
  • Text
  • VLM
  • Generation
Build
Embedl-optimized
Precision
W4A16
Runtime
TensorRT Edge-LLM 0.10.1 with Embedl optimizations
Context length
4,096 tokens
Run Qwen3-VL 4B on AGX Orin

Alibaba's 4B Qwen3-VL vision-language model for describing, reading and reasoning about images and video.

Methodology

  • Throughput: one user, synthetic prompts (random tokens, as in NVIDIA Jetson AI Lab's benchmarks), 2,048 tokens in, 128 out, greedy, no prompt reuse. Average of 50 requests after 3 warm-up requests. End-to-end is the whole request. Prompt processing is the 2,048 prompt tokens over the time to first token.
  • Power and memory: sampled on the device during those runs. Idle is the device at rest with the model loaded; energy per token is the run's energy over its generated tokens; peak memory is the device's.
  • Accuracy: MMLU-mini, 200 MMLU questions, 0-shot, greedy, run on the device. Not comparable with published 5-shot MMLU scores.
  • Vision-language model: measured on text only.

Files

Image Size

Model

docker.io/embedl/model-artifacts:qwen3-vl-4b.huggingface_int4_modelopt_tensorrt-edgellm.851dcac34cbf0b13

 4.02 GiB

Runtime

docker.io/embedl/models:tensorrt-edgellm_arm64_2c081bac0ba4e49edf6a8b0949553a71e6219e35

 0.92 GiB