Skip to content
Search models or paste a model link. Press Enter to search.

Nemotron 3 Nano 4B

nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
Left and right arrow keys switch the vendor.
nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 architecture graph
nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 architecture graph
  • huggingface
  • AGX Thor
Labels
  • Text
  • LLM
  • Generation
Build
Embedl-optimized
Precision
W4A16
Runtime
TensorRT Edge-LLM 0.10.1 with Embedl optimizations
Context length
4,096 tokens
Run Nemotron 3 Nano 4B on AGX Thor

NVIDIA's 4B Nemotron 3 Nano model for chat and agents on edge devices, with optional step-by-step thinking.

Methodology

  • Throughput: one user, real conversations from the public OpenAssistant set, 2,048 tokens in, 128 out, greedy, no prompt reuse. Average of 20 requests after 3 warm-up requests. End-to-end is the whole request. Prompt processing is the 2,048 prompt tokens over the time to first token.
  • Power and memory: sampled on the device during those runs. Idle is the device at rest with the model loaded; energy per token is the run's energy over its generated tokens; peak memory is the device's.
  • Accuracy: full test sets, greedy, the same model files and settings as served. GSM8K 1319, IFEval 541, MMLU-Pro 1008 questions.
  • Size: the runtime container image, compressed as downloaded and unpacked on disk, and the model's files; total size is the model's files plus the container download.
  • Baseline TensorRT Edge-LLM: the original FP16 model on upstream TensorRT Edge-LLM, same settings, same device.

Files

Image Size

Model

docker.io/embedl/model-artifacts:nemotron3-nano-4b.huggingface_int4_embedl_tensorrt-edgellm.94b4147be37a3f61

5.74 GiB

Runtime

docker.io/embedl/models:runtimes.tensorrt-edgellm.arm64.2026101815014ee0

0.91 GiB

Variants

Every build of Nemotron 3 Nano 4B on AGX Thor.

Variant Precision Speed Peak memory Energy / token Runtime

huggingface_fp16_tensorrt-edgellm

FP16 31.7 tok/s
Throughput: End-to-end
9.73 GiB 2.03 J
TensorRT Edge-LLM 0.10.1

huggingface_int4_embedl_tensorrt-edgellm

W4A16 40.3 tok/s
Throughput: End-to-end
7.26 GiB 1.62 J
TensorRT Edge-LLM 0.10.1 with Embedl optimizations
runtimes.tensorrt-edgellm.arm64.2026101815014ee0