Skip to content
Search models or paste a model link. Press Enter to search.

Cosmos3 Edge

nvidia/Cosmos3-Edge
Left and right arrow keys switch the vendor.
Embedl-optimized
Cosmos3 Edge architecture graph
Cosmos3 Edge architecture graph
  • nvidia/Cosmos3-Edge
  • INT4
  • AGX Thor
Labels
  • Image
  • Text
  • VLM
  • Generation
Build
Embedl-optimized
Precision
INT4 weights
Runtime
TensorRT Edge-LLM 0.10.1 with Embedl optimizations
Run Cosmos3 Edge on AGX Thor

NVIDIA's Cosmos3 Edge reasoner: the 2.4B text-and-image-in, text-out part of NVIDIA's open world model for on-device physical AI.

Methodology

  • Throughput: one user, batch 1, greedy, a three-sentence question in, 108 tokens out, with the build's speculative decoding. End-to-end is the whole request (17 ms to the first token); decode excludes prefill. With speculative decoding, throughput depends on the prompt. Text prompts only.
  • Accuracy: greedy, the same INT4 model: MMMU-Pro, AI2D, MMBench and DocVQA 1,500 questions each, RealWorldQA all 765.
  • Memory: peak GPU memory during the throughput run.
  • Size: the TensorRT engines.
  • Baseline TensorRT Edge-LLM: the FP16 model on upstream TensorRT Edge-LLM with plain decode; accuracy only, same questions.