Skip to content
Log in

google/gemma-4-26B-A4B-it

Embedl optimized · TensorRT Edge-LLM
google/gemma-4-26B-A4B-it architecture graph
google/gemma-4-26B-A4B-it architecture graph
  • huggingface
  • AGX Thor
Labels
  • Image
  • Text
  • VLM
  • Generation
Build
Embedl-optimized
Precision
W4A16
Runtime
TensorRT Edge-LLM 0.10.1 with Embedl optimizations
Context length
4,096 tokens
Run Gemma 4 26B-A4B on AGX Thor

Google's mixture-of-experts Gemma 4: 26B parameters, about 4B active per token, with images.

Methodology

  • Throughput: one user, real conversations from the public OpenAssistant set, 2,048 tokens in, 128 out, greedy, no prompt reuse. Average of 20 requests after 3 warm-up requests. End-to-end is the whole request. Prompt processing is the 2,048 prompt tokens over the time to first token.
  • Power and memory: sampled on the device during those runs. Idle is the device at rest with the model loaded; energy per token is the run's energy over its generated tokens; peak memory is the device's.
  • Accuracy: full test sets, greedy, the same model files and settings as served. GSM8K 1319, IFEval 541, MMLU-Pro 1008 questions.
  • Vision-language model: measured on text only.

Files

Image Size

Model

docker.io/embedl/model-artifacts:gemma4-26b-a4b.huggingface_int4_embedl-mtp_tensorrt-edgellm.d89a196778b10a14

 17.83 GiB

Runtime

docker.io/embedl/models:runtimes.tensorrt-edgellm.arm64.2026101815014ee0

 1.01 GiB