Skip to content
Search models or paste a model link. Press Enter to search.

Qwen3.6 35B-A3B

Qwen/Qwen3.6-35B-A3B
Left and right arrow keys switch the vendor.
Embedl-optimized
Qwen/Qwen3.6-35B-A3B architecture graph
Qwen/Qwen3.6-35B-A3B architecture graph
  • huggingface
  • Orin NX 16GB
Labels
  • Text
  • LLM
  • Generation
Build
Embedl-optimized
Precision
W4A16
Runtime
TensorRT Edge-LLM 0.10.1 with Embedl optimizations
Context length
4,096 tokens
Run Qwen3.6 35B-A3B on Orin NX 16GB

Alibaba's mixture-of-experts Qwen3.6: 35B parameters, about 3B active per token, for coding and agent work. Text in, text out.

Methodology

  • Throughput: one user, real text from the public OpenAssistant set (no chat template), greedy, no prompt reuse. The headline decode is 3,072 tokens in and 50 out, 10 requests after 2 warm-up requests. End-to-end, prompt processing and the second decode are 2,048 tokens in and 128 out, 20 requests after 3 warm-up requests; prompt processing is the prompt tokens over the time to first token.
  • Accuracy: greedy, the build as served (its own embedl-serve), the answer read from its last "Answer:" line, at most 2,048 tokens; an answer cut off at that limit counts as wrong. GSM8K: the first 100 questions of the test split. MMLU-Pro: 56 questions of the test split, 4 per subject (of 72 drawn per subject with seed 0).
  • Power and memory: sampled on the device during the 2,048-in runs. Idle is the device at rest with the model loaded; energy per token is the run's energy over its generated tokens; peak memory is the model process's own at its highest: its GPU buffers, its own allocations and its share of the model files it holds in memory.
  • Size: the model image's files: engines, tokenizer, serve script and licences.
  • Baseline: none on this board: the 16-bit model as released (BF16, 67.0 GiB of weights) does not fit in the Orin NX's 16 GB. The headline is those 67.0 GiB over this build's 10.6 GiB peak memory.

Files

Image Size

Model

docker.io/embedl/models:hf.qwen.qwen3-6-35b-a3b.int4-embedl-flashhead-moecache.orin-nx-16gb.2026111612031354@sha256:1d29505143a32661bbaa9225cfb8942bd80848af9b1840ab7d9a470bd0073801

18.27 GiB

Runtime

docker.io/embedl/models:runtimes.tensorrt-edgellm.arm64.2026111414583a48

—