Alpamayo 2 Edge

Methodology

• Latency: three of this page's PhysicalAI-AV drives, the first 30 decisions of each (one per 0.1 s of driving), sent one after another to the server on the H200 as a streaming client on the vehicle sends them: KServe v2 binary data, each new camera frame as raw RGB, an empty element for each frame the server already holds (about 6 of 22 frames new per decision); the first 3 decisions of each drive are excluded. A decision is the server's whole call from the request's frames to the trajectory: on median 116.6 ms in the model, 18 ms the frames to the GPU and NVIDIA's preprocessing there, 1.4 ms reading the request. Decisions per second is back to back: 1000 / the client's median round trip (153.9 ms).
• Memory: the server's GPU memory (nvidia-smi, every 50 ms) at its highest during 45 live decisions on real drive frames; the model is its checkpoint's files.