Vera Rubin NVL72 vs GB300: 30x Agentic Work Per Watt

Free Instant SEO Audit

Vera Rubin NVL72 server emitting blue data streams toward robotic agent icons

By NewSunSEO September 24, 2026

Key Findings at a Glance

Vera Rubin NVL72 racks can run up to 30 times more agentic coding work for each megawatt of power consumed than the prior generation, while cutting the cost of every million tokens by as much as 45 times. The numbers come from SemiAnalysis AgentX, a workload built from recorded real-world agentic coding sessions that preserve actual context growth, tool calls, and sub-agent spawning. For power-constrained AI factories, the result translates directly into more agentic work for the same energy footprint and meaningfully thinner margins on token-heavy production runs.

Why Agentic Workloads Stress Infrastructure Differently

A simple chat exchange or a document summary typically uses input and output sequences between one thousand and eight thousand tokens. Agentic sessions look fundamentally different. Context accumulates across every step, and long-running agents can drive input lengths into the hundreds of thousands of tokens while output length swings widely between requests. The agent only stops reasoning when the task is done, which keeps token demand climbing across the entire workflow.

Industry data on routing traffic bears this out: agentic AI workloads consume roughly 15 times more tokens than a simple chat request, because each step’s accumulated tokens feed the next, and long-context handling becomes central to performance rather than a corner case. As agents move from pilots into production across software development, customer service, and deep research, the infrastructure behind them has to absorb that token demand efficiently or the economics collapse.

What the SemiAnalysis AgentX Benchmark Measures

SemiAnalysis AgentX is built from recorded real-world agentic coding trajectories. It preserves the actual context growth, tool calls, and sub-agent spawning that real sessions produce, instead of distilling the workload down to a single inference request. That matters because measuring only a single inference understates how agents behave in production, where long contexts and parallel tool use are the norm rather than the exception.

The platform covers multiple agentic models, including Kimi K3, MiniMax M3, GLM5.3, Qwen3.5, and DeepSeek V4 Pro. Performance comparisons that follow reflect runs on this workload rather than synthetic chat benchmarks.

From Blackwell to Vera Rubin: The Efficiency Curve

On the SemiAnalysis AgentX workload, the NVIDIA Blackwell platform already posts leading numbers across the agentic model set. GB300 NVL72 delivers up to 15 times better throughput per megawatt than the Hopper architecture on the DeepSeek V4 Pro model, giving customers a higher-performance foundation for agentic workloads. That gap reflects the advantage of a larger scale-up GPU domain and tightly codesigned software in delivering better inference efficiency.

Vera Rubin extends the curve further. It lifts performance across the entire Pareto frontier to deliver as much as 30 times higher throughput per megawatt than GB300 NVL72 on the DeepSeek V4 Pro model. Throughput per megawatt also feeds directly into the cost of every token produced: at up to 45 times lower cost per million tokens than GB300 NVL72, Vera Rubin NVL72 can run agents continuously at scale across the full breadth of customer workloads.

For power-constrained AI factories, throughput per megawatt determines revenue, and cost per million tokens sets the profit margin on that revenue. Both numbers move in the customer’s favor on Vera Rubin.

Techniques That Drive the Gains

The throughput jump comes from a stack of inference optimizations that are especially important for agentic AI. Vera Rubin NVL72 enables all of them through codesign across hardware and software.

Disaggregated serving separates context processing (prefill) from response generation (decode), so each scales on its own.
Rate matching synchronizes prefill and decode GPU speeds to keep tokens flowing without stalls.
Large-scale expert parallelism distributes mixture-of-experts sub-networks across the scale-up GPU domain.
Distributed KV-caching extends memory across the same domain, while KV-cache offloading tiers less-active context to host and storage so previously processed context stays accessible without recomputation.
KV-aware routing directs incoming requests to the GPUs that already hold the relevant cached context, cutting redundant work across long sessions.
Fused CUDA kernels such as MegaMoE combine computation and inter-GPU communication into a single execution pass, keeping GPUs active instead of waiting on data.
Enhanced fifth-generation Tensor Cores and the third-generation Transformer Engine accelerate both prefill and decode stages.
NVFP4 quantization compresses model weights to 4-bit precision, shrinking memory footprint and lifting throughput without sacrificing output quality.
NVLink and sixth-generation NVLink Switches deliver 10 times higher packet rates and 3 times lower latency than off-the-shelf Ethernet alternatives, providing the high-bandwidth, low-latency fabric that large-scale expert parallelism and distributed KV-caching require.
CUDA kernels, the TensorRT LLM inference runtime, and the Dynamo serving framework are codesigned with the hardware to enable these optimizations end to end.

These early Vera Rubin NVL72 results on SemiAnalysis AgentX do not yet reflect the Vera CPU’s contribution to tool calling, which means further gains are likely as that piece lands.

Power Management at AI Factory Scale

Beyond raw throughput, NVIDIA DSX MaxLPS manages power across GPU, rack, and workload levels so that up to 40 percent more GPUs can be provisioned within the same megawatt budget. That further pushes throughput per megawatt at AI factory scale, on top of the efficiency gains already baked into Vera Rubin itself.

The Full Seven-Chip Platform

While the throughput and token-cost numbers reflect current Vera Rubin NVL72 performance, the full platform is a seven-chip architecture purpose-built for AI factories deploying agents at scale. The set includes the Vera CPU, Groq 3 LPU, NVLink 6 Switch, BlueField-4 DPU, Spectrum-6 SPX, and ConnectX-9 SuperNIC, all designed to operate together. Vera Rubin is in full production and is scaling across the partner ecosystem.

FAQ

What is the SemiAnalysis AgentX workload?

It is a benchmark built from recorded real-world agentic coding sessions that preserve actual context growth, tool calls, and sub-agent spawning, so it reflects how agents behave in production rather than distilling work down to a single inference request.

How much more efficient is Vera Rubin NVL72 than GB300 NVL72 on agentic workloads?

On the SemiAnalysis AgentX workload, Vera Rubin NVL72 delivers up to 30 times higher throughput per megawatt and up to 45 times lower cost per million tokens than GB300 NVL72 on the DeepSeek V4 Pro model.

Which techniques drive the throughput gains on Vera Rubin NVL72?

The gains come from a stack of codesigned optimizations including disaggregated serving, rate matching, large-scale expert parallelism, distributed KV-caching with offloading, KV-aware routing, fused CUDA kernels such as MegaMoE, fifth-generation Tensor Cores, the third-generation Transformer Engine, NVFP4 quantization, and sixth-generation NVLink Switches, all running on top of CUDA, TensorRT LLM, and Dynamo.