NVIDIA’s Vera Whitepaper Has a Thread Loose

Hello you fine Internet folks,

NVIDIA has published a 45-page whitepaper explaining Vera, its first server CPU built around the company’s own Olympus core. On paper, Vera is a fascinating chip with an 88-core monolithic compute die, Olympus being a 10-wide Arm v9.2 core that has value prediction, a graph prefetcher, 2 MB of private L2 per core, 164 MB of shared last-level cache, and eight LPDDR5X memory interfaces promising 1.2 TB/s.

Unfortunately, NVIDIA also spends a good part of the paper trying to turn those interesting design choices into a morality play about x86. Traditional simultaneous multithreading is drawn as time-slicing, a configurable NUMA topology is presented as an unavoidable 32-node maze, four SPEC components become “agentic benchmarks,” undefined performance-counter ratios are promoted as causal proof, and an unlabeled pictogram becomes a 1.8x reinforcement-learning result.

The frustrating part is that Vera does not need this help, with early independent testing suggesting Olympus is genuinely formidable. The whitepaper’s strongest case is the hardware; its weakest case is the story wrapped around it, so let’s pull that story apart.

Olympus Deserves Better Than This Marketing

Before getting out the cheese grater, let’s talk about the good stuff. Olympus is a very wide out-of-order Arm core.

Its front end can decode ten instructions per cycle and handle up to two taken branches per cycle. NVIDIA describes a neural branch predictor, value prediction, memory renaming, a large instruction window, six 128-bit SVE pipes, four load pipes, two store pipes, a 96 KB L1 data cache, and roughly 10-cycle access to a 2 MB private L2. Eighty-eight of those cores sit behind a 3.4 TB/s coherency fabric and a distributed 164 MB system-level cache.

Looking closer at the core, the value prediction is one of the more unique additions that Olympus has. This has been a research area for a long time and what value prediction allows Olympus to do is if the core correctly predicts a result, dependent instructions can keep moving instead of piling up behind a long-latency operation. Researchers have discovered that Apple uses value prediction in their cores and AMD talked about how in Family 17h (Zen 1 and 2) they could predict the value of some floating point instructions. However, AMD’s Family 17h implementation was quite limited, while Olympus appears to have a broader value-prediction implementation closer to Apple’s.

However, the graph prefetcher is not unique to NVIDIA. Intel has a similar mechanism called Data-Dependent Prefetcher that has been in shipping silicon since at least 2022. Intel’s newest datacenter CPU, Granite Rapids, also has an Array of Pointers prefetcher which “treats the data prefetched for a constant stride load as a pointer and may issue prefetch requests to the memory addresses corresponding to the pointer’s value.” This is fundamentally the same producer-consumer idea that NVIDIA describes for its graph prefetcher. Intel’s implementation is fairly constrained, so NVIDIA’s implementation may be able to deal with more complex chains than Intel’s implementation. So while Vera’s Graph Prefetcher may be an implementation that can deal with more workloads, producer-consumer prefetching is not a new idea.

Nor is a “neural branch predictor” a new idea. Back in 2012, AMD implemented a perceptron branch predictor in the Piledriver microarchitecture and continued to use a perceptron-based branch predictor in Zen 1. However, starting with Zen 2, AMD used a perceptron BPU only for its initial direction prediction, with a TAGE predictor overriding it because it delivered a 30% reduction in mispredictions. With Zen 5, AMD has likely fully committed to TAGE predictors, if it had not already done so with Zen 3 or Zen 4.

Moving to the SoC side, with how beefy the Olympus core is, NVIDIA has given Vera an equally beefy memory subsystem. Vera pairs eight SOCAMM2 LPDDR5X modules with up to 1.5 TB of capacity and 1.2 TB/s of bandwidth. NVIDIA claims the populated memory subsystem only consumes approximately 50 watts. A conventional EPYC or Xeon platform can offer higher-capacity DIMMs which are easier to replace, but it pays for that flexibility in board area and power.

Most importantly, we have more than NVIDIA’s results to look at. In May, Michael Larabel at Phoronix ran an early Vera system against current Arm and x86 servers. Across the NVIDIA-permitted test set, Vera’s geomean was 10% above a 5 GHz EPYC 9575F, 1.55x a Xeon 6980P, and 1.63x Grace which makes Vera the most performant Arm server CPU we have seen in public testing. There are major caveats with the testing, such as NVIDIA choosing the permitted workload scope and not allowing frequency or power monitoring. The system that Phoronix tested was pre-production and the test window was one day which puts a fairly hard limit on what they could test irrespective of the limits NVIDIA placed. This means that broader coverage will have to wait until Vera can be found in the wild rather than just in NVIDIA’s labs.

Still, the result is strong enough that we can reject the explanation that the charts in NVIDIA’s whitepaper are all fantasy. Olympus appears to be a fast CPU core, so now we can ask whether the whitepaper proves what NVIDIA says it proves.

Spatial Multithreading Is Still SMT

Here is the first major technical error in the document.

Figure 5 contrasts “Traditional SMT (x86)” with NVIDIA’s Spatial Multithreading. The x86 side depicts the branch predictor, decode, execution, load/store, and memory stages alternating between two threads. The caption says Vera avoids “opportunistic time-sharing” by partitioning resources across its two hardware threads.

NVIDIA’s diagram gives a misleading impression of how SMT is usually implemented, both on x86-64 and other ISAs. SMT implementations share various stages in the execution pipeline by either selecting a thread to service every cycle, or by behaving in a thread-agnostic manner. Fetch, decode, and allocate typically service threads on a per-cycle basis, while the execute and memory access stages are thread agnostic and can service micro-ops from both threads in the same cycle. Stages that threads arbitrate for do not leave resources unused when both threads can be fed, as NVIDIA’s diagram suggests. Static partitioning and per-cycle selection would provide the same average throughput to both threads in the absence of per-thread stalls. If there are stalls, per-cycle selection can give otherwise unused throughput to the un-stalled thread.

The same idea applies to thread agnostic stages like execute and cache access. Each thread is permitted to utilize as many execution units or cache ports as it can feed. In contrast, statically partitioning resources as NVIDIA suggests could lead to one thread being compute bound and unable to use half of the core’s execution resources because they’re reserved for the other thread.

Text in NVIDIA’s paper emphasizes “determinism, isolation, and quality of service” as advantages for NVIDIA’s Spatial Multithreading approach. Performance is conspicuously not called out. QoS may be a more important consideration than throughput for NVIDIA’s target market, and Spatial Multithreading may not be a bad design point. But NVIDIA’s figure makes it look like vertical space represents time, and gives a misleading impression that Spatial Multithreading is meant to give larger performance gains than traditional SMT.

By reducing resource interference between threads, Spatial Multithreading improves determinism, isolation, and quality of service compared to traditional SMT approaches. The result is a CPU architecture that can run large numbers of concurrent agent tasks while maintaining more consistent latency and throughput. - NVIDIA’s Vera whitepaper

Vera’s actual SMT performance is unknown of course, and a lot of variables go into SMT gains besides partitioning strategies at fetch, decode, execute, and memory access. Out-of-order resources like the reorder buffer, register files, and memory ordering queues can be duplicated, statically partitioned, watermarked, or competitively shared. Partitioned structures were split between the two logical processors in multi-threaded mode and recombined for one thread in single-thread mode, which was documented in 2002. Various SMT implementations use different strategies for each structure, and those choices can have significant implications for SMT gains.

Also something to note is that it apparently takes 10,000 cycles for an Olympus core to transition back to the single-thread mode once the sibling thread on that core is done. This means that software will have to be very aware about launching a second thread on an Olympus core due to the penalties incurred not only from the partitioning scheme but also from the delay of swapping back to a single thread.

It’ll be interesting to see what strategy Vera uses to partition its out-of-order resources, and how its SMT performance compares to that of other modern cores. NVIDIA’s whitepaper gives no information on that. What it does do is present a misleading diagram that suggests traditional SMT is prone to leaving resources unused, when it may actually be better at keeping the core fed than NVIDIA’s Spatial Multithreading.

The 32-NUMA-Node Straw Man

NVIDIA next tells us that a large two-socket x86 system can expose “as many as 32 NUMA domains,” while Vera presents one per socket. The number is not invented. On a many-chiplet EPYC system, an administrator can expose cache-local regions as separate NUMA nodes. If you turn every locality knob toward maximum granularity, the node count gets large.

What NVIDIA leaves out is that this is configurable with AMD’s tuning guide listing NPS4, NPS2, NPS1, and even NPS0 modes. The optional “LLC as NUMA” setting can expose each last-level-cache domain separately. So “32 NUMA nodes” is not the inevitable user experience of a chiplet CPU, it is one end of a locality-control spectrum. NVIDIA presents an optional high-granularity configuration as though it were an unavoidable reality of x86 systems.

Vera’s one domain per socket simplifies scheduling and memory placement, while multiple domains let tuned software exploit physical locality. Vera chooses the simpler presentation, and NVIDIA is free to argue that this better matches its intended software stack. But an OS-visible NUMA node is an abstraction, not a wormhole. Vera still has 88 cores, distributed cache and home nodes, memory controllers around a large die, and a packet-switched coherency fabric. A flat software topology can make those distances around the large monolithic compute die more consistent, but it cannot make them nonexistent.

The paper’s core-to-core heatmaps would be a good place to quantify that advantage. Instead, NVIDIA provides colored squares with no core identities, no minimum/median/maximum table, no distribution, and no measurement procedure. “Up to 50% lower” captures NVIDIA’s best result, not Vera’s typical behavior.

One NUMA node per socket is genuinely simpler, but the whitepaper compares it against an optional 32-domain x86 configuration and presents that edge case as the baseline. The counterpoint here is that Intel has a Mesh NoC just like Vera has. The difference really between these two setups is that the clustered setup of EPYC has high latency between clusters but within a cluster the latency is low, whereas Vera and Xeon Mesh setup has uniformly average latency; the different configurations are just engineering tradeoffs.

Turning SPEC into “Agentic AI”

The benchmark section is where the whitepaper, ostensibly about a CPU, starts wearing an AI conference badge it found on the floor.

NVIDIA selects four SPEC CPU 2026 integer workloads, CPython, GCC, LLVM, and Cppcheck, and calls them “agentic benchmarks.” SPEC itself describes them as a Python interpreter, two optimizing compilers, and a C/C++ static analyzer. Those are legitimate CPU programs. They stress large instruction footprints, branch-heavy code, allocation, and dependency chains. Agents can absolutely invoke programs like them.

But they are not agents: no model is serving tokens, no agent runtime is choosing tools. No sandbox is starting, blocking on I/O, retrieving context, evaluating an answer, or feeding observations back into a policy. These workloads may be useful proxies for the code-heavy portions of an agentic pipeline. Calling them “agentic benchmarks,” however, turns that partial overlap into a claim that they represent the complete end-to-end workload.

The paper does correctly label the SPEC results as estimates, because the Vera reference hardware was not generally available at the time of the run. Figure 15 shows a 1.7x to 1.8x advantage for the four selected components, normalized per physical core under a fully loaded two-socket system. Flip to the configuration pages and the full estimated SPECrate 2026 Integer Base totals are 925 for two Vera sockets and 898 for two EPYC 9755 sockets which is a 3.0% system-throughput advantage.

Both numbers can be true. Vera uses 176 physical cores across two sockets, while the EPYC system uses 256. Divide each score by physical-core count and Vera is about 50% faster per core across the full integer-rate suite with the selected tests reaching 70 to 80%.

There is another terminology collision. NVIDIA calls Figure 19 “single thread IPC” while describing a fully loaded system.

The published configuration runs 352 copies on 176 Vera cores and 512 copies on 256 EPYC cores, two copies per physical core. Maybe NVIDIA sampled one logical thread while its sibling was active, maybe it aggregated counters and divided, the paper does not say. The SPEC results are useful, and Vera’s per-core performance is genuinely strong. However, framing those tests as agentic workloads and emphasizing normalized figures makes the advantage appear broader than the disclosed results justify.

IPC Without Instructions

NVIDIA attributes Olympus’s reported IPC lead to four counter groups. Depending on the selected workload, Vera supposedly achieves up to 2.3x more branch predictions per cycle, 3.5x more taken branches per cycle, 2.4x more instruction-fetch operations per cycle, and 4.3x more backend operations per cycle.

That sounds technically specific, but it is impossible to audit without the PMU event names and definitions, raw counts, sampling intervals, clock frequencies, etc. Not to mention that an Arm instruction is not the same unit of work as an x86 instruction. An internal backend operation is even less portable: one microarchitecture may split an instruction into several micro-operations while another keeps it fused.

Cross-ISA IPC can still be informative when paired with retired-work counts, clock frequencies, and code analysis. It cannot stand alone as a performance metric. Two binaries can complete the same task in the same amount of time while reporting very different IPC. For one may simply retire more instructions that are doing less work per instruction, then you have to factor in clock frequency which could be wildly different. IPC describes the behavior of the core running a piece of code, not a universal measure of useful work.

Looking at the branch predictor results, more branch predictions per cycle could indicate a capable predictor or it could also mean the Arm binary contains more branches, the benchmark moves through code faster, or NVIDIA’s event counts speculative predictions that the EPYC event does not. Higher backend operations per cycle may correlate with performance while telling us little about which feature caused it. To isolate value prediction, graph prefetching, or the neural predictor, we need on/off experiments or at least event definitions and miss rate deltas. While the IPC advantage of Vera over Turin may be real, the charts in NVIDIA’s whitepaper don’t provide enough granularity of the results to show it.

Vera’s Memory Advantage Is Real and Misattributed

The memory section contains NVIDIA’s strongest result and one of its weakest conclusions.

Against the dual-socket EPYC 9755 system in the paper, Vera reaches roughly 1.1 TB/s in NVIDIA’s loaded-latency plot while Turin levels off near 400 GB/s. Vera also shows 12.7 GB/s per core versus 3.1 GB/s per core for Turin. Those results do not line up with our testing of Turin CPUs.

In our testing of Turin, we were able to get approximately 570 GB/s out of Turin with the 12 channel DDR5-6400 memory subsystem. This is in direct contradiction to NVIDIA’s results which top out at ~400 GB/s of memory bandwidth. This does also throw the per-core memory bandwidth numbers into dispute with the per-core bandwidth increasing to ~4.5GB/s for the EPYC 9755.

Vera still comes out ahead, but our Turin result substantially changes the size of that advantage. Comparing Vera’s roughly 1.1 TB/s against the 570 GB/s we measured gives NVIDIA a 1.9× bandwidth lead rather than the nearly 3× lead shown in the whitepaper. Revising the EPYC 9755’s per-core result from 3.1 GB/s to approximately 4.5 GB/s similarly reduces Vera’s advantage from 4.1× to roughly 2.8×. And if we look at the SKU that AMD actually puts forward as the SKU for AI head nodes, the EPYC 9575F, then the per-core result becomes 12.7 GB/s vs the 9575F’s ~9 GB/s which is about 40% improvement for Vera. Those are still good numbers for Vera, but they tell a considerably less dramatic story.

Looking at the theoretical figures, AMD lists Turin’s limit at 614 GB/s from its 12 DDR5-6400 channels. Our 570 GB/s result reaches approximately 93% of that theoretical limit. Vera’s eight LPDDR5X-9600 interfaces provide 1.2 TB/s, while NVIDIA’s measured 1.1 TB/s reaches roughly 92% of that figure. In other words, both processors convert a remarkably similar percentage of their theoretical memory bandwidth into sustained bandwidth. Vera wins because it has approximately twice the peak bandwidth of one Turin socket and fewer cores competing for it, not because Turin is unusually poor at using its available memory bandwidth.

Despite that, the whitepaper repeatedly credits Vera’s monolithic compute die while contrasting it with “traditional chiplet-based CPUs.” A monolithic die may reduce fabric traversal and improve loaded latency, but our Turin result directly weakens that explanation for the bandwidth difference. A chiplet-based EPYC 9755 reaching approximately 93% of its theoretical limit is clearly not being held back by its chiplet topology in this case. The bulk of Vera’s bandwidth advantage comes from the memory interfaces attached to the processor.

The comparison also aged almost immediately. NVIDIA published its technical blog and whitepaper on July 21, while AMD launched 6th Gen EPYC two days later. The 96-core EPYC 9686F, which is much closer to Vera’s 88-core count, provides 16 memory channels supporting DDR5-8000 or MRDIMM-12800 for 1,024 or 1,638 GB/s per socket with the top MRDIMM speed giving Venice more theoretical bandwidth than Vera both in total memory bandwidth and per-core memory bandwidth depending on what SKU you look at.

What the new EPYC specifications and our Turin testing demonstrate is narrower than NVIDIA’s claim of “3× more memory bandwidth than the latest x86 CPU” depends on a Turin result that does not represent the bandwidth we could extract from the same processor generation. Against our result, Vera delivers approximately 1.9× the total bandwidth and 2.8× the bandwidth per core, that remains an impressive platform result but it is not evidence of a bandwidth advantage for monolithic Arm processors over chiplet-based x86 CPUs.

The Graph and RL Charts Need Data

NVIDIA reports a 2.6x PageRank advantage over EPYC 9755 and shows Vera scaling almost linearly to 32 cores while EPYC flattens to just a 10X performance increase at 32 cores. NVIDIA attributes this improvement over Turin as down to the monolithic compute die with their high-bandwidth Scalable Coherent Fabric, the 1.2 TB/s of memory bandwidth that Vera has, and the graph prefetcher inside the Olympus core.

However, while the paper links GAP Benchmark Suite, it omits what variables NVIDIA used which are important factors on how this test runs. The scaling plot stops at 32 cores even though the machines have 88 and 128 cores per socket. While the 2.6x result is interesting, without the variables that NVIDIA used, the result is likely irreproducible.

For the ClickHouse testing, NVIDIA links directly to Phoronix’s result where Vera led the tested processors across three passes over a 100-million-row dataset. The whitepaper’s 1.2x chart is still selective due to not using the 9575F results, but an outside tester produced the underlying result with a recognizable workload.

Then we reach Figure 24, “Vera drives 1.8x for RL training,” with the figure being a row of little completed-task squares. There is no model, environment, CPU/GPU allocation, framework, batch size, power measurement, repetition count, or error bar. We do not even know whether the squares represent samples, steps, or some random layout of tiles at NVIDIA HQ.

This is not a bad benchmark, it simply is not a benchmark at all.

The surrounding text explains why a faster CPU could improve reinforcement-learning rollouts with faster environment steps and reward computation can feed accelerators more quickly however Figure 24 does not even pretend to measure it.

A Good CPU Does Not Need a Bad Argument

After 45 pages, my position on Vera is more positive than my position on the Vera whitepaper.

Olympus looks like a serious core with a 10-wide fixed-length decoder, large private caches, value prediction, aggressive branch handling, graph-aware prefetching, and a monolithic 88-core die all being choices pointing to a very high-performance CPU core. The memory setup is no slouch either with the LPDDR5X subsystem delivering up to 1.2 TB/s of memory bandwidth that memory bandwidth-hungry server workloads will love, and early independent benchmarks say the silicon can cash at least some of the checks that the whitepaper writes.

However, the paper’s competitive argument is much shakier, with it mischaracterizing x86 SMT, turning an optional NUMA configuration into a default burden, relabelling standard CPU tests as agentic workloads, hiding a 3% two-socket rate lead behind 1.8x per-core bars, comparing undefined cross-ISA counters, attributing a memory-interface win to monolithic virtue, and presenting an illustration as performance data.

None of that makes Vera slow, it simply makes NVIDIA’s proof smaller than NVIDIA Marketing’s prose.

The next round of Vera testing should be straightforward. Give independent reviewers unrestricted production hardware so that we can publish frequency, package power, and wall power figures along with testing Spatial Multithreading on/off results and of course running whatever benchmark/workload we wish on Vera. If Vera is as good as its architecture suggests, those tests will be much more persuasive than drawing x86 SMT as a tiny two-lane traffic light. The way things stand, NVIDIA’s marketing risks tarnishing Vera. NVIDIA has built enough CPU here, it can stop borrowing performance from the marketing pipeline.

Generated by RSStT. The copyright belongs to the original author.

Source