Talking Venice with Mahesh Subramony at AMD In-Depth

Talking Venice with Mahesh Subramony at AMD In-Depth

Chips and Cheese (George Cozma)

Hello you fine Internet folks,

I sat down with AMD’s Mahesh Subramony at AMD In-Depth to talk about the SoC design behind the new EPYC Venice processors. We discussed AMD’s move to 2.5D packaging, the decision to split the I/O die in two, the differences between the SP7 and SP8 platforms, and some of the tradeoffs involved in scaling up to 32-core CCDs.

Hope y’all enjoy!



The transcript below has been edited for readability and conciseness.

George Cozma: Hello, you fine Internet folks. We’re here at AMD In-Depth, where we’re talking about Venice and Zen 6. With me, I have Mahesh Subramony. Tell the audience what you do, since your role has changed since the last time we talked.

Mahesh Subramony: That’s right. I’m now with the server architecture team. I’m a corporate vice president of design engineering, and I lead server architecture. I’m excited to be on board and take on some new challenges.

George Cozma: Awesome. Speaking of new challenges, Venice isn’t AMD’s first CPU to use advanced packaging, but it is AMD’s first server CPU with 2.5D packaging. What challenges did that bring, moving from a more traditional substrate-based design to 2.5D packaging?

Mahesh Subramony: What really pulled us in that direction was the amount of data we now have to move around. In the previous generation, I think we were moving a little under 700 GB/s. With that level of bandwidth between the CPU chiplets and the I/O die, standard packaging and the GMI links we had before made sense from a product standpoint.

But now we’re talking about 1.6 TB/s, with each of the CCDs overprovisioned as they drive data into the I/O die. If we had tried to move that much data using conventional packaging, the energy spent just transporting those bits would have been somewhere around 10–15% of our overall SoC power budget, regardless of the picojoules-per-bit numbers you see claimed in papers.

So moving to advanced packaging made a lot of sense to deliver the performance and power efficiency we wanted.

The challenge is that when you attach a CCD to an I/O die with these very short die-to-die connections, the spacing becomes extremely small. You have to match the widths of the dies at the interface and keep that interconnect short-reach. That means you need enough perimeter on each die to accommodate the CCD connections.

In previous generations, the connections between the I/O die and the CCDs could take relatively long routes through the package. We didn’t have the same restrictions on matching the dies’ dimensions or keeping the distance down to a handful of millimeters.

Those new constraints affect the shape of the I/O die. We needed enough perimeter to bring out the I/O and enough beachfront for the CCD connections, while keeping the dies close together. That’s one of the foundational reasons we ended up splitting the I/O into two dies.

With advanced packaging, we get a low-power, high-bandwidth, low-latency link. So even splitting the I/O die in two didn’t burden us too much in terms of power or latency.

George Cozma: And speaking of that, what other design considerations went into splitting the I/O die in two?

Mahesh Subramony: With the previous generation, having a single I/O die made it easier to put everything you needed on one die and fan it out to all the CCD connections.

One thing we did this time was use the same I/O die design twice, with one rotated 180 degrees. Rather than creating two distinct I/O dies, we built one design that could be mirrored across the package.

Of course, there are some one-off IP blocks, like the root of trust or some of the I/O needed for boot, that only need to exist once from a functional standpoint. Those get duplicated when you use two identical dies, so there is some area overhead associated with the split.

The second consideration was making sure we had the right fabric topology. We needed that mirrored topology to work without putting us at a disadvantage when the two dies operate together as a single SoC.

If you look at the two I/O dies, you can think of them as two domains, but in an NPS1 configuration the Venice SoC has to operate as a single domain. Making sure we could deliver the right quality of service, bandwidth, and latency with that mirrored design was a challenge.

As we scale SoCs to larger and larger core counts, though, I think tiled approaches are going to become increasingly advantageous. The challenges they introduce are things we’ll have to solve along the way.

George Cozma: Speaking of scaling an SoC both up and down, SP8 also has two I/O dies. Are they the same dies as on SP7, or are they different? What drove that decision?

Mahesh Subramony: When we looked at SP8, the enterprise market had somewhat different requirements. Cost is always something we keep an eye on when we build these solutions, and the compute requirements for that market were lower than what we were targeting with the larger SP7 design.

The overhead we were carrying in the SP7 I/O dies didn’t translate particularly well to SP8. At the same time, SP8 needed a large number of I/O lanes, whereas SP7 was more focused on compute and memory bandwidth. We saw a fork in the road.

The simpler solution would have been to build one superset I/O die, but that would have imposed additional cost on both the SP7 and SP8 implementations. Instead, we chose to create two different I/O die designs to meet those requirements.

The SP7 I/O die is built for more compute, fewer I/O lanes, and more memory channels. With SP8, you have half as many memory channels and all 128 PCIe lanes available per socket, while targeting fewer cores: up to 96 on the Zen 6 side and 128 on the Zen 6c side.

Those are distinct product targets, so it made sense to have distinct I/O dies to serve them.

George Cozma: And speaking of distinct needs, the 12-core CCD has three links to the I/O die, while the 32-core CCD, which physically has 33 cores, has four. Why was that decision made?

Mahesh Subramony: At AMD, we design very closely around the product requirements. The four SDP ports on the 32-core design follow roughly the same rule of one port for every eight cores. They’re sized to make sure the CCD can draw the bandwidth it needs for the cores to achieve their performance and IPC targets.

The 12-core CCD is different. Those parts are going to be paired with MRDIMMs, and even though the core count per CCD is lower, each CCD has to be able to draw a lot more bandwidth if we’re going to saturate 16 channels of 12,800 MT/s MRDIMMs.

So having three SDP ports on a 12-core CCD can look excessive if you’re just looking at SDP bandwidth per core. But the goal is to let each CCD contribute toward saturating the overall SoC’s memory bandwidth.

George Cozma: Going back to that 32-core CCD, it physically has 33 cores. Why did you add a redundant core, and how did that affect the mesh topology?

Mahesh Subramony: Designing for cost is one of our main principles. It isn’t necessarily the first thing we do when we’re laying out a design, but if we see an opportunity, we take it.

As CCDs get larger, you’re more likely to encounter defects that affect yield. When we were putting this CCD together, we found we could fit in a 33rd core. That also meant adding a 33rd slice of L3 and having the architectural capability to harvest any one of those cores and its associated cache slice while keeping the mesh intact.

We always keep the number of active cores on that CCD at 32. There won’t be a product with 33 active cores. That keeps things like cache associativity and the mesh topology relatively straightforward.

On these large dies built on advanced process nodes, which are quite expensive, having that spare core gives us the flexibility to harvest a defective core and still produce a fully functional 32-core CCD. That helps us get good yields at volume.

George Cozma: Awesome. To round out the SoC design discussion, another major difference between the 12-core and 32-core CCDs is the amount of L3 cache and the number of stops in the mesh. Is there a latency difference between them because of the extra mesh stops and the larger L3?

Mahesh Subramony: I know you’re going to measure this, so I’ll clue you in. With a larger L3, determining whether a request hits or misses in the cache is going to take longer. From a pure latency standpoint, you’ll spend a few extra clocks checking the 128 MB L3 before sending a request farther out.

There are some techniques we can look at to speculatively hide that additional latency, but the larger cache does impose a lookup penalty. Outside of that, the path from the fabric to memory is almost identical.

But I wouldn’t look at memory latency in isolation. What matters is the effective latency a workload experiences. A larger cache will generally give you more hits, particularly when you have more resident threads on that CCD. That’s one of the reasons we grew the CCD and its cache together.

We believe the additional cache capacity more than balances out the extra lookup latency in the workloads that benefit from it. Those additional hits are much faster than going out to memory, so you can make up for the few extra cycles spent on the L3 lookup.

If you’re measuring an otherwise idle CPU with a closed-page load from DRAM, yes, you’ll see a latency difference between the two CCDs. But that difference is largely attributable to the L3 lookup latency.

George Cozma: Looking a little further into the future, the 12-core CCD can support 3D V-Cache, much like the previous eight-core CCDs. Could you potentially see a dense CCD with 3D V-Cache as well, giving it something like 384 MB of L3 on one CCD?

Mahesh Subramony: The way I’d answer that is to look at what we’re doing with the 96-core offering. We have a 12-core CCD with X3D technology, adding another 96 MB of L3. That brings the total to about 144 MB of L3 per CCD.

With the larger, denser CCD, I think about the tradeoffs a little differently. When fewer cores are active, the existing L3 cache is available to those cores, so the effective cache capacity per active core goes up.

Putting another large X3D die on top of the dense design is expensive, and you have to ask how much additional performance it would actually deliver. From an overall power-budget perspective, when you’re running all those cores, one of the first levers you might want to pull is to give them more power and increase their frequency.

The 256-core part is really designed for balanced throughput with a very tight per-core power budget. Is more cache the first place we’d go to improve performance? Probably not. We’d look at ways to get more bandwidth or more frequency into the part first. Adding that much stacked cache is an expensive approach, so it’s not something we’re focused on at the moment.

George Cozma: Thank you. Usually this is where I’d end the interview by asking what your favorite type of cheese is. But since I’ve already asked you that, have you tried any new cheeses lately?

Mahesh Subramony: No, I haven’t. I think I’m getting to the age where I have to watch what I eat. Cheese is a luxury, so I’ll stick to my favorites when I get the chance.

George Cozma: Well, thank you so much, Mahesh.

Mahesh Subramony: Always a pleasure, George.

George Cozma: If you like interviews like this, there are PayPal and Patreon links, and there will also be a transcript on the Chips and Cheese website if you prefer reading interviews. That’s all for this interview. Thank you so much again, and have a good one, folks!

Generated by RSStT. The copyright belongs to the original author.

Source

Report Page