BWR Episode 18: Agentic CPUs and memory advancements

Quick post from San Jose (for OCP Global Summit). In this episode of The Byrne Wheeler Report, Joe and I discuss the shifting architectures of data-center compute, custom memory interfaces, and AI infrastructure. We analyze the mechanics behind NVIDIA’s NV HBM announcement with Amazon, CXL 3.2 memory pooling and fabrics, and why vendor claims about "agentic AI" CPUs require a closer look.   The discussion also examines Velaura as it pivots toward physical AI and robotics, as well as recent MLPerf Inference benchmark results.


00:00 Introduction and episode overview

01:06 NVIDIA's NV-HBM and custom memory interface

03:16 Moving memory controllers to base die and implications

06:31 Astera's second-generation CXL memory controllers

09:17 Are CPUs made for agentic AI just hype?

13:55 Is local AI worthwhile?

15:06 What is the CPU market TAM?

19:17 Having received a Series A, what is Velaura doing?

25:19 MLPerf benchmarks and AMD's performance

31:25 What's the Qualcomm-Huawei agreement about?

32:55 What's the Qualcomm-Amazon agreement about?

38:48 Arm's new CPU cores and future prospects

41:27 Closing remarks and upcoming events


Comments

Post a Comment

Popular posts from this blog

Decoding Nvidia's Rubin Networking Math

Image
At GTC DC last month, Jensen Huang showed off components of the Vera Rubin NVL144 platform. First, here's the latest roadmap, which now includes BlueField-4 and BlueField-5. For more on that, see BWR Episode 4 .  Source: Nvidia Below is the Vera Rubin compute tray, which includes four Rubin GPUs. By GPU, we mean package not die. Note that the Blackwell NVL72 and Rubin NVL144 both have 72 GPU packages, but the NVL144 moniker denotes Nvidia's new math counting die. The company didn't rename the Blackwell configuration, even though that GPU also has two die. Each compute tray has two Vera CPUs, which are 88-core Arm processors. Two GPUs connect with one CPU using NVLink-C2C, a coherent variant of NVLink. Although the roadmap above shows CX9 as 1600G, each ConnectX-9 is actually 800Gbps, requiring eight chips to deliver the aggregate 800GB/s quoted for the tray. That means each GPU has a pair of 800G Ethernet/InfiniBand NICs for scale-out networking. Finally, a single BlueField...
Read more

CXL Chip Market Poised for Rapid Growth

Image
It's not often that we see a new interconnect come along that's a sure thing. By piggybacking on the PCI Express physical layer, however, CXL has become one of those rare birds. As is always the case with new technologies, it will take time for a multi-vendor ecosystem to mature. CXL offers many incremental steps along the architectural-evolution path, allowing the technology to ramp quickly while offering future iterations that enable truly composable systems. It All Starts with Server CPUs Although not officially launched, Intel's Sapphire Rapids is already shipping to early customers. Development platforms are also in partners' hands, enabling validation and testing of CXL components. AMD's Genoa is also about to launch with CXL support. The caveat for both vendors is that these first CPUs support only CXL 1.1, which lacks important features incorporated in the CXL 2.0 specification. Both versions ride atop PCIe Gen5, however, so the physical layer needn't ch...
Read more

AMD Looks to Infinity for AI Interconnects

Image
With the formal launch of the MI300 GPU, AMD revealed new plans for scaling the multi-GPU interconnects vital to AI-training performance. The company's approach relies on a partner ecosystem, which stands in stark contrast with NVIDIA's end-to-end solutions. The plans revolve around AMD's proprietary Infinity Fabric and its underlying XGMI interconnect. Infinity Fabric Adopts Switching As with its prior generation, AMD uses XGMI to connect multiple MI300 GPUs in what it calls a hive. The hive shares a homogeneous memory space formed by the HBM attached to each GPU. In current designs, the GPUs connect directly using XGMI in a mesh or ring topology. Each MI300X GPU has up to seven Infinity Fabric links, each with 16 lanes. The 4th-gen Infinity Fabric supports up to 32Gbps per lane, yielding 128GB/s of bidirectional bandwidth per link. At the MI300 launch, Broadcom announced that its next-generation PCI Express (PCIe) switch chip will add support for XGMI. At last October...
Read more