Marvell Teralynx Leapfrogs to 51.2T

About 18 months after acquiring Innovium, Marvell announced its next-generation Teralynx 10 data-center switch chip. The 51.2Tbps design moves the Teralynx architecture to 5nm process technology while also incorporating homegrown 112Gbps PAM4 serdes. It marks the end of the road for the 7nm Teralynx 8 chip, which Innovium sampled but was unable to move to production. Multiple sources confirm that third-party serdes IP was the culprit, and Innovium was only one of several affected customers. The end result for Marvell is that it skipped the 25.6Tbps generation and set its sights on 51.2Tbps. Meanwhile, it has shipped more than 5 million 400GbE ports with the first-generation Teralynx 7, mostly to one hyperscale customer.

Teralynx 10 promises an unsurprising feature set, which actually validates Innovium's design approach. The startup found the right balance of programmability and performance, enabling protocol flexibility and P4 in-band networking telemetry (P4-INT). Innovium also touted large packet buffers, and Teralynx 10 weighs in at greater than 200MB. The new chip is the first to handle 512x100GbE ports, twice the radix of announced competitors. The 112Gbps PAM4 serdes blocks build on Marvell's first 5nm PHY, the 88X93160, which sampled nearly two years ago. The major caveat is that Teralynx 10 has not yet reached customers: samples are due in 2Q23.

An announcement in the absence of samples is a good excuse to revisit the checkered history of competitors attempting to unseat Broadcom. The figure below shows the companies still in the running, which eliminates Intel, MediaTek, and Xsight Labs. Although Cisco has yet to announce its 51.2Tbps switch, it deserves recognition as the only vendor aside from Broadcom to have delivered a 25.6Tbps chip. Nvidia chose to skip the 25.6Tbps generation, likely in an attempt to catch up with Broadcom. It has yet to announce Spectrum-4 availability, however, suggesting the chip's schedule may have slipped.

400/800G Ethernet Switch Roadmap

Unfortunately for Marvell, we expect Broadcom will release Tomahawk5 to production by the time Teralynx 10 samples. This partially explains Marvell's positioning of its chip for 1U designs using 32x1.6T OSFP-XD modules, forgoing the 2U 64x800G systems representing the first 51.2Tbps designs. Another factor is the company's leading position in PAM4 DSP chips used in optical modules, by way of its Inphi acquisition. Although pluggable modules are decoupled from switch chips, components for the former give Marvell access to hyperscale customers, opening the door for Teralynx. With more than four years between Teralynx generations, the company must now execute flawlessly as it works to recoup its $1 billion investment. 

UPDATE March 15, 2023: As expected, Broadcom announced Tomahawk5 production.

Comments

Post a Comment

Popular posts from this blog

AMD Looks to Infinity for AI Interconnects

Image
With the formal launch of the MI300 GPU, AMD revealed new plans for scaling the multi-GPU interconnects vital to AI-training performance. The company's approach relies on a partner ecosystem, which stands in stark contrast with NVIDIA's end-to-end solutions. The plans revolve around AMD's proprietary Infinity Fabric and its underlying XGMI interconnect. Infinity Fabric Adopts Switching As with its prior generation, AMD uses XGMI to connect multiple MI300 GPUs in what it calls a hive. The hive shares a homogeneous memory space formed by the HBM attached to each GPU. In current designs, the GPUs connect directly using XGMI in a mesh or ring topology. Each MI300X GPU has up to seven Infinity Fabric links, each with 16 lanes. The 4th-gen Infinity Fabric supports up to 32Gbps per lane, yielding 128GB/s of bidirectional bandwidth per link. At the MI300 launch, Broadcom announced that its next-generation PCI Express (PCIe) switch chip will add support for XGMI. At last October'
Read more

NVIDIA Networks NVLink

Image
I attended several sessions at last week's Hot Chips, and NVIDIA's NVSwitch talk was a standout. Ashraf Eassa did a great job of covering the talk's contents in an NVIDIA blog, so I will focus on analysis here.  SuperPOD Bids Adieu to InfiniBand From a system-architecture perspective, the biggest change is extending NVLink beyond a single chassis. NVLink Network is a new protocol built on the NVLink4 link layer. It reuses 400G Ethernet cabling to enable passive-copper (DAC), active-copper (AEC), and optical links. The build its DGX H100 SuperPOD, NVIDIA designed a 1U switch system around a pair of NVSwitch chips. The pod ("scalable unit") includes a central rack with 18 NVLink Switch systems, connecting 32 DGX H100 nodes in a two-level fat-tree topology. This pod interconnect yields 460.8Tbps of bisectional bandwidth. NVLink Network replaces InfiniBand (IB) as the first level of interconnect in DGX SuperPODs. The A100 generation pod uses an IB HDR leaf/spine inte
Read more

NVIDIA Reveals DGX GH200 System Architecture

Image
We industry analysts sometimes get out over our skis when trying to project details of new products. Following NVIDIA's DGX GH200 announcement at Computex, we noticed industry press making the same mistake. Rather than correct our previous NVLink Network post, we'll explain here what we've since learned. Our revelation came when NVIDIA published a white paper titled NVIDIA Grace Hopper Superchip Architecture . The figure below from that paper shows the interconnects at the HGX-module level. The GPU (Hopper) side uses NVLink as a coherent interconnect, whereas the CPU (Grace) side uses InfiniBand, in this case connected through a Bluefield-3 DPU. From a networking perspective, the NVLink and InfiniBand domains are independent, that is, there is no bridging between the two protocols. HGX Grace Hopper Superchip System With NVLink Switch (Source: NVIDIA) The new DGX GH200 builds a SuperPOD based on this underlying module-level architecture. You've probably seen the headlin
Read more