Broadcom Adds New Architecture With Tomahawk Ultra

Source: Broadcom

Tomahawk Ultra is a misnomer. Although the name leverages Tomahawk's brand equity, Tomahawk Ultra represents a new architecture. In fact, when it began development, Broadcom's competitive target was InfiniBand. During development, however, AI scale-up interconnects emerged as a critical component of performance scaling, particularly for large language models (LLMs). Through luck or foresight, Tomahawk Ultra suddenly had a new and fast-growing target market. Now, the leading competitor was NVIDIA's NVLink. Also happening in parallel, Broadcom built a multi-billion-dollar business in custom AI accelerators for hyperscalers, most notably Google.

At the end of April, Broadcom announced its Scale-Up Ethernet (SUE) framework, which it published and contributed to the Open Compute Project (OCP). Appendix A of the framework includes a latency budget, which allocates less than 250ns to the switch. At the time, we saw this as an impossibly low target for existing Ethernet-switch designs: Tomahawk 5's cut-through latency is on the order of 500ns. To get there, the 51.2T Tomahawk Ultra needed a new architecture, which separates it from Tomahawk 5 and the new Tomahawk 6. Yes, Tomahawk Ultra is pin compatible with Tomahawk 5 and shares the same Peregrine 100G serdes design. Its core architecture, however, is all new.

Not only does Tomahawk Ultra cut latency roughly in half, it delivers more than triple the packet rate of Tomahawk 5. Broadcom designed the new architecture for line-rate performance at minimum packet size (64B), which requires 77 billion packets per second (Bpps) of processing. By comparison, Tomahawk 5 tops out at 21.2 Bpps, which requires an average packet size around 300B. To achieve this level of performance, Tomahawk Ultra needs more packet-processing pipelines and a higher clock speed as well as a new traffic manager and buffer-memory design. New features, required by SUE, include link-layer retry (LLR) and credit-based flow control (CBFC). In addition to more pipelines, Broadcom added an in-network collectives (INC) engine, which can offload collective operations such as AllReduce. To keep the new chip about the same size and power as Tomahawk 5, the packet-buffer size had to be significantly reduced.

That last point means Tomahawk Ultra can't serve as a direct replacement for Tomahawk 5, so Broadcom now has four different architectures serving data-center switching. Tomahawk remains the mainstream line for AI scale-out and front-end networks, with Tomahawk 6 first to 200G serdes. Jericho serves AI scale-out networks that lack endpoint intelligence in the form of smart NICs. Similarly, Trident can serve smart-ToR designs that offload network processing from the server. Now the fourth line, Tomahawk Ultra brings unique features for AI scale-up interconnects, positioning Ethernet as a bonafide alternative to dedicated interconnects such as NVLink.