With AI models and workloads of late becoming ever more onerous, data centres are finding that the problem is not merely a matter of throwing more computing power at it. The networks that tie together thousands of GPUs have to be up to the task as well, all without an undue increase in cost, power draw or infrastructure headaches.
To put an end to the need for a wholesale network rebuild as they scale, NVIDIA and Cisco have put forward USFP (Ultra-Dense Small Form Factor Pluggable). The new optical networking standard was unveiled at the Open Compute Project Global Summit in Santa Clara and is meant to get more out of current data centre infrastructure while boosting bandwidth.
USFP Doubles Bandwidth per Lane to 400 Gb/s
The specification allows for 400 Gb/s-per-lane electrical signals, which is double what was possible with the old 200 Gb/s limit. In the words of the announcement, one can expect four times the bandwidth density of conventional pluggable optics and some 1.3 times the efficiency in terms of power.
Initial transceivers will pack 16 full-duplex lanes into an OSFP form factor for a combined 6.4 Tb/s; later iterations may go as high as 32 lanes and 12.8 Tb/s. For those running AI infrastructure, the ability to move more data in a smaller physical space is key when it comes to GPU training and inference, where a bottleneck in the network can hold back the entire system.
An Open Standard Designed to Prevent Vendor Lock-In
NVIDIA and Cisco are seeing to it that USFP is an open, community-driven Multi-Source Agreement (MSA) rather than a proprietary solution. TE Connectivity and Amphenol are already on board as partners with connectivity components, and others are likely to follow. It is an approach meant to foster interoperability from one generation of network to the next, giving builders the option to mix and match suppliers instead of being tied to a single vendor’s wares.
NVIDIA Expands Its AI Infrastructure Portfolio
There were other moves made by NVIDIA to further its AI infrastructure efforts. The company has put its stamp on the MGX 1.2 specification with Vera Rubin rack and compute tray designs, offering manufacturers a common plan for AI systems that can handle 250 kW deployments with liquid cooling. Meta will be using NVIDIA Spectrum-6 Ethernet switches in its upcoming AI set-up for their 102.4 Tb/s capacity. And Google’s Falcon transport protocol will be supported on future products like the Spectrum-X platform.
Why USFP Matters for the Future of AI Data Centres
When you look at the trajectory of AI factories, it is clear that open specs and denser optical links are going to be essential. USFP is a case in point, providing higher-capacity connections in a module size operators are already used to. For anyone putting together the next round of AI systems, the issue is no longer just how to put in more GPUs but how to link them in a way that ensures the network does not become the limiting factor.
