InfiniBand vs Ethernet

InfiniBand and Ethernet are the two network fabrics that connect GPUs in a training cluster. The choice between them sets how much of a cluster’s theoretical compute turns into finished work. The interconnect sets the ceiling. Accelerator generation fixes the theoretical peak. The network decides how much of that peak a distributed training job ever reaches.

Most procurement requests get written around chip generation while the fabric decision comes later. That ordering fails for any workload that synchronizes across more than a rack. Nearly every public account of large-cluster performance tells the same story: throughput came from network engineering, and the silicon never changed.

What Is Model FLOPs Utilization, and Why Does It Expose the Network?

Model FLOPs Utilization (MFU) measures the share of a cluster’s theoretical peak arithmetic that becomes useful training work. ByteDance’s MegaScale system reached only 55.2% MFU while training a 175-billion-parameter model across 12,288 GPUs, according to the paper presented at USENIX NSDI 2024. At that level, nearly half of the compute in the cluster produced nothing.

The figure deserves attention for a second reason. MegaScale’s 1.34x improvement over the Megatron-LM baseline came from network and communication engineering, according to the same USENIX paper. Faster accelerators played no part. The team closed the gap by reworking routing and congestion control.

Consider what that implies for a purchase order. Two clusters can hold the same accelerators and the same software stack, yet differ by tens of points in delivered throughput. The difference lives in the switches and in the congestion parameters set on them. Therefore, a specification that names a GPU part number and stops has described about half of the machine.

Worse, the missing half is the part enterprises can least audit after delivery. Accelerator specifications are public and easy to verify. Fabric behavior under a real all-reduce shows up only once the job runs and the invoice has started.

Why Collectives Turn the Network Into the Ceiling

A single GPU server rarely causes trouble. Runs stall because thousands of GPUs exchange gradients and activations in tight synchronization. Every microsecond of delay then multiplies across the collective operation.

Meta’s engineering team documents this in a paper titled “RDMA over Ethernet for Distributed AI Training at Meta Scale,” presented at SIGCOMM 2024. The company built specialized data center networks for its GPU clusters. It chose “RDMA Over Converged Ethernet version 2 (RoCEv2) as the inter-node communication transport for the majority of our AI capacity,” according to Meta’s engineering blog. That choice was deliberate. It decided whether a very large GPU fleet delivered rated throughput or lost it to queuing delay.

The mechanism is simple to state and hard to escape. Large model training depends on collective communication, mainly all-reduce and all-to-all. Every GPU in the group waits for the slowest link in the exchange. A congested switch port, a dropped packet or an unevenly loaded path stalls the whole synchronized step. One slow port holds every other GPU still.

Scale Multiplies the Odds Against You

Cluster growth from hundreds to tens of thousands of GPUs raises the chance that at least one link runs degraded during any given step. Consequently, tail latency across the fabric becomes the real ceiling on job completion time. Peak spec sheet numbers stop predicting anything useful.

Academic work frames the problem the same way. A paper on GPU cluster scheduling for network-sensitive deep learning, published on arXiv, separates job completion time, queueing delay and communication latency into distinct first-class metrics. Communication latency often drives the wait time teams actually experience. In practice, that is uncomfortable for anyone budgeting purely in GPU-hours. Adding accelerators without fixing the fabric raises cost faster than it raises throughput.

InfiniBand: Deterministic Latency by Design

InfiniBand earned its position in AI supercomputing through architecture. Credit-based flow control avoids packet drops by construction. According to a technical comparison published by IP Infusion, InfiniBand “uses credit-based flow control, sending only when the receiver has advertised buffer credits.” A sender holding no credit does not transmit. Buffers therefore never overflow, and the fabric never has to recover from a drop it caused itself.

Determinism converts into measurable latency. TrendForce’s analysis of the scale-out networking market describes InfiniBand as the performance leader, with native RDMA delivering latency under 2 microseconds and no packet loss risk. Thousands of GPUs synchronize every few milliseconds. That sub-2-microsecond floor compounds across a multi-week run.

The tradeoff is vendor control and cost. TrendForce also notes that NVIDIA’s Mellanox subsidiary has long dominated InfiniBand switching, which keeps the ecosystem relatively closed. The same analysis puts InfiniBand hardware cost at “approximately three times those of Ethernet switches.” Buyers accept a narrower supplier pool in exchange for the lowest available latency floor.

Whether the trade pays depends on how long the jobs run. On a dedicated pretraining cluster, every point of MFU shortens wall-clock time on work measured in weeks. Our guide on how to choose the right GPU for AI workloads explains why accelerator choice and network architecture need joint evaluation rather than sequential sign-off.

Ethernet Closes In: RoCEv2, Spectrum-X and Ultra Ethernet

RoCEv2 in Production

Ethernet has spent years re-engineering the parts of itself that suited it poorly to lossless, latency-sensitive traffic. RoCEv2 is the mechanism. IP Infusion describes it plainly: “RoCEv2 runs RDMA over open, multi-vendor Ethernet.” The transport stays lossless using Priority Flow Control and Explicit Congestion Notification. Data Center Quantized Congestion Notification handles rate control.

That model reacts where InfiniBand prevents. Signals must arrive and change sender behavior before buffers fill. Careful tuning therefore separates a working RoCE fabric from one that reproduces the head-of-line blocking of earlier Ethernet designs.

Meta’s production experience shows how much tuning that takes. The company built a purpose-built topology instead of deploying generic data center Ethernet. Meta’s engineering blog describes “a two-stage Clos topology for AI racks, known as an AI Zone.” A rack training switch handles scale-up connectivity inside the rack. A spine tier of cluster training switches provides scale-out connectivity across racks.

Load balancing needed similar attention. Default Ethernet hashing spreads flows poorly when a handful of long-lived RDMA connections carry all the traffic. Meta reports configuring switches to perform “Enhanced ECMP” that additionally hashes on the destination queue-pair field of a RoCE packet, with measurable performance gains over baseline ECMP. One field in the hash function moved the needle across that fleet.

Spectrum-X and the Ultra Ethernet Consortium

NVIDIA’s Ethernet platform, Spectrum-X, targets a different buyer than a from-scratch RoCE build. An independent technical breakdown from Weka frames the split clearly: Spectrum-X suits “a cloud provider that deploys thousands of GPU servers using standard Ethernet who wants to optimize for AI without changing their entire networking fabric.” By contrast, InfiniBand remains preferable for a research institute running massive LLM training in a dedicated cluster that prioritizes performance above all else, per Weka’s analysis. Workload shapes the answer, and it changes per contract.

Standardization is the newest variable. The Ultra Ethernet Consortium, a Linux Foundation project backed by major silicon and system vendors, released its first complete specification in mid-2025. Per the consortium’s announcement, the specification is “engineered to meet the demanding needs of modern AI and HPC workloads” and spans NICs, switches, optics and cables under one interoperable stack. Multi-vendor commitment is the whole point of the exercise. It answers the single-vendor economics behind InfiniBand switch pricing.

Comparing the Fabrics

One point escapes the table. Neither fabric is a purchase an enterprise can leave unmanaged after installation. Delivered cluster performance depends on how well topology and congestion parameters match the specific workload, whatever logo sits on the switch.

What Else Belongs in a Build Spec

Fabric type is the first line of a build spec, and several others matter almost as much. Scale-up and scale-out are separate problems. Meta’s AI Zone design makes the split visible: a rack training switch inside the rack, a spine tier across racks. A serious spec states both tiers and the oversubscription ratio between them.

Cabling and optics deserve the same line-by-line treatment as accelerators. The UEC specification covers optics and cables for a reason. Link flap and transceiver failure are fabric problems, yet they surface to the user as training stalls.

Fault domains come next. Ask which failure removes a rack from the job and how the run restarts afterward. A checkpoint interval tuned without knowledge of the fabric’s failure behavior is a guess.

Telemetry finishes the picture. A fabric without per-queue counters and drop visibility cannot be debugged mid-run. Teams then blame the model code for a problem living in a switch buffer.

What This Means for Procurement

Specifying an accelerator generation and stopping there asks half the question. MegaScale’s 55.2% MFU on a well-engineered 12,288-GPU cluster, per the USENIX NSDI 2024 paper, shows what the other half is worth. Sophisticated operators still leave close to half of theoretical compute unused. The gains they recovered came from the network layer. Our piece on GPU utilization in enterprise AI covers the gap between rated and realized throughput in more detail.

The practical decision splits along workload lines. Dense, tightly synchronized pretraining on a dedicated cluster benefits most from InfiniBand’s deterministic latency. There, the cost premium buys back wall-clock time on jobs that run for weeks. Inference-heavy and mixed workloads are often better served by RoCEv2 or Spectrum-X deployments tuned to the traffic pattern, particularly on fleets already standardized on Ethernet. Our Blackwell GPU comparison covers how generation-specific interconnect requirements feed into that call.

Vendor lock-in is the variable procurement teams most often underweight. TrendForce’s coverage of the scale-out market describes the current period as a live competitive fight, with the InfiniBand camp on one side and an assembling Ethernet coalition standardizing around UEC on the other. Nothing about that contest has settled. Committing a multi-year fabric decision before the standard matures carries real switching-cost risk.

Delivery model matters here. Axe Compute Access exists so teams can specify the fabric alongside the accelerator across numerous global locations. Axe Compute Build applies the same principle at dedicated AI factory scale, with topology designed against the customer’s actual collective communication pattern.

How Do You Measure What the Network Costs You?

Job completion time is the metric that exposes network problems, and most teams never separate it from GPU utilization. The arXiv paper on network-sensitive GPU cluster scheduling treats those measures as distinct. Conflating them hides where the bottleneck sits.

A cluster can report high GPU utilization and still be network-bound. Utilization counters measure busy time. Busy time includes waiting on a peer, so inefficient batching can mask stalled collective operations behind a healthy-looking dashboard.

Questions Worth Asking a Provider

Ask for topology detail with the same seriousness you apply to accelerator specifications. Is the fabric a two-stage or three-stage Clos design? Does ECMP hashing account for RDMA queue-pair fields, or does it rely on defaults that collapse traffic onto a few paths? Which congestion control scheme runs at the switch layer, and who tuned its thresholds?

Answers separate infrastructure engineered for AI collective communication from generic data center Ethernet repurposed for the job. A provider that cannot describe its congestion control settings has not tuned them. Our guide on how to evaluate a GPU cloud provider treats interconnect architecture as a first-order criterion for exactly that reason.

Where the Fabric Goes Next

The UEC 1.0 specification is barely a year old. Compliance programs that make multi-vendor UEC gear interoperate reliably at scale are still being built. Meanwhile, InfiniBand keeps moving, with NVIDIA pushing latency and bandwidth on its own roadmap alongside newer GPU generations. As a result, expect the gap between the fabrics to keep narrowing for mainstream training work and to stay wide for the largest, most latency-sensitive runs.

Buyers should plan for that split. Write fabric requirements into the next contract. Ask for the congestion control configuration in writing. Measure job completion time separately from utilization on the first workload you run. A team that does this learns within one training run whether the network is costing it a third of its cluster, while the contract terms are still open to change.

Specify the fabric and the accelerator together on your next build. Reserve capacity at portal.axecompute.com, or contact us at info@axecompute.com to review topology options against your collective communication pattern.

About Axe Compute

Axe Compute Inc. (NASDAQ: AGPU) is a neocloud AI infrastructure platform built on a fundamental premise: AI innovation should not be constrained by hardware choice or inventory limitations. Axe Compute gives enterprises and AI innovators choice across hardware, geography, and deployment speed through two delivery models: Axe Compute Access, providing the latest GPU compute options in as fast as 48 hours across numerous global locations, and Axe Compute Build, enabling enterprises to access large-scale dedicated AI factories, all backed by enterprise-grade SLAs and support.

Frequently Asked Questions

What is the main difference between InfiniBand and Ethernet for AI clusters?

InfiniBand uses credit-based flow control to avoid packet loss by design, giving it deterministic, sub-2-microsecond latency. Ethernet fabrics for AI, such as RoCEv2, achieve similar lossless behavior through reactive mechanisms like Priority Flow Control, Explicit Congestion Notification, and Data Center Quantized Congestion Notification, which require careful tuning to match InfiniBand’s consistency.

Is InfiniBand always faster than Ethernet for GPU training?

InfiniBand generally has a lower and more predictable latency floor, which benefits tightly synchronized training workloads. However, well-tuned Ethernet fabrics running RoCEv2 or NVIDIA’s Spectrum-X platform can perform effectively for many production workloads, particularly on infrastructure already standardized on Ethernet operations.

Why is InfiniBand more expensive than Ethernet?

InfiniBand switching has been dominated by a single vendor ecosystem, which keeps hardware pricing elevated. According to TrendForce’s market analysis, InfiniBand hardware costs run roughly three times those of comparable Ethernet switches.

What is the Ultra Ethernet Consortium and why does it matter?

The Ultra Ethernet Consortium is a Linux Foundation project developing an open, multi-vendor Ethernet standard for AI and HPC networking. Its Specification 1.0, released in mid-2025, covers NICs, switches, optics, and cables, offering an alternative to InfiniBand’s largely single-vendor ecosystem.

How does network performance affect GPU utilization?

Large model training depends on collective communication operations where every GPU waits for the slowest link in the exchange. Network congestion, packet loss, or poor routing can stall entire synchronized training steps, which is why network-layer engineering, not just GPU count, determines effective utilization such as Model FLOPs Utilization.

Should every AI workload use InfiniBand?

No. Dedicated, large-scale training runs that prioritize raw performance tend to benefit most from InfiniBand’s deterministic latency. Inference-heavy or mixed workloads, especially on infrastructure already running standard Ethernet, are often better served by RoCEv2 or Spectrum-X deployments tuned to the specific traffic pattern.

What should buyers ask a GPU cloud provider about networking?

Buyers should ask about the network topology (two-stage versus three-stage Clos design), whether routing accounts for RDMA queue-pair fields to avoid hashing collisions, and which congestion control scheme runs at the switch layer. These details separate infrastructure engineered for AI workloads from generic data center Ethernet.

About Axe Compute

Sources

• Meta Engineering, “RoCE networks for distributed AI training at scale (RDMA over Ethernet for Distributed AI Training at Meta Scale)”, Meta Engineering Blog / SIGCOMM 2024
• Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, et al. (ByteDance, Peking University), “MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs”, USENIX NSDI 2024
• IP Infusion, “RoCE vs InfiniBand for AI Clusters”, IP Infusion Technology Blog
• TrendForce, “InfiniBand vs Ethernet: Broadcom and NVIDIA Scale-Out Tech War”, TrendForce Insights
• Weka, “NVIDIA Spectrum-X Ethernet Platform: How it Works and More”, Weka Learn / Enterprise Technology
• Ultra Ethernet Consortium, “UEC Launches Spec 1.0 Transforming Ethernet for AI and HPC at Scale”, Linux Foundation
• Aakash Sharma, et al., “GPU Cluster Scheduling for Network-Sensitive Deep Learning”, arXiv