NVIDIA Bets Ethernet Can Beat InfiniBand for AI

SONiCopen networkingdata centerAI fabricEthernetautomation

In brief

What NVIDIA Spectrum-X and Spectrum-6 mean for Australian AI fabric buyers comparing Ethernet, InfiniBand, SONiC, 400G, and 800G.

Key takeaways

What NVIDIA Spectrum-X and Spectrum-6 mean for Australian AI fabric buyers comparing Ethernet, InfiniBand, SONiC, 400G, and 800G.

What Happened: NVIDIA Positions Ethernet as an AI Fabric, Not Just a Cloud Network

The portfolio now spans five generations of Spectrum ASICs, from the SN2000 series (up to 100 Gb/s) through to the new SN6000 series based on the Spectrum-6 ASIC. The SN6000 family introduces co-packaged silicon photonics, which NVIDIA says doubles bandwidth per lane compared to the previous generation and significantly improves network resiliency.

Critically for open networking buyers, NVIDIA explicitly lists Pure SONiC as a supported network operating system alongside Cumulus Linux. Pure SONiC is described as ‘a community-developed, open source network operating system based on Linux that runs on switches from multiple vendors and powers some of the largest data centers in the world.’ This is a notable signal: NVIDIA is not locking its highest-performance Ethernet switches behind a proprietary NOS.

Why It Matters: SONiC Moves from Cloud-Only to AI-Ready

SONiC (Software for Open Networking in the Cloud) has long been production-hardened in hyperscaler environments. According to the SONiC Foundation, it is ‘an open source network operating system (NOS) based on Linux that runs on switches from multiple vendors and ASICs’ and ‘offers a full suite of network functionality, like BGP and RDMA, that has been production-hardened in the data centers of some of the largest cloud service providers.’

The project is housed under the Linux Foundation and backed by a broad ecosystem. Its architecture is containerized, with each network function running in its own Docker container, which the SONiC GitHub repository describes as providing ‘better fault isolation, easier debugging and troubleshooting, simplified upgrades and maintenance, and enhanced scalability.’

What is changing now is the workload profile. SONiC was built for cloud-scale leaf-spine fabrics carrying general-purpose traffic. NVIDIA’s endorsement of SONiC on Spectrum-X and the SN6000 series implies that the same NOS can support RDMA over Converged Ethernet (RoCE) traffic patterns required by GPU clusters. NVIDIA’s product page highlights ‘zero-touch accelerated RDMA over Converged Ethernet (RoCE)’ as a Spectrum switch portfolio feature.

For open networking buyers, this is a meaningful shift. It means the SONiC ecosystem is no longer limited to traditional data center east-west traffic. AI fabric requirements — low latency, congestion-aware flow control, and lossless Ethernet behavior — are now being addressed within a SONiC-compatible hardware and software stack.

The Australian Angle: Sovereign AI Infrastructure Meets Open Networking

Australia’s data center market is in a unique position. The country is investing in sovereign AI infrastructure, and operators are designing for AI workloads that demand new levels of network scale and cooling. In an OCP Podcast episode recorded in January 2026, David Hirst, CEO of Macquarie Data Centres, described how AI workloads are shifting data center design from ‘real estate’ to ‘chip-out thinking,’ with liquid cooling and megawatt-per-rack designs becoming necessary.

Hirst also highlighted what makes the Australian market different: data sovereignty requirements, power constraints in dense urban environments, and the role of compliance as a market advantage. These factors favor infrastructure choices that avoid vendor lock-in and provide operational flexibility.

Open networking on SONiC aligns with this direction. An Australian operator building an AI fabric does not need to commit to a single vendor’s NOS or management plane. With SONiC running on disaggregated switch hardware, the network operating system can be selected independently from the ASIC and platform. This is the core value proposition that the Open Compute Project’s Networking project has pursued: ‘a set of technologies that are disaggregated and fully open, allowing for rapid innovation in the network space,’ as stated on the OCP Networking project page.

For Australian enterprises and colocation providers evaluating AI infrastructure, the combination of SONiC on high-performance Ethernet switches is worth evaluating alongside proprietary alternatives. The question is not whether SONiC can run AI workloads — hyperscalers have already proven that — but whether the ecosystem of supported hardware, optics, and operational tooling is mature enough for Australian-scale deployments.

What the Product Portfolio Actually Looks Like

NVIDIA’s Spectrum Ethernet switch family now covers a wide range of deployment scenarios:

Series	ASIC	Max Port Speed	Target Use Case	SONiC Support
SN2000	Spectrum	100 Gb/s	HCI, SDS, leaf-spine	Listed
SN3000	Spectrum-2	200 Gb/s	Leaf-spine, full-rack connectivity	Listed
SN4000	Spectrum-3	400 Gb/s	Cloud-scale distributed apps	Listed
SN5000	Spectrum-4	800 Gb/s	Deep learning, GPU compute	Listed
SN6000	Spectrum-6	800 Gb/s (co-packaged optics)	AI factories, silicon photonics	Listed

NVIDIA also lists OEM partners and software tools including DSX Air (digital twin simulation), Cumulus Linux, NetQ (real-time visibility), and Pure SONiC.

A point of interest for xSONiC-aligned buyers: the emphasis on ‘open NOS support’ and the listing of SONiC alongside Cumulus Linux suggests that NVIDIA sees the open NOS ecosystem as a competitive advantage, not a concession. Whether this openness extends to full feature parity between Cumulus and Pure SONiC on Spectrum-6 hardware is a question that requires vendor confirmation.

The InfiniBand Question: Why Ethernet for AI Is a Real Debate Now

NVIDIA’s simultaneous push of InfiniBand (Quantum-X800) and Ethernet (Spectrum-X) for AI workloads is worth noting. The company has a financial interest in selling both. But the market signal is real: Ethernet-based AI fabrics are gaining traction because they offer operational familiarity, multi-vendor interoperability, and a larger talent pool.

For Australian operators, the InfiniBand vs. Ethernet debate has practical implications. InfiniBand fabrics require specialized skills and vendor-specific management tools. Ethernet fabrics can leverage existing operational knowledge, standard protocols, and — critically for open networking buyers — SONiC as the NOS.

The OCP Networking project’s scope explicitly includes SONiC as a sub-project, and the project aims to give ‘end users the ability to forgo traditional closed and proprietary network switches - in favor of a fully open network technology stack.’ This framing is relevant to the AI fabric decision. An operator choosing Ethernet for an AI cluster can stack SONiC on top of open switch hardware and maintain NOS portability across generations of ASIC silicon.

What Australian Buyers Should Evaluate

For enterprise and colocation operators in Australia planning AI infrastructure, the following evaluation criteria are relevant:

NOS portability: Does the switch platform support SONiC with full feature parity for RoCE, DCBX, and congestion management? Confirm this with the hardware vendor, not just the ASIC vendor’s marketing page.
Optics strategy: Co-packaged optics (as in the SN6000) change the transceiver supply chain. Evaluate whether your optics suppliers can support the connector types required, and whether open-market transceiver options exist for non-co-packaged platforms.
Fabric management: SONiC provides the NOS, but fabric-level automation, telemetry, and troubleshooting require additional tooling. Evaluate the operational stack, not just the switch.
Sovereign compliance: For workloads subject to Australian data sovereignty requirements, confirm that the entire networking stack — hardware, NOS, and management plane — can be operated without dependency on offshore control planes or telemetry services.
Cooling and power: AI fabric switches consume significant power. Align switch selection with your facility’s cooling capacity, especially if you are designing for the liquid-cooled, high-density racks described by Australian operators like Macquarie Data Centres.

Editorial Take: Open Networking Has a Window in the AI Fabric Market

The convergence of three factors — SONiC maturity, Ethernet’s AI readiness, and NVIDIA’s willingness to support open NOS on its highest-performance switches — creates a window for open networking in the AI infrastructure market. This window is particularly relevant in Australia, where sovereign infrastructure requirements, a preference for operational flexibility, and a growing AI investment cycle favor disaggregated architectures.

xSONiC’s positioning in the open networking space aligns with this market direction. The brand’s focus on SONiC-based data center switches, enterprise campus infrastructure, and supporting components (optics, NVMe SSDs, packet brokers) maps directly to the stack that Australian operators will need to evaluate as they build AI-ready networks.

What can be stated is the direction: the market is moving toward open, disaggregated AI networking, and Australian operators have structural reasons to prefer that direction. The editorial opportunity is to document the evaluation criteria, surface the trade-offs, and position xSONiC as a credible participant in the conversation.

Ethernet vs InfiniBand Evaluation Matrix

Evaluation area	Acceptance evidence	Rework trigger
Workload fit	Training, inference, storage, and east-west traffic mapped to latency, throughput, and congestion targets	Fabric choice is made from vendor positioning rather than workload evidence
Ethernet readiness	RoCE v2, PFC, ECN, DCBX, 400G/800G optics, and telemetry validated on target switches	Packet loss, pause storm, or ECN mismatch cannot be explained during testing
Operations	SONiC or vendor NOS support, automation, rollback, and support ownership documented	Team gains performance but loses day-2 operational control
Expansion	128, 256, and 1,024 GPU growth scenarios modelled for power, cabling, optics, and spine density	Scale-out requires a forklift fabric change

Engineering Evidence Floor

For this topic, treat the article as buyer-ready only when the fabric claim is tied to measurable acceptance data. The baseline evidence package should include the selected switch SKU, SONiC image, ASIC/SAI version, NIC firmware, optics list, RoCE policy, telemetry counters, support owner, and rollback method. In a small pilot, capture at least 30 minutes of traffic at the intended 100G/400G/800G speed, one link failure, one switch reboot, one optics fault, and one support handoff. That evidence separates a credible AI fabric plan from a bandwidth brochure.

Evidence area	What to validate	Acceptance gate	Rework trigger
Transport	PFC, ECN, DCBX, MTU, queue mapping, and CNP counters	30 minutes load test at 100G/400G/800G	RoCE is asserted but not measured
Platform	Switch SKU, ASIC, SAI, SONiC image, and optics list	2 switch roles pass upgrade and rollback	Generic compatibility is used as proof
Failure	Link loss, switch reboot, route convergence, and workload impact	3 failure cases captured with timestamps	Steady-state throughput is the only evidence
Telemetry	Queue depth, drops, optics DOM, gNMI, and packet visibility	Operators explain a slowdown within 15 minutes	GPU and network teams use different data
Support	APAC escalation, RMA, spares, and patch lifecycle	12 months operating plan approved	Ownership splits across vendors

Engineering FAQ

What should be proven before adopting EVPN-VXLAN on SONiC? Prove underlay routing, BGP sessions, VTEP behaviour, MAC/IP learning, route scale, multi-homing design, failure convergence, and observability. The overlay should be accepted as a system, not a feature checkbox.

Why does the underlay design still matter in an overlay network? EVPN-VXLAN depends on a stable routed underlay. MTU, ECMP, addressing, route policy, link failure behaviour, and telemetry determine whether the overlay remains predictable under load and during faults.

What should be included in an EVPN-VXLAN operations runbook? Include naming, IP plan, BGP policy, VNI mapping, change process, rollback commands, failure checks, telemetry fields, backup and restore steps, and escalation ownership for the selected SONiC image.

Sources Reviewed

Product fit

Where xSONiC fits

xSONiC can help validate the switch, optics, software image, telemetry, and support assumptions against the actual deployment before a production order is released.