MC-LAG, STP, and Policy-Based Routing

SONiCopen networkingdata centerAI fabricEthernetautomation

In brief

Engineering guidance on MC-LAG, STP and PBR for Australian campus refresh teams evaluating open networking, failover and policy control.

Key takeaways

Engineering guidance on MC-LAG, STP and PBR for Australian campus refresh teams evaluating open networking, failover and policy control.

The campus network is overdue for a rethink

Enterprise campus networks in Australia are approaching a inflection point. Many organisations built their current access and aggregation layers on proprietary switch stacks five to eight years ago. Those stacks are reaching end-of-support, end-of-sale, or simply end-of-capacity as PoE device counts, IoT endpoints, and security policy complexity continue to climb.

At the same time, the SONiC ecosystem has matured well beyond its hyperscaler origins. SONiC (Software for Open Networking in the Cloud) is a Linux-based, open-source network operating system maintained under the Linux Foundation. According to the SONiC Foundation, it offers a full suite of network functionality including BGP and RDMA, built on a container-based architecture that decouples hardware from software. The GitHub repository confirms SONiC runs on switches from multiple vendors and ASICs, is licensed under Apache 2.0, and has accumulated nearly 3,000 commits with an active contributor community.

This raises a practical question for Australian enterprise network teams: can SONiC-based campus switching deliver the MC-LAG, STP, and policy-based routing features that campus networks depend on, without the vendor lock-in that comes with proprietary alternatives?

What MC-LAG actually solves in campus aggregation

Multi-Chassis Link Aggregation (MC-LAG) allows two aggregation or distribution switches to present as a single logical LACP endpoint to downstream access switches. The practical benefit is straightforward: active-active uplinks without spanning tree blocking, faster failover than STP reconvergence, and simplified cabling.

In traditional campus designs, STP handles loop prevention but at the cost of blocked redundant links. MC-LAG eliminates that waste by operating at the link aggregation layer rather than relying on STP to manage topology changes. For Australian enterprises running campus buildings with dozens of access closets, this translates to more usable bandwidth and more predictable failover behaviour.

However, MC-LAG is not a universal replacement for STP. Most campus deployments still need STP as a safety net for misconfigurations, rogue switches, and edge cases where MC-LAG peers lose synchronisation. The interplay between MC-LAG and STP is where deployment complexity lives, and where vendor-specific implementations have historically locked buyers into single-vendor stacks.

Policy-based routing: moving beyond destination-only forwarding

Policy-based routing (PBR) gives network operators the ability to forward traffic based on criteria beyond the destination IP address. Source subnet, protocol, port, DSCP marking, and ACL match can all influence the forwarding decision.

In campus networks, PBR serves several practical purposes:

Directing guest traffic through a specific security appliance or internet breakout
Steering IoT device traffic to segmented VLANs without requiring client-side changes
Enforcing application-aware routing for voice, video, and collaboration tools
Load sharing across multiple WAN or internet paths at the campus edge

For Australian organisations with multi-site campus footprints, PBR at the distribution or core layer simplifies policy enforcement without requiring changes at every access switch. The alternative, without PBR, is typically a combination of static routes, VRFs, and ACLs that grows harder to audit as the network scales.

The open networking question: can SONiC deliver campus-grade features?

SONiC’s architecture is well documented for data centre use. BGP, EVPN-VXLAN, RDMA support, and a container-based modular design give it credibility in spine-leaf fabrics. The question is whether the same architecture can support campus-specific requirements.

NVIDIA’s Ethernet switching portfolio offers a signal. NVIDIA lists Pure SONiC alongside Cumulus Linux as supported network operating systems for its Spectrum Ethernet switches. The Spectrum switch family spans from the SN2000 series (up to 100 Gb/s) through to the SN6000 series (up to 800 Gb/s), with products designed for leaf, spine, and aggregation roles. This breadth of hardware support, combined with SONiC as a validated NOS option, suggests that the ecosystem is moving beyond data-centre-only positioning.

The SONiC Foundation describes the platform as offering multi-vendor support through the Switch Abstraction Interface (SAI), which accelerates hardware innovation by decoupling the NOS from specific ASIC vendors. For Australian enterprises evaluating campus refresh options, this decoupling is the core value proposition: the ability to choose switching hardware based on port density, PoE budget, and price-performance rather than being constrained to a single vendor’s NOS.

What this means for Australian campus refresh decisions

Australian enterprise campus networks face a specific set of pressures:

Ageing infrastructure from the last major refresh cycle (typically 2016-2019)
Growing PoE device counts from IoT, security cameras, and wireless access points
Increasing security and segmentation requirements driven by compliance frameworks
Budget pressure that favours disaggregated procurement over single-vendor bundles
Shortage of network engineering talent familiar with proprietary CLI environments

The talent issue is worth highlighting. SONiC is Linux-based and uses standard Linux interfaces and tools. Engineers with Linux experience can transition to SONiC operations faster than learning another proprietary CLI. For Australian organisations competing for skilled network engineers, this matters.

The competitive gap: proprietary campus stacks are not standing still

It would be misleading to frame this as a simple open-vs-closed choice. Incumbent vendors have invested heavily in campus automation, cloud-managed operations, and AI-driven network assurance. Their MC-LAG and PBR implementations are mature and well-documented.

The gap is not in feature lists but in procurement flexibility. Proprietary campus stacks typically require:

Switches from the same vendor at access, distribution, and core layers
Vendor-specific licensing for advanced features like PBR or telemetry
Support contracts tied to specific hardware generations
Limited ability to mix vendors for cost optimisation across sites

Open networking with SONiC-based switching addresses each of these constraints. The buyer can select hardware from multiple vendors, use a single NOS across campus and data centre domains, and avoid per-feature licensing. For Australian organisations managing campus networks across multiple sites with varying requirements, this flexibility has tangible cost and operational benefits.

Decision criteria for Australian enterprises evaluating open campus switching

For organisations considering SONiC-based campus switching with MC-LAG, STP, and PBR, the evaluation should cover:

Criterion	What to verify	Acceptance evidence
MC-LAG implementation	Active-active LACP with 2 aggregation peers and dual-homed access switches	Both uplinks forward traffic, peer failure restores service within the agreed 5s campus change target, and traffic counters prove no silent black-hole state
STP compatibility	RSTP or MSTP behaviour across at least 10 representative VLANs	MC-LAG failure, rogue-switch attachment, and loop-prevention tests are captured in the change record before rollout
PBR granularity	Source subnet, destination subnet, DSCP, L4 port, protocol, and ACL match criteria	Test traffic proves each policy takes the intended next hop and falls back cleanly when that next hop fails
PoE support	Per-port PoE or PoE+ requirement and per-switch power budget	AP, camera, badge-reader, and IoT endpoint classes are modelled against the real closet power budget with at least 20% headroom
Management and automation	NETCONF/YANG, gNMI, Git-backed config and rollback workflow	A pilot change can be deployed, verified, and rolled back in a controlled 30 minutes window
Hardware ecosystem	SAI-compatible switch options across the target 25G, 100G, or 400G uplink tiers	The bill of materials names supported optics, NOS image, ASIC/SAI version, and local spare strategy
Australian support	Local technical support, RMA path, and escalation ownership	Support response, replacement logistics, and out-of-hours escalation are written into the acceptance plan

Campus Refresh Acceptance Matrix

Workstream	Engineering proof required before rollout	Reject or rework trigger
Access closet inventory	Document every uplink, endpoint class, PoE load, VLAN and unmanaged device before the pilot	Unknown uplinks, more than 10% undocumented endpoints, or no rollback path for legacy voice/camera devices
Resilience test	Pull one MC-LAG peer, one uplink and one access switch power feed during a controlled test	User VLANs fail to reconverge, monitoring misses the event, or PBR next hops become inconsistent
Policy migration	Translate incumbent PBR rules into SONiC configuration and validate hit counters with synthetic and real flows	Any compliance, guest, voice or IoT class follows an unintended path during failure testing
Operations readiness	Train service desk and network operations on commands, telemetry, backup and restore	The team cannot identify the failed peer, affected VLAN and rollback command from the runbook

Where xSONiC fits

xSONiC’s enterprise campus switching portfolio is designed around this exact evaluation. The Access and Aggregation Switches provide the hardware foundation for campus access, distribution, and core layers, running SONiC-based NOS with support for MC-LAG, STP, and PBR. Data Center AI Switches give engineering-led teams the option to deploy custom or community SONiC builds on open switching hardware.

For deeper guidance on campus-specific deployments, the MC-LAG and STP solution guide covers multi-chassis link aggregation design, STP interoperability, and failover behaviour. The Policy-Based Routing guide addresses PBR use cases for campus traffic steering, segmentation, and application-aware forwarding.

Organisations planning a campus refresh can start with the Campus Refresh solution for architecture guidance, or contact xSONiC for a campus network assessment tailored to Australian site requirements.

The editorial takeaway

MC-LAG, STP, and policy-based routing are not new technologies. What is new is the emergence of SONiC as a credible campus NOS candidate, backed by Linux Foundation governance, multi-vendor hardware support, and validation from major networking vendors including NVIDIA. For Australian enterprises facing campus refresh decisions, the question is no longer whether open networking works for campus, but whether their next refresh should be built on the same proprietary stack or take advantage of the procurement flexibility and operational consistency that SONiC-based switching offers.

The answer depends on where an organisation sits on the risk tolerance spectrum. Early adopters with Linux-skilled teams and multi-site campuses have the most to gain. Organisations with deep incumbent investment and single-vendor support contracts may prefer to wait for broader campus SONiC adoption before making the move. Either way, MC-LAG, STP, and PBR should be on the evaluation checklist.

Engineering FAQ

What should be tested before moving campus switching to SONiC or open networking? Test PoE behaviour, NAC integration, VLAN and policy design, STP or MC-LAG interaction, multicast, monitoring, upgrade rollback, and help-desk workflows across at least 1 representative building. Campus readiness is an operations test, not only a forwarding test.

Where do campus refresh projects usually carry hidden risk? The risk often sits in closets: power budget, old cabling, undocumented uplinks, mixed endpoint types, voice devices, cameras, badge systems, and 15 minutes change windows. Those details should be inventoried before selecting switch models.

How should Australian campus teams structure a pilot? Choose one representative site or building, document endpoint classes, run PoE and failover tests, verify monitoring, train operations staff, and keep the pilot stable for 2 weeks before expanding to the broader estate.

Sources Reviewed

Product fit

Where xSONiC fits

xSONiC can help validate the switch, optics, software image, telemetry, and support assumptions against the actual deployment before a production order is released.