Overview

Fast CNP is a congestion feedback optimization for RoCEv2 fabrics. Traditional congestion control relies on a congested switch marking ECN in packets, the receiver observing those marks, and the receiver then sending Congestion Notification Packets (CNPs) back to the sender. That feedback loop can be too slow in high-bandwidth AI networks where many flows converge on the same queue.

In an xSONiC AI fabric, Fast CNP shortens the control loop by allowing the congested switching node to notify senders more directly. The goal is to reduce queue buildup before buffer pressure turns into packet loss or widespread PFC pause behavior.

Traditional CNP Path

Sender servers
      |
      v
Congested switch marks ECN
      |
      v
Receiver observes marked traffic
      |
      v
Receiver sends CNP back to sender
      |
      v
Sender reduces rate

This process works, but the feedback path includes the receiver side of the conversation. In bursty many-to-one AI traffic, that delay can allow queue occupancy to keep growing while the fabric is waiting for rate reduction.

Fast CNP Path

Sender servers
      |
      v
Congested xSONiC switch detects pressure
      |
      v
Switch identifies affected RoCEv2 flows
      |
      v
Switch sends CNP-style notification toward senders
      |
      v
Senders reduce rate earlier

Fast CNP is most useful when the switch can identify the active RoCEv2 flows that are contributing to congestion and build the notification information that endpoint NICs expect.

Key Concepts

Term	Meaning	Why It Matters
Flow	A packet stream identified by common attributes such as IP addresses, ports, and queue pair information.	Lets the switch associate congestion with affected senders.
Flow Table	State maintained by the switch for active RoCEv2 sessions.	Provides the metadata needed to build targeted notifications.
CNP	Congestion Notification Packet used by RoCEv2 congestion control.	Tells senders to reduce their sending rate.
Queue Pressure	Buffer occupancy or forwarding delay that indicates congestion risk.	Provides the trigger for faster notification.

Flow Table Lifecycle

Fast CNP depends on accurate flow awareness. The switch learns and maintains flow state as RoCEv2 sessions are created, used, and removed.

Stage	Switch Behavior	Operational Check
Session establishment	Learn sender, receiver, and queue-pair information from control traffic.	Confirm expected flows are discovered during workload bring-up.
Data transfer	Refresh active entries as Send, Write, Read, and ACK traffic passes through.	Verify active flows do not age out during long-running jobs.
Session teardown	Remove entries when disconnect activity is detected.	Confirm stale entries are cleared after workload completion.
Aging control	Expire inactive or least-active entries when table limits are reached.	Size table and timeout values for real workload scale.

Congestion Detection

Fast CNP should be tied to measurable congestion signals rather than raw link utilization alone. In a lossless Ethernet environment, a link can appear busy while queues remain healthy, or queue depth can spike during microbursts even when average utilization looks safe.

Signal	Interpretation	Response
Forwarding delay crosses threshold	Queueing delay is becoming visible.	Identify affected flow entries and prepare notification.
Queue depth grows rapidly	Burst pressure may exceed available buffer.	Notify senders before PFC pause dominates.
Repeated ECN marking	Congestion is persistent rather than isolated.	Review workload fan-in, routing, and traffic class design.

AI Fabric Scenario

POD A servers  --->  Leaf A  --->  Spine  --->  Leaf B  --->  Target server
                         |
                         +-- congestion forms here
                         +-- Fast CNP notifies senders earlier

During all-reduce, checkpointing, or storage-heavy phases, many senders can target a small number of receivers. If congestion appears near a leaf or spine queue, Fast CNP can reduce the time it takes for senders to respond.

Deployment Guidance

1. Confirm RoCEv2 traffic classes, PFC, and ECN policy are already designed.
2. Identify where congestion is most likely: leaf uplinks, storage paths, or pod boundaries.
3. Size flow table and aging behavior for expected session counts.
4. Set delay or queue thresholds based on load-test evidence.
5. Validate sender rate reduction during incast and all-reduce tests.
6. Monitor CNP rate, ECN marks, PFC pause events, and queue depth together.

xSONiC Platform Fit

Fast CNP is best aligned with xSONiC 400G and 800G AI fabric switches where high fan-in traffic can overwhelm queues quickly. It also benefits 100G and 200G storage or frontend fabrics when RoCEv2 traffic is sensitive to congestion feedback delay.