Advances with Ultra Ethernet 1.1 and OCP SUE-T
19 hour ago / Read about 23 minute
Source:Converge Digest
Ultra Ethernet 1.1 and OCP’s SUE-T specification advance Ethernet-based AI Scale Up networking through complementary transport, endpoint and network requirements.

Ethernet’s role in tightly coupled AI systems is advancing through complementary work on accelerator endpoints, transport protocols and switching fabrics. The Ultra Ethernet Consortium released UE Specification 1.1 on October 12, adding Scale Up optimizations, application-compatibility improvements, In-Network Collectives, Congestion Signaling and Programmable Congestion Management. Separately, the Open Compute Project’s Scale-Up Ethernet Transport (SUE-T) Base Specification v1.0.0, effective October 8, connects accelerator transactions to UEC’s Unified Local transport profile.

The technical challenge extends beyond Ethernet bandwidth. A distributed AI system must move both large transfers and short, latency-sensitive operations, with predictable completion and congestion behavior. Headers, acknowledgments and idle intervals consume a larger proportion of link capacity when useful payloads are small. UE 1.1 addresses that overhead through a reduced inter-packet gap, a Unified Forwarding Header and compact Unified Local transport. UEC says the gap reduction saves up to 11 idle bytes, while UFH saves 20 bytes compared with a traditional IP packet.

OCP’s work separates two architectural responsibilities. Ethernet for Scale-Up Networking (ESUN) addresses the network—including framing, switching, error handling and lossless delivery—while SUE-T addresses accelerator endpoint communication. This distinction matters when evaluating implementations: support for a switching protocol does not establish support for accelerator memory semantics, and endpoint transport support does not establish the behavior of the complete fabric.

AI INFRASTRUCTURE · OPEN STANDARDS
Inside the Ultra Ethernet Stack
Five building blocks connecting accelerator transactions, Ethernet fabrics and programmable traffic control.
UE 1.1
Communications stack
PACKET EFFICIENCY
Less overhead. Expanded Scale Up support.
• Unified Local transport, compact headers, collective operations and congestion-management enhancements.
SUE-T
Accelerator endpoint
MEMORY TRANSACTIONS
Loads, stores and atomic operations.
• Maps accelerator commands onto UEC Unified Local, with defined acknowledgment and delivery behavior.
ESUN
Switching fabric
NETWORK BEHAVIOR
Efficient framing and lossless delivery.
• Defines Ethernet network requirements covering switching, flow control and link reliability.
CSIG
Congestion feedback
IN-BAND TELEMETRY
Identify the bottleneck along the path.
• Switches update compact packet tags with congestion information for endpoint control algorithms.
PCM
Programmable policy
ADAPTABLE CONTROL
Tune how endpoints respond.
• Portable interfaces support congestion-control and load-balancing policies where NIC capabilities permit.
The architectural connection: Accelerator operations → endpoint transport → Ethernet fabric, informed by congestion telemetry and control policy.

“A central goal of v1.1 is to reduce migration and deployment costs,” UEC wrote in its announcement, authored by Technical Advisory Committee chair Tom Emmons.

From Accelerator Commands to Ethernet Packets

SUE-T transports loads, stores and atomic operations. Its specification permits packing up to 16 transactions into a packet and distinguishes two acknowledgment modes: Delivery ACK confirms arrival at the destination accelerator; Operational ACK confirms processing and associates the response with the request. The latter trades additional endpoint state for reduced response overhead. Address translation and memory registration remain outside SUE-T’s scope.

That distinction has practical consequences for system design. Receiving a packet, completing an operation and making its result available to subsequent work are different events. An accelerator interface must preserve the completion behavior expected by its execution model. Consequently, headline link bandwidth alone cannot establish the performance of a distributed memory operation; request processing, response generation and transport overhead also contribute.

Reliability and Lossless Delivery

SUE-T uses ordered delivery with optional GoBackN retransmission. Omitting retransmission reduces buffering and state, but the specification warns that packet loss then becomes fatal to the connection. It requires lossless network service through Priority Flow Control or Credit-Based Flow Control, and separates requests and responses into different traffic classes to avoid deadlock. These are architectural requirements, rather than optional tuning details.

ESUN 1.0 supplies complementary network mechanisms. Its requirements cover link-level retry, flow control and efficient framing, including a compact four-byte ESUN header replacing the conventional IP/UDP stack. Link retry and end-to-end transport recovery operate at different scopes: repairing an error locally can avoid involving the source endpoint, while transport recovery handles failures visible across the connection. Flow control addresses buffer availability; congestion control regulates offered traffic. These functions work together but solve different problems.

Congestion Feedback and Programmable Response

UE 1.1’s Congestion Signaling uses four- or eight-byte Layer-2 tags that switches update with bottleneck information. The revision also introduces switch-based In-Network Collectives and programmable congestion-management interfaces. These additions address three different concerns: observing congestion, controlling traffic and reducing communication work.

The distinction between observation and response is important. Telemetry can reveal that a path is congested, but an endpoint still needs a policy deciding how much traffic to inject and when to change paths or credits. Faster measurements alone do not guarantee stability or fairness; the control algorithm and its execution timing determine how the system reacts.

The October 9 PCM research paper makes that programming model more concrete. Algorithms read signals such as acknowledgment counts, round-trip time and queue backlog; triggers determine when handlers execute; controls adjust parameters such as rate, window size or credits. Vendor backends map those abstractions onto native hardware. Portability therefore depends on the signals and controls a platform actually exposes: the researchers’ ConnectX backend, for example, rejects load-balancing handlers because its interface lacks path selection.

The paper also distinguishes processing capacity from live network validation. Its ConnectX-8 incast experiment used one isolated 100 Gbit/s plane of an 800 Gbit/s port. Separate trace-replay tests examined controller scalability. This is useful implementation evidence, but it should not be described as a complete 800 Gbit/s production-fabric interoperability demonstration. The researchers also found that numerical precision and delayed reactions can affect algorithm behavior despite source-code portability.

Analysis

The architectural significance is that Ethernet Scale Up increasingly involves an explicit relationship between accelerator commands, endpoint transport and network behavior. ESUN describes the fabric’s responsibilities; SUE-T connects accelerator operations to transport; UEC supplies shared protocol mechanisms. OCP’s documentation presents these as coordinated efforts with distinct scopes.

For buyers and system developers, this changes the questions worth asking. Beyond port speed and switch capacity, evaluation should establish which transactions an endpoint supports, what an acknowledgment guarantees, how ordering is maintained, how congestion propagates and how the system recovers from errors. Those answers determine whether components can support the intended execution model together.

The implementation tradeoffs also explain why Scale Up and Scale Out need different optimization points. Compact, ordered endpoint transport can reduce state, while broad multipath fabrics may prioritize different delivery and recovery behavior. Common Ethernet infrastructure creates opportunities for shared components and operational tools, but system performance still depends on the selected transport profile and topology.

The next milestones are interoperable implementations and workload measurements. Useful evidence would include transaction latency under contention, useful payload throughput, recovery behavior and accelerator utilization across a specified topology. Specification publication establishes a common engineering framework; those measurements will show how effectively implementations use it.

Organization Profile — Ultra Ethernet Consortium
OrganizationUltra Ethernet Consortium (UEC)
StructureIndustry consortium; Joint Development Foundation project hosted by the Linux Foundation.
LaunchedJuly 19, 2023
Technical leadershipTom Emmons — Chair, Technical Advisory Committee
FocusOpen Ethernet communications stack for AI and HPC; Scale Up and Scale Out transport, congestion management and interoperability.
SpecificationUE Specification v1.1 released October 12, 2026; initial v1.0 released June 11, 2025.
Standards coordinationOCP ESUN and SUE-T collaboration; IEEE coordination on CSIG.
Websiteultraethernet.org