Oak Ridge Opens Lux AI Supercomputer With 800G Ethernet Fabric
6 hour ago / Read about 9 minute
Source:Converge Digest
Oak Ridge’s Lux opens to researchers at No. 14 on the TOP500, pairing AMD Instinct MI355X GPUs and Pensando DPUs with HPE Juniper switches that support 800G Ethernet.

Oak Ridge National Laboratory opened its Lux AI supercomputer to scientific users on November 16, as the HPE-built system debuted at No. 14 on the TOP500 list with 220.79 petaflops on the High Performance Linpack benchmark. HPE disclosed at SC26 that Lux uses Juniper QFX switches that support 800 Gigabit Ethernet. HPE calls it the first deployment of its networking technology at a U.S. Department of Energy national laboratory.

HPE and AMD co-designed Lux around direct liquid-cooled HPE ProLiant Compute XD685 servers with AMD EPYC CPUs, Instinct MI355X GPUs and Pensando data processing units. HPE says the open Ethernet fabric uses load balancing and congestion management. Its Networking Data Center Director software handles deployment and lifecycle management, and AMD’s ROCm provides the GPU software stack.

Lux gives multiple research teams cloud-like access to resources for AI training and inference alongside modeling and simulation. HPE says half of the system’s 3.5 million annual node-hours will support the DOE’s Genesis Mission. Oak Ridge identified Earth system prediction, materials development, biology and fusion energy as areas that could use the capacity.

Lux will “provide researchers with additional capacity and flexibility to advance innovative SI approaches,” said Georgia Tourassi, associate director of computing and computational sciences at the Oak Ridge Leadership Computing Facility.

Lux at a Glance
• No. 14 on the November 2026 TOP500 · 220.79 petaflops on High Performance Linpack (HPL), as reported by Oak Ridge.
• 800G-capable Ethernet · HPE Juniper QFX switches with load balancing and congestion management.
• AMD platform · EPYC CPUs, Instinct MI355X GPUs, Pensando DPUs and the ROCm software stack.
• System · Direct liquid-cooled HPE ProLiant Compute XD685 servers.
• Access · Opened to scientific users November 16. HPE says half of the system’s 3.5 million annual node-hours will support the DOE’s Genesis Mission.

Analysis

Oak Ridge now operates two HPE-built systems with distinct network designs and workloads. Frontier, No. 3 in the November TOP500, uses HPE Cray Slingshot-11 and remains Oak Ridge’s exascale platform. Lux pairs a Juniper Ethernet fabric with AMD accelerators for a shared AI service. For HPE, Lux is a deployment of Juniper data center switching alongside its servers in a federal research environment, following the July 2025 acquisition.

For a multi-tenant cluster, the fabric has to handle traffic from multiple jobs at once. Lux’s HPL score measures system-level performance on a dense linear algebra workload. HPE disclosed the QFX switches’ 800G capability. The announcement leaves the switch model, port speeds in use, topology, optics and per-job network performance open.

Company Profile | Hewlett Packard Enterprise
HeadquartersSpring, Texas, USA
LeadershipAntonio Neri, President and CEO
Founded2015
ListingNYSE: HPE
BusinessEnterprise infrastructure, networking, hybrid cloud, AI systems and data center platforms.
NetworkingHPE completed its acquisition of Juniper Networks in July 2025. The portfolio includes Juniper QFX data center switches.
Role in LuxCo-designed and built the cluster with AMD. Supplied ProLiant XD685 servers, Juniper QFX switching and Networking Data Center Director.
Websitehpe.com

We’re tracking how Ethernet fabrics are deployed in AI and HPC clusters. Follow our AI Infrastructure coverage.