At last week's Open Compute Project Global Summit in San Jose, d-Matrix ran a live inference pipeline that routed attention layers to NVIDIA H100 GPUs and expert feed-forward layers to its own Corsair XPUs — and cut time-per-output-token from 2.59 milliseconds on a standalone H100 to 1.28 milliseconds, a 50.6 percent reduction on the GPT-OSS-20B mixture-of-experts model. That result, delivered on commercially shipping silicon rather than simulation hardware, confirms that Attention-FFN Disaggregation moves from academic design space to production architecture — but only for a specific class of models, and at the cost of a proprietary software stack that operators must weigh before committing.

What Is Attention-FFN Disaggregation?

Transformer-based language models interleave two structurally different computations inside every layer: attention, which reads and updates a growing key-value cache and is bounded by memory bandwidth as context lengthens; and feed-forward networks, which apply fixed weight matrices to each token position and are primarily compute-bound. In a dense model like GPT-3, these two operations run on the same hardware in the same pass. In a mixture-of-experts model, the feed-forward block is replaced by a bank of expert subnetworks — in GPT-OSS-20B's case, 32 experts of which only 4 activate per token — which concentrates the weight-loading bottleneck in the expert layers.

Attention-FFN Disaggregation routes each operation to hardware suited for it: GPUs handle the attention pass, where their HBM bandwidth handles KV-cache loads efficiently; a separate accelerator handles the expert FFN pass, where a different memory architecture removes the bandwidth wall. The concept appeared in research literature as early as 2025, when StepFun's Step-3 model paper proposed it theoretically, and a 2026 academic paper from Georgia Tech, Intel, Google, and Google DeepMind researchers confirmed it provides "interactivity gains under strict latency" service-level objectives for MoE serving. D-Matrix's OCP demonstration was, by the company's account, the first time AFD ran on available silicon rather than simulation infrastructure.

Why SRAM Bandwidth Beats HBM3 for Expert Layers

The case for using a separate accelerator for FFN computation rests on a bandwidth gap that is larger than most coverage of AI chips acknowledges. NVIDIA's H100 draws on HBM3 memory at approximately 3.35 terabytes per second — the fastest high-bandwidth DRAM available in a commercial GPU. Corsair's on-chip SRAM delivers a claimed 150 TB/s, roughly 45 times that figure.

The reason is architectural. HBM stacks DRAM dies on the same package as the processor, but DRAM cells still require periodic refresh and cannot be read at the speed SRAM allows. SRAM stores each bit in a six-transistor flip-flop circuit that toggles without refresh; it accepts all address bits simultaneously. That makes it faster but far less dense: where H100 carries 80 gigabytes of HBM3 on-package, each Corsair card holds 4 gigabytes of SRAM. Bandwidth per bit is far higher; total capacity is far lower.

For the FFN expert pass, the relevant figure is bandwidth, not capacity. Loading the active expert weight tensors for a single inference step at 4-bit precision requires moving roughly 3.6 billion parameters for the four active experts in GPT-OSS-20B — a task that completes faster when the memory bus runs at 150 TB/s. The capacity constraint (4 GB per card) is addressed by distributing expert weights across multiple Corsair cards in a SquadRack. A single 8-server SquadRack carries 64 Corsair cards, giving 256 gigabytes of SRAM performance memory — enough to run models up to 100 billion parameters at reduced precision.

Communication between the H100 servers and Corsair cards runs through d-Matrix's JetStream network interface cards at 400 gigabits per second per cluster. JetStream uses device-initiated memory writes — the Corsair card reads results directly into its own memory without routing through the host CPU — reducing round-trip latency for each attention-to-FFN handoff. For a 24-layer model, that means 24 round trips without CPU per forward pass, all handled directly between the GPU and Corsair cards.

The OCP Demo Results: Numbers That Matter

The live demonstration used GPT-OSS-20B, a publicly released mixture-of-experts model with 20.9 billion total parameters. Of those, 19.12 billion reside in the 32 expert blocks; approximately 3.6 billion are active on any given token. The model ran at 4-bit precision, fitting the active expert weights inside Corsair's SRAM capacity.

Time-per-output-token — the primary latency metric for interactive applications — fell from 2.59 milliseconds on a standalone H100 at 1,000-token context to 1.28 milliseconds under the AFD configuration, a reduction of 50.6 percent. A separate throughput measurement showed 845 tokens per second per user at interactive quality-of-service thresholds; d-Matrix's materials also report 20,415 tokens per second across 64 concurrent users. Note that 845 multiplied by 64 equals approximately 54,000 tokens per second rather than 20,415, which suggests the two figures were measured under different concurrency and batching configurations rather than describing the same single test — readers should treat them as performance envelopes from distinct scenarios rather than a consistent benchmark.

D-Matrix also reported a 10-times speedup in LM-head processing — the final projection from model hidden states to vocabulary logits — achieved by sharding LM-head weights across cards in parallel.

Is AFD Only Useful for Mixture-of-Experts Models?

This is the constraint the announcement materials underplay. The Georgia Tech/Google academic paper that validated AFD's theoretical basis was explicit: the technique "excels under strict latency service-level objectives for MoE serving" but "throughput-focused workloads often favor aggregated chunked-prefill approaches." More significantly, AFD provides no benefit for dense transformer architectures.

The reason is that MoE provides a natural partition boundary. Each expert FFN block is invoked independently and asynchronously; its inputs arrive from the attention pass and its outputs return to the residual stream before the next layer's attention begins. That structure maps directly onto a GPU-to-XPU communication model. In a dense transformer — where the FFN is fully connected and operates on every token in every layer — the communication overhead of shipping activations to a separate accelerator and waiting for results eliminates the bandwidth advantage.

As of mid-2026, the most capable public frontier models — including several of the largest open-weight systems — use MoE architecture. But the trajectory of model design is not fixed. An operator who builds data-center capacity around the GPU+Corsair AFD stack is making a bet that MoE architectures remain dominant and that their model portfolio does not shift toward dense designs. That bet may prove correct, but it is a bet.

There is also a software integration cost. D-Matrix's AFD pipeline requires its Aviator compiler-to-runtime stack and JetStream NICs; neither is interchangeable with the standard CUDA/cuDNN toolchain or commodity RoCE networking. Operators also inherit a proprietary hardware-software stack that becomes harder to replace if model architectures shift or if d-Matrix's roadmap diverges from their workload needs.

What Squadron Looks Like in Practice

Squadron is d-Matrix's full-cluster product built on top of the SquadRack building block it first announced at OCP Summit 2025. At maximum configuration, Squadron aggregates 16 SquadRacks — each an 8-server rack — for a total of 1,024 Corsair XPUs. The platform is air-cooled; single servers draw under 10 kilowatts, far below the 120 kW or higher figures for next-generation GPU racks that require liquid-cooling or immersion infrastructure. Operators can deploy into existing data-center facilities without retrofitting cooling systems, though a full 16-rack cluster still demands megawatt-scale electrical capacity.

Squadron was developed with Arista (Leaf Ethernet switching, connected to JetStream NICs), Broadcom (PCIe switches for within-node scaling), and Supermicro (X14 AI Server Platform and systems integration). Supermicro's president and CEO Charles Liang emphasized the manufacturing dimension: "Bringing next-generation AI infrastructure to market at scale requires deep engineering, systems integration, and manufacturing expertise. Our collaboration with d-Matrix demonstrates Supermicro's ability to deliver innovative AI technologies through our proven Data Center Building Block Solutions." Supermicro launched Super Micro Federal LLC in October 2025 to serve US government AI server deployments — a market that may align with d-Matrix's sovereign-entity customer target.

Squadron is now shipping to priority customers for deployment in their own data centers. D-Matrix has deployed a Squadron cluster in its own Arizona facility for internal research. Corsair servers and SquadRacks are generally available. D-Matrix's next-generation Raptor XPU, planned for late 2027, is expected to extend the platform's per-card performance further.

Parasail and the Path to Production

The clearest independent signal of Squadron's commercial trajectory came in July 2026, when AI deployment company Parasail announced it was running a Corsair-plus-NVIDIA-GPU hybrid configuration in production — with NVIDIA GPUs handling prefill and Corsair hardware handling decode. Parasail cited "up to 10 times faster interactive inference for select workloads" and claimed three-times better energy efficiency. Those figures are vendor-stated and derived from early production data rather than independent benchmarks; production results at scale remain pending.

Parasail's deployment nevertheless matters as a corroboration mechanism. The AFD academic paper relied on simulation rather than live commercial silicon; Parasail's real-world deployment of the same architectural pattern — GPU for prefill, XPU for decode — provides a parallel production data point, separate from d-Matrix's own demonstration. D-Matrix plans to scale AFD configurations to larger MoE models including GLM 5.x and Kimi K3 in near-term deployments.

The company completed a $275 million Series C funding round in November 2025 at a $2 billion valuation, bringing total funding to $450 million. Earlier in 2026, d-Matrix acquired GigaIO's data center assets to accelerate rack-scale deployment capabilities.

Frequently Asked Questions

What is Attention-FFN Disaggregation, and is it different from standard GPU inference?

Standard GPU inference runs both the attention and feed-forward network computations inside the same hardware. Attention-FFN Disaggregation splits those operations: attention layers, which are bounded by memory bandwidth as the key-value cache grows, stay on the GPU; expert FFN layers, which load large weight matrices, move to a separate accelerator with higher memory bandwidth. The architectural argument is that each operation runs on hardware optimized for its actual bottleneck rather than a general-purpose compromise. D-Matrix's implementation uses NVIDIA H100 GPUs for attention and its Corsair XPU — with 150 TB/s on-chip SRAM bandwidth versus the H100's 3.35 TB/s HBM3 — for the expert FFN pass. The technique only delivers its latency benefit for mixture-of-experts models, where the FFN and attention blocks are structurally independent per token; it does not work for dense transformers.

Does AFD require operators to replace their existing GPU infrastructure?

No — that is the central commercial proposition. AFD positions Corsair XPUs as an addition to an existing GPU deployment rather than a replacement. Operators keep their H100 servers for attention and prefill; Corsair racks handle the expert FFN decode. D-Matrix has also integrated Corsair into NVIDIA's MGX rackscale architecture via NVLink Fusion for heterogeneous deployments. The adoption cost is real, however: it requires d-Matrix's proprietary Aviator software stack, JetStream NIC hardware, and a willingness to operate a heterogeneous cluster management environment that does not map to the standard CUDA toolchain.

How do the 845 tokens-per-second-per-user and 20,415 total-tokens-per-second figures relate to each other?

They do not multiply cleanly, and readers should not treat them as a single consistent benchmark. At 64 concurrent users, 845 tokens per second per user would imply roughly 54,000 total tokens per second — not 20,415. D-Matrix has not publicly clarified the discrepancy; the two figures were most likely measured under different concurrency levels, hardware ratios, or batching configurations during different phases of the OCP demonstration. The TPOT comparison — 2.59 milliseconds on a standalone H100 versus 1.28 milliseconds with AFD — is a more directly comparable figure drawn from the same 1,000-token context test.

What models does Squadron currently support, and what is planned?

The OCP demonstration used GPT-OSS-20B, an open-weight mixture-of-experts model. D-Matrix has named GLM 5.x and Kimi K3 as near-term AFD scale-out targets. A single SquadRack (64 Corsair cards, 256 gigabytes of SRAM performance memory) can run models up to approximately 100 billion parameters at reduced precision; the full 16-rack Squadron cluster at 1,024 cards extends capacity into the hundreds of billions. The company's next-generation Raptor XPU, planned for late 2027, is expected to raise per-card performance for future models.