Check out full showcase at: https://ngi.fyi/ocpsummit26yt to get the attendee and sponsor perspectives from the show floor of the #OCPGlobalSummit 2025

Sree Ganesan, VP of Product at d-Matrix, presents a groundbreaking live demonstration of heterogeneous compute architecture for large language model inference. Speaking at the Open Compute Project (OCP) event, Ganesan unveils Squadron, d-Matrix’s cluster-scale solution, and showcases what the company claims is the industry’s first live demonstration of attention feed-forward disaggregation running across GPU and specialized hardware.

The demonstration features a 20 billion parameter model achieving impressive throughput metrics by splitting workloads between NVIDIA H100 GPUs and d-Matrix’s Corsair nodes. Ganesan explains the technical rationale behind this architectural approach and how it addresses the memory-bound nature of expert layers in mixture-of-experts models.

– How d-Matrix’s Squadron enables cluster-scale AI inference deployment
– The technical implementation of attention feed-forward disaggregation across heterogeneous compute platforms
– Why separating attention and expert layers optimizes memory bandwidth utilization
– Performance results from running a 20B parameter model on hybrid GPU-Corsair architecture
– How data parallel scaling improves throughput in memory-bound workloads
– d-Matrix’s roadmap for expanding this approach to larger models

Want to be involved our video series? Contact info@nextgeninfra.io
https://ngi.fyi/ocp26-dmatrix-sree