Large language models have a speed problem that goes beyond raw hardware. Even on the fastest GPUs available, the standard autoregressive loop — generate one token, wait, generate the next — leaves most of the chip sitting idle. Speculative decoding was supposed to fix that: a small draft model guesses a run of tokens; the large target model verifies them all in parallel; accepted tokens are kept without touching the output distribution. In practice, the fix has been incomplete. The draft model itself still generates tokens one at a time, capping real-world speedups at roughly 2–3× even with state-of-the-art methods like EAGLE-3.

Researchers Jian Chen, Yesheng Liang, and Zhijian Liu at UC San Diego's z-lab released DFlash on June 24, and the results — confirmed independently by NVIDIA's engineering team the previous day — suggest the field's drafter-side bottleneck may finally have a structural solution. DFlash replaces the autoregressive draft loop with a lightweight block diffusion model that proposes an entire token block in a single forward pass. The target model then verifies that block in parallel, just as in standard speculative decoding, preserving the lossless guarantee: the final output distribution is identical to what the target would have produced on its own.

On NVIDIA Blackwell hardware running TensorRT-LLM, NVIDIA measured DFlash serving more than 15 times the concurrent user load of standard autoregressive decoding at an interactivity target of 500–600 tokens per second per user — roughly 1.5 times the throughput of EAGLE-3 at the same operating point. In UC San Diego's own single-stream benchmarks on Qwen3-8B, DFlash averaged a 4.86× lossless speedup versus a 1.76× average for EAGLE-3 at matched tree size. On MATH-500, DFlash peaked at 6.08×.

Why Diffusion Drafting Changes the Arithmetic

The reason earlier speculative decoding systems plateau around 2–3× speedup is architectural, not incidental. An autoregressive drafter generates tokens left to right: one token, then the next, then the next. Its latency cost grows linearly with the number of speculative tokens it proposes. That growth forces systems like EAGLE-3 to use extremely shallow draft networks — EAGLE-3's drafter is a single transformer layer — because a deeper network would add too much latency to pay back in acceptance rate.

A block diffusion drafter operates differently. It treats the entire proposed token block as a joint denoising problem, starting from masked or noisy placeholders and converging to a coherent sequence in a single forward pass. The latency cost is essentially flat regardless of how many tokens the block contains. That flatness is the economic insight: DFlash can afford a five-layer draft network — eight layers for the Qwen3-Coder variant — because adding depth does not add wall-clock time to the drafting step. A deeper model produces longer accepted runs; longer accepted runs mean fewer target-model forward passes; fewer forward passes means lower latency per generated token.

Earlier attempts to use diffusion models for speculative drafting ran into a different problem: they required massive 7-billion-parameter drafters, which made the drafting cost too high to recover in verification savings. DiffuSpec and SpecDiff-2 both hit this wall, capping speedups near the 3–4× range. DFlash avoids it by keeping the drafter compact while conditioning it heavily on information already computed by the target model.

The KV Injection Insight: Target Models Know What Comes Next

The architectural core of DFlash is an observation about what large autoregressive models already know. At each layer of a target model, the hidden states encode information not just about the current position but about several tokens that come next. This phenomenon — implicitly documented in prior research on predictive coding in transformer networks — means the target model has already done much of the forecasting work that a separate drafter would need to replicate from scratch.

DFlash extracts hidden states from multiple target layers, fuses them into a single compact feature vector, and then injects that vector into the Key and Value projections of every layer in the draft network. This is a meaningful architectural choice. Earlier systems like EAGLE-3 injected target features only at the input embeddings of the drafter. Signal introduced at the input dilutes as it propagates through deeper layers — the deeper the drafter, the weaker the guidance. By wiring target knowledge directly into the KV cache at every draft layer, DFlash keeps that signal intact across the full depth of the drafter.

The result is that acceptance length — the number of draft tokens the target model actually keeps — scales with draft depth in a way earlier methods could not achieve. A five-layer DFlash drafter generating 16-token blocks outperforms EAGLE-3 generating 8-token blocks, and does so with lower latency, not higher.

This finding carries an implication that extends beyond speculative decoding: it demonstrates that a parallel, non-autoregressive process can reliably exploit the target model's own latent representations of future tokens, suggesting that the strict sequential ordering constraint in autoregressive generation may be more broadly relaxable at intermediate stages without sacrificing distributional fidelity.

Two Throughput Numbers That Measure Different Things

DFlash's benchmarks produce two headline figures that are both real and refer to different things. Understanding the distinction matters for infrastructure teams evaluating adoption.

The first is the 4.86× average figure from UC San Diego's own evaluation: single-stream lossless acceleration on Qwen3-8B with greedy decoding, measured across seven tasks including math, coding, and open-ended chat. This number reflects what a single user experiences in reduced latency. The gains are highest on structured reasoning tasks where long, predictable token sequences give the diffusion drafter the most material to work with — 6.08× on MATH-500, 5.62× on AIME-25, 5.15× on GSM8K — and more modest on conversational outputs where sequences are shorter and less repetitive (2.75× on MT-Bench).

The second figure, the one NVIDIA reported on June 23, measures batch throughput at a fixed interactivity target on gpt-oss-120b across eight Blackwell GPUs in a DGX B300 system using TensorRT-LLM. At the 500–600 tokens-per-second-per-user range that defines an interactive experience, DFlash serves more than 15 times the concurrent user load of standard autoregressive decoding — roughly 1.5 times more than EAGLE-3 at the same operating point. On a per-user, fixed-concurrency basis, DFlash averages 2.3× versus EAGLE-3's 1.7× on gpt-oss-120b, and 2.8× versus 2.2× on Llama 3.1 8B Instruct. These figures are not contradictory; they measure latency reduction for a single stream versus throughput expansion for a production serving cluster.

Each NVIDIA Blackwell Ultra GPU combines two reticle-sized dies connected at 10 TB/s of chip-to-chip bandwidth, delivering 15 PFLOPS of dense NVFP4 compute. DFlash's block-parallel drafting exposes more parallel work at each decoding step, making that compute available in a way that standard autoregressive drafting cannot.

What DFlash Supports Right Now

DFlash ships with 20 model checkpoints on Hugging Face covering Qwen, LLaMA, Gemma, Kimi K2.6, and gpt-oss model families, with separate recipes for Blackwell and Hopper hardware. Native integration is available for SGLang, vLLM, and TensorRT-LLM — frameworks that are among the most widely deployed for production LLM serving.

Adoption requires no application-level code changes: an operator replaces an EAGLE-3 checkpoint reference in the serving configuration with a DFlash checkpoint reference and restarts the server. On vLLM, the swap is a single config line. The Hugging Face Transformers backend supports Qwen3 and LLaMA-3.1 models via a spec_generate call. The paper is available at arXiv:2602.06036, with code on GitHub.

Gains compress as concurrency rises, a standard characteristic of speculative decoding systems. At high batch sizes where the target model is already compute-saturated, the per-token verification step dominates and the drafter's marginal contribution shrinks. Infrastructure teams serving latency-sensitive interactive workloads — coding agents, reasoning pipelines, real-time chat — will see the largest returns; teams running batch inference at high utilization will see smaller but still positive gains.

Frequently Asked Questions

How does DFlash differ from EAGLE-3 speculative decoding?

EAGLE-3 drafts tokens autoregressively — one at a time, left to right — using a single transformer layer kept shallow to limit drafting latency. DFlash replaces that sequential process with a block diffusion model that generates an entire token block in a single parallel forward pass. It also injects target model hidden states into the Key and Value projections of every layer of the draft network, not just the input, which keeps the target's knowledge of upcoming tokens active throughout the draft model's full depth. The result is longer accepted runs, lower latency per accepted token, and performance that scales with drafter depth in a way EAGLE-3's approach cannot match.

What does the 15x throughput figure actually mean for a production deployment?

The 15× figure is not a single-user latency improvement — it is a batch throughput measurement on gpt-oss-120b across eight NVIDIA Blackwell GPUs in a DGX B300 system. It means that at a fixed interactivity target of 500–600 tokens per second per user, DFlash can serve more than 15 times as many concurrent users as standard autoregressive decoding while maintaining the same per-user responsiveness. For a single user in isolation, the relevant figure is the 4.86× average lossless speedup measured in UC San Diego's own benchmarks, peaking at 6.08× on MATH-500.

What does DFlash reveal about how transformer models handle sequence prediction?

DFlash's KV injection design relies on a property of large autoregressive models that researchers had noted but not fully exploited in speculative drafting: the hidden states at each target layer encode information about multiple future tokens, not just the current position. DFlash exploits this by feeding target hidden states into every layer of its parallel diffusion drafter. That the approach works — producing accepted runs that outperform a single-layer autoregressive drafter generating half as many tokens — is evidence that the target model's residual stream carries forward-looking predictive information a parallel process can extract without violating the distributional guarantee of the final output.

Does DFlash work with models other than Qwen3?

DFlash ships with 20 checkpoints on Hugging Face covering multiple model families: Qwen3 (including Qwen3-Coder), LLaMA-3.1, Gemma 4 31B, Kimi K2.6, and gpt-oss. Integrations are available for SGLang, vLLM, and TensorRT-LLM, with no application code changes required — operators swap the speculative draft model reference in their serving configuration.