A new post-training quantization framework eliminates the calibration dataset long required to deploy large language models at 4-bit precision — and it does so at a moment when hardware support for FP4 inference is converging across three major accelerator lines at once.

The method, called DualQuant, was posted to arXiv on October 9 by researchers affiliated with d-Matrix, a Santa Clara AI inference chip startup. The framework targets the same W4A4 inference problem as SmoothQuant — compressing both weights and activations to 4-bit precision simultaneously — but requires no representative input data at quantization time. On a real packed W4A4 inference path for Llama-3.1-8B running on an RTX 4090, DualQuant delivered 2.61x memory reduction benchmark results and up to 2.46x throughput speedup over a BF16 baseline while matching SmoothQuant's accuracy. The news comes as the Open Compute Project's annual Global Summit — the standards-setting body for the MXFP4 format that DualQuant targets — runs this week in San Jose.

Why This Week Is the Right Moment

The Open Compute Project Global Summit runs October 12–15, 2026, in San Jose, and it is the institutional home of the Microscaling (MX) format standard — the open specification that defines MXFP4, the 4-bit block floating-point format now supported by AMD's Instinct MI350 series and Intel's forthcoming Xe3P data center GPU. NVIDIA's competing 4-bit format, NVFP4, is natively implemented in Blackwell NVFP4 Tensor Core specs that power the B100, B200, and GB200 series.

Both formats are already in active production deployment. NVIDIA has published NVFP4 checkpoints for DeepSeek models including V3.2 and V4, with DeepSeek V4 Blackwell NVFP4 performance reaching approximately 3,500 tokens per second on Blackwell hardware. AMD's MI355X surpassed one million tokens per second in MLPerf Inference 6.0. Intel's Xe3P architecture — detailed at Hot Chips 2026 in August — supports FP4 and MX microscaling with 480GB of LPDDR5X memory.

The quantization software, however, had not caught up. Until DualQuant, no dual-scaling method was formally designed to handle the hierarchical block structure that both MX and NV formats use — and that structural mismatch had practical consequences for MXFP4 power-of-two scaling accuracy.

Activation Outliers and the Calibration Tax

Post-training quantization LLM overview — compressing a model's numerical precision after training rather than during it — has become the dominant technique for LLM deployment at scale. Reducing weights from 16-bit to 4-bit cuts memory by roughly 4x and accelerates inference on hardware with native low-precision arithmetic support. The problem is that reducing weights alone does not capture the full efficiency potential of FP4-capable hardware.

Today's leading accelerators are built around the expectation that both weights and activations arrive in low precision. When only weights are quantized, activations still flow through the system in BF16 representation, leaving significant bandwidth and throughput gains unrealized. The trouble is that activations are far harder to quantize than weights. Unlike weights, which are fixed at inference time and can be characterized offline, activations vary with every token processed and tend to show sharp spikes—"outliers"—in particular channels that resist compression to 4-bit formats.

SmoothQuant (MIT/NVIDIA), the state of the art, addressed this by offline-migrating quantization difficulty from activations to weights via a per-channel scaling transformation. The transformation is computed from a small representative calibration sample — typically 512 to 2,048 input sequences run through the model before deployment. The approach works well enough that it has been incorporated into SmoothQuant TensorRT-LLM production integration and Amazon SageMaker. But it carries a hidden operational cost: for every new model variant, fine-tune, or deployment context, operators must maintain a curated calibration dataset, manage its versioning, and accept the risk that a dataset tuned to one distribution may subtly distort accuracy in another.

DualQuant eliminates that operational dependency entirely.

The Dual-Scale Insight

DualQuant's central contribution is the observation that solving the weight-quantization problem already generates the information needed to solve the activation problem — with no external data required.

The framework casts quantization as a non-differentiable weight-reconstruction problem: find per-channel scaling factors that minimize the mean-squared error between the original floating-point weights and their quantized representations. This formulation is standard for weight-only quantization methods. The new insight is that the optimal scaling factors for weights carry, as an immediate by-product, useful rescalings of the corresponding activation channels, as described in the DualQuant weight reconstruction MSE formulation.

Mathematically, weights and activations interact at every matrix multiplication in a transformer layer. A scaling factor that balances a weight channel's numerical range against the quantization grid implicitly encodes information about the co-occurring activation magnitudes. By reading per-channel activation rescalings from the weight-reconstruction solution, DualQuant can smooth activation outliers without seeing a single activation at runtime.

To keep the optimization tractable, the authors use variable splitting and block coordinate descent, with each subproblem solved by a closed-form DualQuant solver — avoiding the iterative, gradient-based procedures that make some methods expensive to apply. The entire offline computation runs on model weights alone, with no dataset required.

Why MXFP4 Is a Harder Target Than It Looks

Research presented at ICLR 2026 established that MXFP4's power-of-two block scaling severely degrades accuracy in a way that prior calibration-based methods were not designed to address. MXFP4 uses 32-element blocks with a shared scale encoded in E8M0 format — meaning the shared exponent is rounded to a power of two. This is intentionally cheaper to implement in hardware — the OCP MX standard achieves roughly 12% relative area savings in tensor cores compared to NVFP4 — but it introduces additional quantization error that simple scaling strategies cannot recover from.

NVFP4, NVIDIA's proprietary format used in Blackwell Tensor Cores, uses 16-element blocks with an E4M3 shared scale that requires a real floating-point multiply — costlier in hardware but more accurate. The two formats are built on different block structures with different scale granularities, and prior dual-scaling methods including SmoothQuant were designed around tensor-level or per-channel quantization schemes that do not map cleanly onto either.

DualQuant's formulation parameterizes the weight-reconstruction problem to match each target format's specific block structure. The authors claim it is the DualQuant first MX NVFP4 method — the first dual-scaling method to formally support both the full MX family and NVIDIA's NVFP4.

Results and a Critical Benchmark Caveat

The researchers evaluated DualQuant across Llama-3.1-8B, Llama-3.1-70B, and Mixtral-8x7B on standard DualQuant benchmark evaluation results, including WikiText-2 perplexity, C4 perplexity, and zero-shot downstream tasks. Across most models and formats, DualQuant matched or exceeded SmoothQuant's accuracy despite having no access to calibration data, and it substantially outperformed other calibration-free baselines. The largest accuracy gains appear specifically on MX and NV block formats — consistent with the claim that the algorithm's format-native formulation gives it an advantage precisely where prior methods were not designed for the block structure.

The hardware validation on the RTX 4090 Ada Lovelace specs delivered 2.61x memory reduction and 2.46x throughput. A clarification is warranted for practitioners targeting Blackwell or AMD MI350: the RTX 4090 is an Ada Lovelace–architecture GPU (2022) that lacks hardware-native FP4 Tensor Cores. The benchmark implements a custom packed W4A4 integer kernel that packs 4-bit values into 8-bit or 16-bit registers — a genuine 4-bit computation path, but via integer packing rather than the E2M1 or E4M3 Tensor Core instructions that Blackwell and MI350 hardware execute. The accuracy parity between this real kernel and the paper's simulated ("fake") quantization results is meaningful — it confirms the algorithm works in hardware — but practitioners should be aware that throughput benchmarks on Blackwell or MI350 with native FP4 Tensor Cores were not reported in the paper. Those results, when they come, may differ.

What Calibration-Free Actually Means for Deployment

Eliminating calibration data has more practical significance than benchmarks alone suggest. Calibration creates a hidden operational lifecycle: for every new model variant, fine-tune, or deployment context, operators must assemble representative inputs, run the calibration pass, store the resulting scaling statistics, and track them through the model lifecycle. Overfitting also poses a risk: scaling factors tuned on one prompt distribution may produce subtly degraded accuracy on another, an error mode that is hard to catch without extensive evaluation.

A calibration-free method collapses this to a single offline computation on model weights alone — no dataset, no environment-specific tuning, no versioning. That simplification matters most for teams deploying models that are frequently fine-tuned (where recalibrating after every fine-tune is expensive), for domains where representative input data is proprietary or scarce, and for any workflow targeting multiple hardware backends at once, since DualQuant's single algorithm covers both MXFP4 and NVFP4 targets simultaneously.

As a minor note: quantization in general — not specifically DualQuant — has been documented to subtly affect LLM safety alignment properties at lower bit widths, per a quantization LLM safety alignment study. Practitioners deploying any 4-bit quantized model in safety-sensitive applications should re-evaluate alignment properties after quantization, regardless of the method used.

Frequently Asked Questions

What is calibration data, and why does it matter for LLM quantization?

Calibration data is a small representative set of input sequences — typically a few hundred to a few thousand samples — that quantization methods like SmoothQuant feed through a model before deployment to measure the statistical distribution of activation values. Those statistics are then used to compute per-channel scaling factors that make activations easier to compress. Without calibration data, most combined weight-and-activation quantization methods produce significantly degraded accuracy. DualQuant's contribution is showing that scaling factors derived from weight reconstruction alone—with no activation data needed—achieve equivalent smoothing.

What is the difference between MXFP4 and NVFP4?

MXFP4 is an open standard defined by the Open Compute Project's Microscaling (MX) specification, supported by AMD, Intel, Qualcomm, Microsoft, and others. It uses 32-element blocks with a power-of-two shared exponent (E8M0), which is efficient to implement in hardware but introduces more quantization error than alternatives. NVFP4 is NVIDIA's proprietary variant used in Blackwell Tensor Cores; it uses 16-element blocks with an E4M3 shared scale that requires a real multiply and achieves higher accuracy at slightly higher hardware cost. DualQuant is the first dual-scaling method formally designed to cover both.

Does 4-bit quantization hurt LLM accuracy?

With well-designed quantization, the accuracy hit at W4A4 precision can be made small. DualQuant's paper reports matching SmoothQuant — the calibration-based state of the art — on most tested models including Llama-3.1-8B, Llama-3.1-70B, and Mixtral-8x7B, while substantially outperforming calibration-free baselines. Research from Meta presented at ICML 2026 found that naively applied MXFP4 quantization degrades accuracy by around 10% on average relative to BF16 baselines; DualQuant's format-native formulation is designed specifically to address that gap.

What hardware can currently run W4A4 inference with hardware-native FP4 support?

NVIDIA Blackwell GPUs (B100, B200, GB200) support NVFP4 natively in their Tensor Cores. AMD Instinct MI350 series (CDNA4 architecture) supports MXFP4 natively. Intel Xe3P (Crescent Island), detailed at Hot Chips 2026, supports both FP4 and MX microscaling. The RTX 4090 benchmark in DualQuant's paper uses a custom integer-packing kernel rather than hardware-native FP4 Tensor Cores, as Ada Lovelace predates FP4 Tensor Core support.