Model-Preserving Adaptive Rounding | Summary

Version note: This retrospective review of a paper selected for 2025-W22 uses arXiv:2505.22988v3, dated 3 June 2026, including its ICML 2026 revision. It does not describe only the version available in May 2025.

This article explains the key points of Model-Preserving Adaptive Rounding.

• 2025-05-29 (arXiv), Proceedings of the 43rd International Conference on Machine Learning, PMLR 306, 2026
• Tseng, Albert, Sun, Zhaofeng, De Sa, Christopher.
• Department of Computer Science, Cornell University
• Paper
• Github

Read this article in Korean

Summary

• Compressing a large language model (LLM) into low-precision weights can change its output probability distribution even when individual layers retain similar activations. Yet Another Quantization Algorithm (YAQA) reports approximately 30% lower Kullback–Leibler (KL) divergence, a measure of the difference between output distributions, than GPTQ/LDLQ, equivalent methods that round weights using local activation error, with the same quantizer and preprocessing and no additional finetuning.
• YAQA extends adaptive rounding, which selects quantized weights using feedback from rounding errors, with information about both input and output channels. It uses a Kronecker-factored Hessian approximation: the Kronecker product of two smaller matrices approximating the matrix of second derivatives of output divergence. This incorporates the effect of later layers while keeping rounding tractable.
• Experiments cover Llama, Gemma, and Qwen models with multiple quantizers, the rules defining representable weight values, and optional recovery finetuning, which adjusts remaining unquantized parameters to compensate for earlier quantization. The most consistent result is distribution preservation: downstream accuracy does not improve in every condition, and the theoretical guarantee bounds a single layer’s second-order approximation to output divergence.

1. Introduction

Quantization separates the choice of representable values from the rounding algorithm that assigns original weights to those values. The paper targets the latter: minimizing immediate layer activation error can miss how subsequent layers transform that error, so YAQA constructs rounding feedback from curvature associated with divergence at the model output.

2. Background

The background connects weight quantization to structured curvature approximations used in optimization. It distinguishes approximating local activation reconstruction from approximating preservation of the original model’s complete output distribution.

2.1. Large Language Model Quantization

Post-training quantization (PTQ) compresses an already trained model, whereas quantization aware training (QAT) incorporates quantized representations into training. The paper also describes intermediate approaches that optimize quantized weights or assignments within restricted parameter spaces.

• 2.1.1. LOCAL LAYERWISE ROUNDING: GPTQ and LDLQ use the input activation second moment, the expected outer product of layer inputs, and feedback along input channels. These statistics require only forward passes. Incoherence measures how uniformly important rounding directions are distributed; random orthogonal preprocessing makes these directions less concentrated.
• 2.1.2. FULL MODEL ADAPTIVE ROUNDING: Methods including SqueezeLLM and GuidedQuant incorporate the empirical Fisher, a gradient second-moment matrix computed from next-token task loss. The paper distinguishes this matrix from the curvature of divergence to the original model and questions the training-convergence assumptions used to relate them.
• 2.1.3. GRADIENT-DESCENT-BASED METHODS: PV-Tuning learns codebooks, sets of representable weight vectors, and their assignments; DiscQuant updates weights within a constrained subspace; and cross-block quantization (CBQ) uses trainable adapters and rounding regularization. The paper argues that these methods generally restrict practical choices to scalar quantizers, which represent weights individually, or unstructured vector quantizers, which represent groups of weights.

2.2. Hessian Approximations

The Hessian of divergence to the original model equals the real Fisher, whose gradient samples use targets drawn from that model’s output distribution. This differs from the empirical Fisher based on observed targets. The paper draws on KFAC, an optimizer using Kronecker-factored approximate curvature, and Shampoo, an optimizer using matrix-valued gradient statistics, to motivate separate input-side and output-side factors.

3. Yet Another Quantization Algorithm

YAQA combines a generalization of LDLQ rounding with two scalable Hessian sketches, structured estimates of the otherwise prohibitively large curvature matrix. It rounds layers separately while using information about full-model output error. Because it changes weight assignments within an existing quantized representation, the chosen quantizer continues to determine inference efficiency.

3.1. End-to-End Layerwise Adaptive Rounding

3.1.1. WHEN IS ADAPTIVE ROUNDING FAST? The paper introduces structural nilpotence degree (SND), the dependency depth induced by the nonzero pattern of triangular rounding feedback. For an m-by-n weight matrix, a dense approximation can require mn serial steps, whereas Kronecker factors permit O(m+n) steps and efficient matrix multiplications.

• Equation 6 adds output-channel feedback and an interaction between input-channel and output-channel feedback to LDLQ’s input-channel correction.
• The paper reports that rounding itself takes roughly twice as long as LDLQ in practice. This cost is separate from collecting curvature statistics and introduces no additional inference operations for a fixed quantizer.

3.1.2. END-TO-END BOUNDS & HESSIAN DESIDERATA: Theorem 3.4 bounds the true second-order error using the approximation’s cosine similarity to the Hessian, a measure of directional agreement, together with quantizer error and properties of both factors. Higher directional agreement reduces the mismatch term; low incoherence and suitable low-rank structure, meaning that relatively few directions dominate a factor, improve other terms in the bound.

• LDLQ is recovered by using an identity output factor and the local input factor. The comparison of trace-based bounds favors YAQA under the factor and rank conditions derived in the paper.
• The theorem assumes positive definite factors, whose quadratic forms are positive for nonzero vectors, and nearest or stochastic rounding with bounded mean squared error. It concerns the quadratic approximation obtained by changing one layer while leaving the other weights at their original values.

3.2. Scalable Near-Optimal Kronecker Factored Hessian Sketches

3.2.1. HESSIAN SKETCH A assumes token independence within a linear layer and applies power iteration, repeated updates toward the best-fitting factors, to this approximate curvature. This introduces bias but reduces estimation variance by treating tokens as samples; initialization uses LDLQ’s input statistics and an identity output factor.

• Sketch A converges empirically within 3 full iterations. The reported reference cost is around 20 GPU-hours, summed graphics-processor runtime, for a 10B-parameter model and 20M tokens.

3.2.2. HESSIAN SKETCH B computes one round of power iteration from identity factors using gradients over entire sequences, retaining dependencies between tokens. Both factors are collected in one dataset pass; the reference cost is around 30 GPU-hours for a 10B model using 64K sequences of 2K tokens.

• 3.2.3. EVALUATING SKETCH A AND B: Figure 1 shows decaying output-factor spectra, Figure 2 shows how random orthogonal preprocessing changes the trace-based bound, and Figure 3 shows stronger Hessian alignment than LDLQ’s approximation.
• Sketch B generally benefits from more sequences, but Sketch A can have higher alignment when limited data makes Sketch B’s estimation variance outweigh Sketch A’s bias.

The output-factor spectra decay across three selected layers and models, supporting the approximate low-rank structure used to motivate tighter rounding bounds. Similar spectra for Sketch A and Sketch B do not establish equal alignment with the full Hessian.

Ratios below 1 indicate that randomized Hadamard preprocessing reduces the trace-based component of the error bound. The cumulative distributions, which show the fraction of layers below each ratio, indicate that most plotted layers benefit, although the 70B Sketch B curve includes a noticeable fraction above 1.

The selected layers show substantially better curvature alignment for both YAQA sketches than for LDLQ’s approximation. The left and center panels rescale alignment so that the largest plotted value is 1, rather than reporting absolute cosine similarity. Higher agreement between the sketches in query and key projections, which form attention scores, is consistent with Sketch A’s treatment of token dependencies.

4. Experiments

The main experiments evaluate weight-only PTQ, meaning that weights are compressed while activations remain unquantized, principally without additional finetuning. YAQA-A uses Sketch A with 8K context, 20M tokens, and 3 full power-iteration rounds; YAQA-B uses Sketch B with 2K context and 64K sequences.

• Evaluation measures output KL divergence, perplexity, a measure of predictive fit to target tokens, and zero-shot accuracy, performance without task demonstrations. Comparisons include LDLQ, GuidedQuant, DiscQuant, and Google’s Gemma 3 QAT model.
• The paper prioritizes KL because matching target-token likelihood or benchmark accuracy does not establish that two models have similar complete output distributions.

4.1. Second Order Quantization Algorithms

Table 1 compares LDLQ, GuidedQuant, and YAQA using QTIP, a quantizer based on trellis-structured representations, with incoherence processing and no finetuning. At 2 bits on Llama 3.1 8B Instruct, YAQA-B reduces Wikitext2 KL from 0.356 to 0.241, reduces perplexity from 9.39 to 8.39, and raises average zero-shot accuracy from 62.51 to 64.32.

• YAQA-B has the lowest KL in every model and bit-width condition shown in Table 1. Accuracy rankings vary: at 4 bits on Llama 3.2 3B Instruct, GuidedQuant reaches 64.10 average accuracy while YAQA-B reaches 63.31.
• These entries support a consistent distribution-preservation advantage. They do not support the source section’s blanket claim of superiority across all metrics.

Keeping QTIP and preprocessing fixed isolates the effect of rounding and curvature estimation. YAQA-B has the lowest KL throughout the displayed conditions, but average zero-shot accuracy sometimes favors GuidedQuant, LDLQ, or YAQA-A.

4.2. First Order Quantization Methods

With INT4, a 4-bit integer quantizer, Table 2 shows that YAQA also improves a standard scalar representation: on Llama 3.1 8B Instruct, YAQA-A has KL 0.028 and average zero-shot accuracy 69.11, compared with DiscQuant’s 0.061 and 67.68. YAQA-A also slightly outperforms YAQA-B on KL, perplexity, and average accuracy in this condition.

• Table 3 compares 4-bit YAQA PTQ with a 4.5-bit Google QAT Gemma 3 12B Instruct model. YAQA-B has lower KL, 0.056 versus 0.089, but QAT has higher average accuracy, 70.83 versus 69.90, and lower Wikitext2 perplexity, 7.56 versus 7.94.
• The Gemma result supports better preservation of the original distribution. The authors estimate that QAT used 100× more data; this is an estimate, and the compared representations have different effective bit widths.

YAQA’s advantage also appears with scalar INT4 quantization: YAQA-A improves KL, perplexity, and average accuracy over both LDLQ and DiscQuant. Its slight advantage over YAQA-B shows that Sketch B does not dominate in every experimental condition.

4.3. Finetuning

Table 4 tests recovery finetuning with QTIP and incoherence processing. Finetuning reduces the gap between LDLQ and YAQA, while YAQA-B retains lower KL across the reported Llama 3.1 8B and 70B conditions.

• At 2 bits on the 8B model, KL decreases from LDLQ’s 0.185 to YAQA-B’s 0.147, with average zero-shot accuracy increasing from 65.44 to 66.38. YAQA-A reaches a higher accuracy of 67.54 in that same condition.
• At 4 bits on the 70B model, YAQA-B reaches KL 0.029 versus LDLQ’s 0.036. The results demonstrate compatibility with finetuning, while downstream rankings remain condition-dependent.

YAQA-B retains lower KL than LDLQ after recovery finetuning across both model sizes and all displayed bit widths. Average accuracy has no consistent ordering, so the distributional advantage and downstream gains require separate assessment.

5. Conclusion

The conclusion connects tractable rounding dependencies, Kronecker-factored curvature, and scalable estimation into a method for preserving model outputs. It reports approximately 30% lower KL than GPTQ/LDLQ and no added inference overhead, with the latter applying to the rounding algorithm under an unchanged quantizer.

Impact Statement

The Impact Statement describes the work as advancing machine learning and acknowledges potential societal consequences without identifying a specific consequence for further discussion. It provides no separate empirical assessment of societal impact.

Appendix

• A.1. Experimental Setup and Implementation Details: Curvature statistics use RedPajama v1, a text calibration dataset, and KL and perplexity evaluation use the OPTQ Wikitext2 and C4 dataset splits. The appendix specifies sequence length 8192 for these evaluations and chat templates for zero-shot evaluation; Tables 8 and 9 explicitly label the Qwen evaluations as 4K context. Llama 3.1 70B Instruct leaves the 0_v layer unquantized. The Gemma comparison uses google/gemma-3-12b-it-qat-q4_0-gguf, dequantized with Transformers 4.52.0.dev0. Code is provided at https://github.com/Cornell-RelaxML/yaqa-quantization.
• A.2. Hessian Sketch Memory and Compute Requirements: For an m-by-n weight matrix, storing both factors requires O(n²+m²) memory, roughly twice the Hessian storage of input-only methods in the reported setting. The authors report numerical stability with FP32, 32-bit floating-point arithmetic, and small diagonal regularization, but not with TF32, the alternative computation format tested. Sketch B is empirically slower because it needs more sequences for acceptable variance.
• A.3. GuidedQuant as LDLQ interprets GuidedQuant as separate LDLQ computations over output-channel groups, with serial depth n but arithmetic increasing with the number of groups. A.4. KL Divergence vs. Perplexity illustrates the distinction using Llama 1 7B and Llama 2 7B: Wikitext2 perplexities are 5.68 and 5.47, while their KL divergence is 0.197, much larger than the log perplexity ratio of 0.038.
• A.5. Incoherence Processing with the Random Hadamard Transform describes randomized sign changes followed by a Hadamard transformation, a fast orthogonal mixing operation applied to weights and curvature. A.6. YAQA Rounding Algorithm supplies fixed-point pseudocode, which repeatedly applies the rounding update until unchanged, and an iterative Python implementation that the authors describe as faster for slower quantizers.
• A.7. Proofs derives convergence from rounding dependency graphs and establishes scalar and block-quantizer error bounds before transferring the proxy bound to true curvature through cosine similarity. A.8. Modified Pytorch Backward Pass to Compute Sketch A and A.9. Modified Backward Pass to Compute Sketch B provide distributed implementations for collecting the corresponding factors.
• A.10. Additional Results includes per-task zero-shot results, comparisons with CBQ, experiments without incoherence processing, and Qwen 3 8B evaluations. YAQA-B improves most displayed perplexity results against CBQ, but at INT4 its Wikitext2 perplexity is 5.54 versus CBQ’s 5.52; the PV-Tuning comparison is indirect through previously reported QTIP results. Without incoherence processing, YAQA-B reduces KL from 0.033 to 0.021 using INT4 with a 16-bit scale per 32 contiguous weights, giving 4.5 effective bits per weight. On 2-bit Qwen 3 8B with QTIP and no finetuning, KL falls from 0.285 to 0.205; at 4 bits, MMLU, the reported zero-shot knowledge benchmark, favors LDLQ with 73.04 accuracy versus YAQA-B’s 72.78.
• A.11. Number of Sequences Needed for Sketch B tests 2-bit Qwen 3 8B from 2K to 64K calibration sequences. With 2K sequences and 1 GPU-hour, YAQA-B achieves KL 0.227 versus LDLQ’s 0.285 using 4K sequences and 1.5 GPU-hours. Increasing YAQA-B to 64K sequences and 30 GPU-hours lowers KL to 0.205, although task accuracy does not improve monotonically with sequence count.

Brief Thoughts

The main contribution is the connection between tractable rounding feedback and an explicit curvature-approximation objective. Matched-quantizer experiments provide consistent evidence for lower KL, while the Gemma comparison shows that closer output distributions and higher benchmark accuracy are distinct outcomes. Deployment decisions should account for backward-pass calibration and extra curvature storage; the theorem bounds a single-layer quadratic approximation rather than the actual divergence of the fully quantized model.