Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps | Summary

This article explains the key points of Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps.

• 2025-01-16 (arXiv)
• Ma, Nanye, Tong, Shangyuan, Jia, Haolin, Hu, Hexiang, Su, Yu-Chuan, Zhang, Mingda, Yang, Xuan, Li, Yandong, Jaakkola, Tommi, Jia, Xuhui, et al.
• NYU, MIT, Google
• Paper

Read this article in Korean

Summary

• Diffusion models, generators that turn noise into images through repeated denoising, obtain diminishing quality gains from simply adding denoising steps. On DrawBench, a text-prompt evaluation benchmark, the FLUX.1-dev generator with ensemble-guided random search raises the LLM Grader score, a large language model’s image-quality assessment, from 84.29 with 30 denoising evaluations and no search to 88.18 with 2880 additional search evaluations. The ensemble combines three candidate-scoring models, and computation is counted in number of function evaluations (NFEs), meaning generator forward evaluations.
• The framework separates verifiers, models or metrics that score generated candidates, from search algorithms. Random Search selects among independent candidates, Zero-Order Search refines neighboring initial noises without gradients, and Search over Paths branches and selects intermediate denoising trajectories. Experiments on class-conditioned generation, which uses target class labels, on the ImageNet labeled-image benchmark and on text-conditioned generation find that the best combination depends on the task and evaluation criteria.
• Additional search can amplify verifier bias: improving individual image scores can reduce diversity across generated samples or degrade another quality criterion. Local search, which restricts candidate changes, partly mitigates diversity loss on ImageNet. Combining verifiers improves broad evaluation on DrawBench but remains inferior to ImageReward, a learned human-preference scorer, alone on every reported category of T2I-CompBench, a benchmark for generating specified attributes and object relationships.

1. Introduction

The paper treats inference-time computation as an additional resource available after training. Because initial noise and any subsequently injected noise influence the generated image, search can spend this resource on finding preferable randomness after gains from more denoising steps have plateaued.

• The proposed design space has two axes: how candidate quality is measured and how new candidates are found.
• The experiments progress from ImageNet class conditioning to richer text conditioning, where multiple evaluators expose competing notions of image quality.

The four curves show that allocating additional generator evaluations to search continues to improve the plotted ImageNet and DrawBench metrics after increasing denoising evaluations alone produces little benefit. The panels measure different quality criteria and do not establish one universally optimal search configuration.

2. Background and Motivation

Diffusion models and the related flow-based generators considered in the paper learn transformations that reverse a reference noising process. Sampling starts from Gaussian noise and follows an ordinary differential equation (ODE), a deterministic evolution, or a stochastic differential equation (SDE), an evolution with additional randomness. The learned model approximates the vector field, the direction of sample movement at each noise level, used to reach clean data.

• Increasing denoising computation has diminishing returns because both model approximation and numerical discretization, the use of finite solver steps, affect sampling quality.
• Once the noise inputs and sampling procedure are fixed, the generated image is determined, making the noise inputs suitable search variables.

3. How to Scale at Inference Time

A verifier maps an image and optional conditioning information to a scalar score; a search algorithm uses that score and the pretrained generator to select sampling noise. The ImageNet demonstration uses SiT-XL, a pretrained image generator, at 256 × 256 resolution with a second-order Heun numerical solver and 250 NFEs for final generation. Unless otherwise specified, classifier-free guidance (cfg), a mechanism for strengthening conditioning during sampling, has weight 1.0, corresponding here to conditional generation without additional guidance.

• Random Search generates images from independent Gaussian noise candidates and selects the highest-scoring candidate, implementing best-of-N selection: retaining the best of N candidates.
• Oracle Verifiers use privileged evaluation information. Inception Score (IS), a metric based on classifier predictions, is targeted through confidence in the conditioning class; Fréchet Inception Distance (FID), a distance between generated and reference image-feature statistics, is targeted through greedy candidate-batch selection against ImageNet statistics.
• The oracle experiments demonstrate improvements from noise selection, but their access to final evaluation information makes them proof-of-concept results.

The diagram distinguishes independent candidate selection from local refinement of initial noise and refinement of intermediate trajectories. These restrictions on exploration help interpret the algorithms’ different behavior when maximizing individual-image scores reduces diversity.

Oracle-guided Random Search initially improves the targeted FID and IS across the tested guidance weights, with FID eventually flattening or worsening slightly. Because selection uses privileged evaluation information, the curves demonstrate the potential of noise selection under that condition.

3.1. Search Verifiers

Supervised Verifiers use conditioning information through CLIP, a model relating image and text representations, and DINO, a model providing visual representations learned without labels. On ImageNet, search uses the conditioning-class score from a CLIP zero-shot classifier, which constructs class representations from text without task-specific classifier training, or a pretrained DINO linear classifier. Both improve IS as the search budget grows.

• These scores assess individual images without explicitly preserving population diversity. Increasing search can therefore improve classifier confidence while worsening FID, a failure the authors call Verifier Hacking: optimizing a verifier while degrading other evaluation criteria.
• Self-Supervised Verifiers omit conditioning information and instead measure feature cosine similarity, the directional agreement between image representations, between the model’s clean-image prediction at noise level σ = 0.4 and its final sample at σ = 0.
• The reported correlation with classification logits, the classifiers’ class scores, is r = 0.7415 for DINO and r = 0.3204 for CLIP. These results support the similarity surrogate more strongly for DINO than for CLIP.

Both classification verifiers improve IS with increasing search computation, and DINO generally reaches higher values than CLIP under the plotted settings. The plot does not establish whether the selected population retains its diversity.

The correlations differ substantially between DINO and CLIP, with r = 0.7415 and r = 0.3204, respectively. The accompanying scaling curves show that feature similarity can guide improved IS without giving the verifier the conditioning class.

3.2. Search Algorithms

Zero-Order Search refines an initial Gaussian pivot by generating N neighboring candidates, evaluating their resulting images, and continuing from the best candidate without computing gradients; the paper abbreviates this configuration as ZO-N. Search over Paths instead branches intermediate sampling trajectories by adding noise, denoising each branch, and retaining the top N candidates; configurations with N initial paths are called Paths-N.

• Search over Paths requires Δb > Δf, so each backward denoising interval Δb exceeds the forward noising interval Δf and the process progresses toward a clean sample. Verifiers receive clean-image predictions rather than noisy intermediate states.
• Both algorithms restrict search locally and partly alleviate the ImageNet diversity problem while retaining improvements in IS. Increasing the Zero-Order Search neighborhood count beyond N = 4 provides limited additional benefit in the reported experiments.
• For Search over Paths, fewer initial paths are more efficient at small budgets, while more paths become useful as the budget increases. The appendix finds that gradient-based search converges faster initially but offers little final advantage under the paper’s approximate compute comparison.

With DINO-LinearHead fixed, both local algorithms improve FID and IS over their starting points. Zero-Order Search eventually shows worsening FID, demonstrating that restricting locality does not eliminate the risk of excessive search.

4. Inference-Time Scaling in Text-to-Image

Text-to-image experiments use FLUX.1-dev on DrawBench, with 200 prompts across 11 categories and one image per prompt, and T2I-CompBench, with 1800 validation prompts and two images per prompt. T2I-CompBench evaluates attribute binding, meaning assignment of requested properties to objects, object relationships, and complex compositions.

• The verifiers are Aesthetic Score Predictor, which predicts visual appeal; CLIPScore, which measures image-text feature alignment; and ImageReward, which estimates human image preferences.
• Verifier Ensemble combines the three by taking each candidate’s unweighted average rank across verifier outputs, avoiding direct averaging of scores with different numerical scales.
• Gemini-1.5-Flash-002 supplies the LLM Grader score from Accuracy to Prompt, Originality, Visual Quality, Internal Consistency, and Emotional Resonance. The main text reports a 0–100 scale for each dimension and uses an averaged overall score.
• T2I-CompBench uses BLIP-VQA, an image question-answering evaluator, for attribute binding; UniDet, an object detector, for spatial and counting evaluations; and a weighted combination of evaluators for complex compositions.

Selected examples show that additional search can change composition, framing, and semantic content after denoising-only outputs have stabilized. They illustrate possible changes across three generators but do not quantify their frequency or average magnitude.

4.1. Analysis Results: Verifier-Task Alignment

On DrawBench, verifier choice changes which qualities improve. With Random Search and 3840 search NFEs, Verifier Ensemble improves Aesthetic Score by 5.1%, CLIPScore by 9.1%, ImageReward by 46.6%, and LLM Grader by 4.5% relative to generation without search.

• Searching only Aesthetic Score improves that score by 10.6% but reduces CLIPScore by 2.2%; searching CLIPScore improves it by 16.5% but reduces Aesthetic Score by 2.0%. These tradeoffs show that the verifier objectives are only partially aligned.
• At 2880 search NFEs, ensemble-guided Random Search obtains an LLM Grader score of 88.18, compared with 87.25 for ZO-2 and 86.84 for Paths-2; the no-search baseline is 84.29.
• The authors attribute Random Search’s advantage on these per-image evaluations to its broad exploration, while local methods spend computation refining their current candidates.

At 3840 search NFEs, the negative cross-metric changes for Aesthetic and CLIPScore expose competing objectives. ImageReward and the ensemble improve every plotted metric, with the ensemble producing the largest reported LLM Grader improvement of 4.5%.

Ensemble-guided Random Search improves all four plotted metrics as its budget increases, but their relative gains differ greatly. ImageReward grows much more strongly than LLM Grader, so the reward increase does not establish an equivalent gain in overall judged quality.

On T2I-CompBench, ImageReward-guided Random Search with 1920 search NFEs outperforms the ensemble in all six categories. Relative to the 30-NFE no-search baseline, it raises Color from 0.7692 to 0.8303, Shape from 0.5187 to 0.6274, Texture from 0.6287 to 0.7364, Spatial from 0.2429 to 0.3151, Numeracy from 0.6167 to 0.6789, and Complex from 0.3600 to 0.3810.

• Aesthetic-guided search reduces five of the six baseline scores, with Spatial the exception.
• The results are consistent with the authors’ explanation that adding an aesthetic criterion can weaken an ensemble when evaluation emphasizes compositional correctness; the experiment does not isolate that criterion’s causal contribution.
• The table’s verifier-specific rows and surrounding analysis establish the comparison, although its caption describes the samples collectively as using Verifier Ensemble.

At 1920 search NFEs, ImageReward achieves the highest score in every compositional category. Aesthetic-guided search falls below the no-search baseline in five categories, supporting the concern that an aesthetic objective can conflict with this benchmark’s requirements.

The table fixes the search budget at 2880 NFEs and uses N = 2 for both local methods. Ensemble-guided Random Search achieves the highest LLM Grader score, 88.18, while the strongest individual metric values follow their corresponding verifier objectives.

4.2. Search is Compatible with Finetuning

Search also improves Stable Diffusion XL (SDXL) after Diffusion-DPO, a diffusion adaptation of Direct Preference Optimization that fine-tunes a model using preference data. On DrawBench, Random Search uses an ensemble with PickScore, a preference evaluator associated with the Pick-a-Pic dataset, replacing ImageReward, with 40 denoising NFEs and 1280 search NFEs.

• The DPO-finetuned model scores 5.59 on Aesthetic, 0.74 on CLIP, and 22.54 on PickScore. Adding search raises these to 5.66, 0.76, and 23.54.
• This experiment supports compatibility between search and preference fine-tuning for the tested model and metrics.

All three reported metrics increase when search is added to the DPO-finetuned model. The result supports an additional benefit from inference-time selection after preference fine-tuning under a 40-NFE denoising budget and a 1280-NFE search budget.

5. Axes of Inference Compute Investment

The paper separates inference computation into search iterations, denoising evaluations per search iteration, and final-generation evaluations. Cheaper search iterations improve scores sooner, while more accurate candidate generation can reach better final scores; on ImageNet, gains from increasing NFEs/iter, the denoising evaluations per search iteration, beyond 50 are limited.

• The default ImageNet configuration uses 50 NFEs per search iteration and 250 NFEs for final generation.
• The default text-to-image configuration uses 30 NFEs per search iteration and 30 for final generation, reflecting the earlier denoising plateau of FLUX.1-dev.

The curves reveal a budget-dependent tradeoff between cheap candidate evaluation and more accurate denoising during search. Very small per-iteration budgets plateau earlier, while moving above 50 NFEs per iteration provides limited gains relative to the added computation.

5.1. Effectiveness of Investing Compute

Across model sizes, the paper compares estimated billions of floating-point operations (GFLOPs), a measure of arithmetic computation, because one NFE has different costs for different generators. SiT-L with search can outperform SiT-XL in some limited-compute regions, while SiT-B’s weaker baseline limits the quality attainable through additional search.

• On DrawBench, the smaller PixArt-Σ generator with ensemble-guided search reaches an LLM Grader score of 85.62 at an estimated compute ratio of ∼0.09 relative to one FLUX.1-dev generation without search, whose score is 84.29.
• At ∼0.09 compute, PixArt-Σ scores 6.03 on Aesthetic, 0.71 on CLIP, and 0.97 on ImageReward, compared with 5.79, 0.71, and 0.97 for the FLUX baseline. At ∼2.59 compute, its scores reach 6.20, 0.73, 1.15, and 86.95 on LLM Grader.
• These are estimated arithmetic-cost comparisons; Table 4 explicitly notes that the compute estimates may be imprecise.

Estimated GFLOPs permit comparison across generator sizes, with separate search configurations selected for FID and IS. SiT-L is competitive with SiT-XL in some compute regions, while SiT-B remains limited by its weaker generation baseline.

At an estimated compute ratio of ∼0.09, searched PixArt-Σ exceeds the unsearched FLUX baseline on Aesthetic and LLM Grader while matching its rounded CLIP and ImageReward scores. Improvement across every listed metric is supported at ∼2.59 compute, and the source explicitly qualifies these ratios as approximate.

6. Related Work

The related work connects the framework to inference-time search in language models, preference fine-tuning for diffusion models, sample selection, gradient-based noise optimization, and learned predictors of favorable noise. The paper studies verifier bias, search algorithms, task requirements, and computation budgets together.

• Preference fine-tuning changes model parameters, while the proposed search selects noise using a fixed generator.
• Gradient-based methods differentiate through sampling, introducing time and memory costs. Learned noise predictors move computation into an additional training stage and require suitable training pairs.

7. Conclusion

The experiments establish noise search as an additional inference-time scaling axis across the tested image-generation models and benchmarks. Additional computation has task-dependent effects: verifier alignment and search locality influence whether gains in selected scores accompany better composition, broader quality, or reduced diversity.

Appendix

• The Appendices begin with A. Experiment Settings. A.1. Training Settings and A.2. Sampling Settings specify that SiT-XL, FLUX.1-dev, and PixArt-Σ use pretrained weights, while SiT-B and SiT-L are trained for 800K iterations. Default sampling uses second-order Heun for SiT-XL, Euler, a first-order numerical solver, for FLUX.1-dev, and DDIM, a deterministic diffusion sampler, for PixArt-Σ; guidance scales are 1.0, 3.5, and 4.5, with resolutions of 256, 1024, and 1024, respectively. A.3. Search Settings studies Random Search candidate counts |S| = 2^k for k ∈ {1, 2, 3, 4, 5, 6, 7, 8}, as well as Zero-Order Search iterations, neighbor counts, and step parameter λ. Search over Paths varies path width M, the number of noise branches per path, initial path count N, and evaluations per backward segment L, with other settings fixed to σ = 0.11, Δb = 0.81, and Δf = 0.78. The ablations show that settings favoring IS can worsen FID. A.4. Verifier Settings details the classifier features, aesthetic predictor, CLIPScore transformation, ImageReward model, and rank-based ensemble. A.5. Evaluation Setting uses 50000 generated ImageNet samples with global batch size 256, averages DrawBench scores over 200 prompts, and evaluates two samples per T2I-CompBench prompt. The reproduced LLM Grader prompt specifies 0–10 component scores and allows a weighted or unweighted overall average, whereas the main text and appendix evaluation description specify 0–100 component scores and an averaged overall score.
• B. Verifier Hacking Leads to Degeneracy in Evaluation Metrics shows increasing precision, indicating sample fidelity, and decreasing recall, indicating coverage of the reference distribution, under classifier-guided Random Search; this supports diversity loss as an explanation for worsening FID.
• C. Zero-Order and First-Order Search compares gradient-based search at learning rate η = 0.01 with Zero-Order Search at λ = 0.995. It uses gradient checkpointing, which recomputes intermediate activations to save memory, and an approximate 3× per-iteration cost adjustment; first-order search converges faster initially but reaches similar final results.
• D. Self-Supervised Verifiers have Marginal Effect in Text-to-Image Setting reports limited gains from CLIP, DINO, and SigLIP, an additional image-text representation model, used as feature-similarity verifiers on DrawBench. Their correlations with CLIPScore are r = 0.3448, r = −0.0963, and r = 0.3494, respectively. The authors propose missing text information and different sampling dynamics as explanations, without isolating those causes experimentally.
• E. More Visualizations on Scaling Behavior provides selected examples for E.1. SiT-XL, E.2. FLUX.1-dev, and E.3. PixArt-Σ, using Zero-Order Search with N = 2 and λ = 0.95. SiT-XL compares 10, 20, and 250 denoising steps with search configurations costing 450, 1850, and 6650 equivalent NFEs; FLUX.1-dev compares 4, 16, and 30 steps with 120, 960, and 2880 equivalent NFEs; PixArt-Σ compares 4, 8, and 28 steps with 112, 896, and 2688 equivalent NFEs. SiT-XL uses the DINO classification verifier with guidance scale 4.0, while FLUX.1-dev and PixArt-Σ use Verifier Ensemble with guidance scales 3.5 and 4.5.

Brief Thoughts

The most useful evidence combines positive scaling results with measured failures of proxy objectives: classifier confidence can improve while diversity falls, and an ensemble can help broad DrawBench evaluation while weakening compositional evaluation. Verifier selection and evaluation using criteria outside the search objective therefore deserve attention when allocating additional generation compute.

The efficiency comparisons use estimated computation and do not establish deployment latency. Broad image quality is assessed by one model-based grader without a reported human validation study or uncertainty intervals, and its documented scoring scales are inconsistent. The reported score gains consequently provide limited evidence about the magnitude of human preference improvements.