Scalable Chain of Thoughts via Elastic Reasoning | Summary

This article explains the key points of Scalable Chain of Thoughts via Elastic Reasoning.

• 2025-05-08 (arXiv)
• Xu, Yuhui, Dong, Hanze, Wang, Lei, Sahoo, Doyen, Li, Junnan, Xiong, Caiming.
• Salesforce AI Research
• Paper
• Github
• Project Page

Read this article in Korean

Summary

• Scalable Chain of Thoughts via Elastic Reasoning addresses how language models can preserve answer quality when strict output limits interrupt their intermediate reasoning. On MATH500, a mathematical problem-solving benchmark, the paper’s adapted E1-Math-1.5B model achieves 83.6% single-attempt accuracy (Pass@1) using 1619 tokens, the units of generated text, per question, compared with 83.6% using 1796 tokens for L1-Max and 79.9% using 1959 tokens for L1-Exact, two length-control baselines evaluated under varying generation budgets.
• Elastic Reasoning divides generation into independently budgeted thinking and solution phases: thinking contains chain-of-thought, the intermediate reasoning before an answer, and the solution contains the final response. Budget-constrained rollout, training on generated responses whose thinking can be forcibly stopped, teaches the model to produce solutions from incomplete reasoning and transfer to other tested inference budgets without further training.
• With the 64K generation limit used for the paper’s unconstrained code comparison, the adapted E1-Code-14B model reduces average output length on LiveCodeBench, a code-generation benchmark, from 17,815 to 11,145 tokens relative to DeepCoder-14B-Preview, while accuracy increases from 58.1% to 58.4%. The evidence covers mathematical and programming tasks; the method assumes that thinking and solution generation can be meaningfully separated.

1 Introduction

Large reasoning models generate extended intermediate reasoning, which can improve problem solving but makes inference cost difficult to control. Long2Short methods train shorter reasoning sequences, while S1 uses special tokens to regulate generation and L1 learns explicit length constraints through reinforcement learning, optimization using task rewards. Elastic Reasoning reserves output capacity for the solution and trains the model to handle interrupted thinking; code is available at https://github.com/SalesforceAIResearch/Elastic-Reasoning.

• The math model uses 200 training steps with a maximum response length of 2K tokens, compared with the reported 700 steps for L1-Exact and 820 for L1-Max with 4K responses.
• In the 24K-limit comparison on AIME2024, a mathematical competition evaluation, E1-Math-1.5B reaches 35.0% accuracy, versus 27.1% for L1-Max, 24.2% for L1-Exact, and 41.0% for the original model.

The AIME comparison separates inference control from training: reserving solution capacity improves over whole-output truncation and S1, while E1-Math-1.5B further improves accuracy across the plotted token range. The ordering motivates both components of Elastic Reasoning.

2 Related works

The related work addresses three questions: how additional inference computation improves reasoning, how generation length can be controlled, and how reasoning can become more efficient. Elastic Reasoning combines explicit inference limits with training under those limits.

2.1 Test-time scaling in large language models

Test-time scaling means increasing computation during inference to improve performance. Prior approaches include generating intermediate reasoning and self-consistency, which samples multiple reasoning paths and selects their most consistent answer; other studies examine trade-offs between model size and inference computation. This paper focuses on maintaining useful reasoning under strict output limits.

2.2 Length control in large language models

Earlier length-control methods modify positional encodings, the representations of sequence positions, penalize deviations from target lengths, or train on instructions specifying output length. The authors argue that these approaches do not directly address the consequences of interrupting reasoning; budget forcing, truncation or padding using special tokens, can also produce incomplete outputs. Elastic Reasoning imposes limits through generation control and training without requiring length instructions in the prompt.

2.3 Efficient reasoning in large language models

Prior efficiency methods reward shorter reasoning, use supervised fine-tuning, training on provided examples, to teach concise reasoning, compress intermediate steps into latent internal representations rather than explicit text, or adapt inference without retraining. Elastic Reasoning uses reinforcement learning under fixed thinking and solution limits to balance answer quality with output cost.

3 Methodology

The method has two components: Separate Budgeting reserves independent capacities for thinking and solution generation, and budget-constrained rollout applies the same interruption procedure during training. Together they address missing answers caused by whole-output truncation and poor answers caused by incomplete thinking.

3.1 Preliminaries: Reasoning Language Models

For an input x, the output is y = (y^think, y^solution), where y^think contains intermediate reasoning enclosed by <think> and </think>, and y^solution contains the final solution. The paper states that thinking typically accounts for over 90% of generated tokens, making it the main target for budget control.

3.2 Elastic reasoning

3.2.1 Budget-constrained inference: A conventional token cap enforces |y| ≤ c, where |y| is output length and c is the generation budget. Truncating the entire output at that limit can leave the solution incomplete or absent, producing an unusable prediction even after substantial reasoning has been generated.

3.2.2 Separate budgeting for thinking and solution: Split the total budget as c = t + s, allocating t tokens to thinking and s tokens to the solution. If the model emits </think> early, solution generation begins immediately; otherwise, the procedure appends </think> when t is exhausted and permits up to s solution tokens. This reserves solution capacity, although a finite solution budget does not guarantee that every answer will be complete or correct.

3.2.3 Budget-constrained rollout: Training generates thinking up to t*, inserts </think> if necessary, and then generates the solution within s*. GRPO, the reinforcement-learning algorithm used here, updates the model’s response-generation policy using rewards normalized relative to other sampled responses; the objective is J(θ) = E[r(y)], the expected task reward under the constrained generation policy. The default training allocation is (t*, s*) = (1K, 1K), and the resulting models are evaluated at other thinking budgets without additional fine-tuning.

• The reward is described as task-specific, but the paper does not provide a detailed reward specification for each task.
• Separate Budgeting alone can still fail when interrupted thinking provides insufficient support for the answer, motivating training with the same constraint.

Training uses fixed thinking and solution limits to generate rewarded samples, while inference varies the thinking allowance and retains solution capacity. The diagram shows that training and inference share the same procedure for interrupting thinking and continuing with a solution.

4 Experiment results

The experiments compare single-attempt accuracy with generated token usage on mathematical and programming tasks. Further analyses exchange the models used for each output phase and vary training budgets to examine which components improve and whether improvements transfer to other inference limits.

4.1 Models and datasets

The starting models are DeepScaleR-1.5B-Preview and DeepCoder-14B-Preview, derived from DeepSeekR1-Distill-Qwen-1.5B and 14B through iterative context lengthening, training with progressively longer generation limits. Mathematical training uses AIME (1984–2023), AMC, Omni-Math, and STILL, mathematical reasoning datasets; code training uses TACO, SYNTHETIC-1, and LiveCodeBench (2023/05/01–2024/07/31), programming datasets.

• Mathematical evaluation uses AIME2024, MATH500, AMC, Olympiad-Bench, and Minerva Math, benchmarks of mathematical problem solving.
• Programming evaluation uses LiveCodeBench (2024/08/01–2025/02/01), Codeforces, a competitive-programming evaluation reported through rating and percentile, and HumanEval+, a code-generation benchmark.

4.2 Mathematical reasoning results

Validation on AIME2024 is measured every 10 training steps. Accuracy starts near 0.07 and peaks near 0.20 around step 150, while the plotted final value at step 200 is approximately 0.16; average training reward rises with substantial fluctuations. The curves show improvement during training, but validation accuracy is not monotonic.

Reward generally increases, but validation accuracy fluctuates, peaks near 0.20 around step 150, and ends near 0.16 at step 200. The plotted trajectory shows improvement during training without continuous validation improvement.

Across five mathematical benchmarks, E1-Math-1.5B generally improves the accuracy–token trade-off over S1 and L1-Exact and remains competitive with L1-Max. On MATH500 it obtains 83.6% Pass@1 with 1619 average tokens, matching L1-Max’s 83.6% with 1796 tokens and exceeding L1-Exact’s 79.9% with 1959 tokens.

• With a 24K generation limit, E1-Math-1.5B reaches 35.0% AIME2024 accuracy, 6.0 percentage points below the original model’s 41.0%. The Introduction reports 27.1% for L1-Max, a 13.9 percentage-point gap from the original; the Results section separately describes L1-Max’s performance degradation as 12.9%, so the paper’s figures are internally inconsistent.
• AIME2024 average output length falls by 32.1% relative to the original model. Accuracy generally increases approximately linearly with the logarithm of generated token usage over the tested range.

Across five mathematical benchmarks, E1 generally offers a better accuracy–token trade-off than S1 and L1-Exact and is competitive with L1-Max. The 24K-limit points also show that E1 remains below the original model in accuracy, making the efficiency trade-off visible.

4.3 Code reasoning results

On LiveCodeBench, DeepCoder-14B-Preview remains below 10% accuracy at inference budgets below 4K even when Separate Budgeting is applied. E1-Code-14B reaches 37.3%, 41.6%, 44.1%, and 47.0% with respective thinking budgets of 1K, 2K, 3K, and 4K, each paired with a 1K solution budget. Because both models use Separate Budgeting, reserving answer capacity alone does not account for the gains under tight limits.

With a 64K generation limit, E1-Code-14B obtains 58.4% on LiveCodeBench, a Codeforces rating of 1987 at the 96.0 percentile, 91.4% on HumanEval+, and 70.6% on AIME2024. DeepCoder-14B-Preview scores 58.1%, 1945 at the 95.4 percentile, 90.8%, and 71.7%, respectively; average LiveCodeBench output length decreases from 17,815 to 11,145 tokens, a 37.4% reduction.

• The Codeforces result is close to O1-2024-12-17 (Low), reported at rating 1991 and the 96.1 percentile; this comparison concerns Codeforces rather than equivalence across all benchmarks.
• The E1 code model is trained exclusively on code data during this adaptation. Its AIME2024 result is 1.1 percentage points below the starting model’s result.

Increasing E1-Code-14B’s thinking budget improves the reported results across all four evaluations. Under the larger limit, it improves on DeepCoder by 0.3 percentage points on LiveCodeBench, 42 Codeforces rating points, and 0.6 percentage points on HumanEval+, while AIME accuracy decreases by 1.1 percentage points. The subscript gains are changes from the baseline, not uncertainty estimates.

Both models use Separate Budgeting, yet E1-Code-14B performs substantially better at short outputs. Holding this inference procedure constant supports a contribution from the adapted model beyond reserving output capacity for the solution.

4.4 Analysis and discussions

4.4.1 Which part is enhanced after training? The authors exchange thinking and solution generation between DeepScaleR-1.5B-Preview and E1-Math-1.5B on AIME2024. At a 0.5K+1K thinking-plus-solution budget, the original model scores 2.10%, replacing only thinking raises accuracy to 3.50%, replacing only the solution raises it to 10.8%, and using E1 for both yields 13.5%.

• The larger gain from replacing the solution phase supports improved answering from incomplete reasoning as a major contributor.
• The authors hypothesize that improved solution generation explains transfer across budgets; the component exchange does not establish a complete causal account.

Exchanging generation phases shows that E1 improves both thinking and solution generation. At 0.5K+1K, replacing only the solution raises accuracy by 8.7 percentage points, compared with 1.4 points from replacing only thinking; using E1 for both produces the highest accuracy at every listed budget.

4.4.2 Ablation of training budget t*: Training thinking budgets are varied over {0.5K, 1K, 2K, 3K}, with the solution budget fixed at 1K. All configurations transfer across tested inference budgets, and the authors select 1K+1K as the default based on its favorable accuracy–token trade-off across five mathematical benchmarks.

Models trained at different thinking limits retain useful performance across evaluation budgets. The 1K thinking configuration has a favorable overall accuracy–token trade-off, supporting its selection as the default; increasing the training allowance does not consistently improve results.

4.4.3 Token allocation between thinking and solution: Tightening inference limits primarily reduces thinking tokens, while solution length increases somewhat on AIME2024 and remains relatively stable on LiveCodeBench. Under the larger evaluation limits, AIME2024 output length decreases from 10,050 to 6825 tokens and LiveCodeBench length from 17,815 to 11,145 tokens. The trained models therefore also produce shorter outputs when generation is allowed to continue beyond the training budget.

The stacked bars show that most output savings come from the thinking segment. Under the larger limits, solution length stays close to the original model while total usage falls by 32.1% on AIME2024 and 37.4% on LiveCodeBench.

4.4.4 Iterative training: A second training round increases the thinking budget from 1K to 3K while retaining a 1K solution budget. AIME2024 accuracy decreases from 13.5%, 17.5%, 24.8%, and 27.9% to 11.5%, 17.1%, 22.9%, and 26.7% at respective evaluation budgets of 0.5K+1K, 1K+1K, 2K+1K, and 3K+1K. This tested continuation provides no improvement.

Every listed AIME2024 evaluation budget scores lower after the second training round with expanded thinking capacity. This result provides evidence against the tested continuation configuration, without establishing that every form of iterative training would fail.

5 Conclusion

The paper concludes that separately budgeting thinking and solution generation, combined with training on interrupted thinking, improves length control and robustness across the tested mathematical and programming tasks. The evidence also shows shorter outputs under larger generation limits, although retaining the original model’s accuracy depends on the task and evaluation condition.

Appendix

• A Limitations and broader impacts: The method assumes distinct thinking and solution phases, which may not suit dialogue or commonsense tasks with interleaved reasoning and answering. Evaluation is concentrated on mathematics and programming, leaving scientific question answering and multi-hop retrieval, retrieval requiring several linked steps, untested. The authors identify possible efficiency benefits and risks from omitting important justifications, but do not measure energy savings, user trust, or effects in high-stakes applications.
• B Training details: Both models use a learning rate of 1 × 10⁻⁶, batch size 128, and separate 1K-token thinking and solution limits. E1-Math-1.5B trains for 200 steps using VeRL, the named training framework, while E1-Code-14B trains for 30 steps. The appendix states that other training settings follow the starting models.

Brief Thoughts

The code comparison and component exchange provide the clearest evidence for training beyond inference-time budget control. Both code models use Separate Budgeting, yet the adapted model performs better under tight limits; exchanging mathematical generation phases also shows a larger gain from replacing solution generation than from replacing thinking alone.

The measured efficiency gains concern token usage and accuracy: the paper does not report wall-clock latency, training costs adjusted for hardware differences, or uncertainty across repeated runs. Fewer training steps and shorter response limits support a smaller training workload, but do not quantify an end-to-end cost reduction. The 6.0-percentage-point AIME2024 accuracy loss under the 24K limit makes the trade-off with the original math model explicit.