Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers | Summary

This article explains the key points of Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers.

• 2025-09-03 (arXiv)
• Huang, Xingyue, Rishabh, Franke, Gregor, Yang, Ziyi, Bai, Jiamu, Bai, Weijie, Bi, Jinhe, Ding, Zifeng, Duan, Yiqun, Fan, Chengyu, et al.
• CAMEL-AI.org
• Paper
• Github
• Project Page

Read this article in Korean

Summary

• Loong addresses the shortage of verified reasoning examples outside mathematics and programming for large language models (LLMs), models that generate language responses. With GPT-4.1-mini generating questions and code and DeepSeek-R1 judging the outputs, few-shot prompting, which supplies seed examples as demonstrations, achieves execution-and-judge pass rates of 92.6% in Logic and 93.9% in Physics, outperforming Self-Instruct and Evol-Instruct in both domains.
• The framework combines LOONGBENCH, a human-vetted seed dataset of 8,729 examples across 12 domains, with LOONGENV, an environment that generates questions, executable code, and answers. Its verification design compares code-derived answers with independently generated Chain-of-Thought (CoT) solutions, natural-language reasoning leading to a final answer, and uses LLM-as-a-judge, a language model assessing correctness, to check semantic agreement.
• Self-Instruct recursively generates diverse structured prompts, while Evol-Instruct transforms seed questions through generalization, specification, and complexity scaling. Evol-Instruct produces Advanced Physics questions that are closer to seeds in the reported similarity measurements but harder for the two tested models to solve. Reinforcement Learning with Verifiable Reward (RLVR), learning from automatically checked reward signals, remains a future application; the paper reports benchmark and generation evaluations without a downstream training experiment.

1 Introduction

The introduction argues that reasoning improvements require both verifiable answers and enough high-quality examples. Logic, graph theory, physics, and finance lack the comparable supply of verified data available in mathematics and programming, motivating a framework that expands curated seed data through automated generation and verification.

• The proposed agent-environment loop generates a question and solution code, executes the code to obtain an answer, asks a trainable agent for a CoT response, and compares the two final answers.
• The reported contributions are the seed dataset, the modular generation environment, benchmarking across model types, and analysis of synthetic-data correctness, diversity, and difficulty.

Generated questions reach the trainable agent, while synthetic answers and the agent’s CoT answer reach the verifier. The reward arrow depicts the proposed training interface; the reported experiments evaluate generation and benchmarking without running this reinforcement learning loop.

2 Loong Project

Loong Project separates the curated seed dataset from the environment that expands it. LOONGBENCH provides domain-specific questions and verified code-based solutions; LOONGENV supports interchangeable generation policies and verifiers to produce further examples.

• The design rests on the authors’ premise that code execution can provide more reliable computational answers than natural-language reasoning alone in suitable domains.
• The verifier checks semantic agreement, meaning that differently expressed answers can be accepted when they are equivalent.

2.1 LOONGBENCH: Human-Vetted Seed Datasets Across Multiple Domains

LOONGBENCH is intended to seed synthetic generation rather than serve directly as a large training corpus. The paper describes each example as containing a natural-language question, a verified final answer, an executable Python rationale, and metadata covering license, source, domain, dependencies, name, contributor, creation date, difficulty, and tags.

• Advanced Math: 1,611 problems come from the MATH training split at difficulty levels 4 or 5. o3-mini generates code using SymPy, a library for symbolic mathematics; retained outputs must execute and match the reference through MathVerifier, a tool for parsing and evaluating LaTeX-formatted mathematical expressions.
• Advanced Physics: 429 problems come from SciBench and Olympiadbench, scientific problem-solving benchmarks. Generated code outputs are checked after unit conversion, with default relative tolerance 0.01 and adjustments for reference precision; each problem receives at most three attempts to generate a correct rationale.
• Chemistry: 3,076 examples are selected from ChemistryQA, a chemistry question-answer dataset, for conceptual clarity and computational relevance. o3-mini reformulates unclear examples while preserving chemistry concepts and removing implementation-specific wording.
• Computational Biology: 51 examples remain after collecting biology questions, generating code with GPT-4o, checking execution, and using GPT-4o-mini to assess whether the code solves the question, matches the reference, and performs computation instead of hardcoding the answer. Rule-based regular-expression matching, which compares outputs through text patterns, provides an additional check.
• Finance: 235 examples are drawn from QuantLib resources, a financial-modeling library, and the FinAI Financial Reasoning Dataset. Questions are retained or reformulated to express financial reasoning tasks and paired with Python solutions.
• Board Game: 926 questions concern Blackjack. The pipeline collects 50 distinct losing rounds against fixed opponents, augments state-action pairs with expert rationales and reflection guidelines, and retains only actions ranked first by perfect-information Monte-Carlo roll-outs, simulated continuations using full game information to estimate winning probabilities.
• Graph & Discrete Math: 178 examples are derived from networkx documentation, a network-science library. The authors inspect 896 functions, identify 370 with usage examples, exclude random sampling and multiple valid outputs, rewrite questions, and manually check execution and intended results.
• Logic: 130 examples include Einstein’s Puzzle, a deductive assignment puzzle, and Analytical Reasoning questions from the Law School Admission Test. Both are constraint satisfaction problems, assignments that must satisfy a set of rules; GPT-4.1-mini generates code using python-constraint, a constraint-solving library, to ensure solvability and a unique solution.
• Mathematical Programming: 76 constrained optimization problems, which optimize an objective subject to rules, are curated from Gurobipy, PySCIPOpt, CVXPY, and Statsmodels tutorials covering optimization and statistical modeling. The authors select clear practical notebooks, retain or regenerate associated data, and correct solution-code inconsistencies.
• Medicine: sampling up to 30 entries per calculator from MedCalc-Bench, a medical-calculation benchmark, initially yields 1,192 examples. GPT-4.1-generated rationale code executes for 1,117 examples, and output matching retains 916.
• Security & Safety: 516 examples concern cryptographic Capture-The-Flag problems, challenges requiring recovery of an expected flag or result. Problems are curated across 3 major categories and 44 subcategories, diversified through substitutions of plaintext and ciphertext, the original and encrypted messages, checked by code execution, and manually reviewed.
• Programming: 585 examples use LeetCode problems and a two-stage code-critic pipeline, in which a model identifies and attempts to fix solution errors. Claude-3.5-Sonnet generates candidate solutions; DeepSeek-R1 judges correctness and attempts corrections when needed.

The domain counts total 8,729 and are uneven: Chemistry contributes 3,076 examples, compared with 51 for Computational Biology and 76 for Mathematical Programming. The dependencies identify the computational tools supporting different domains, so the breadth of coverage should be read alongside sample sizes and tooling.

2.2 LOONGENV: Modular Synthetic Generation with Verifiable Supervision

LOONGENV separates question synthesis, code-based answer synthesis, and verification. It supports few-shot prompting, Self-Instruct, and Evol-Instruct, allowing question-generation strategies to change independently of downstream modules.

• A coder agent generates and executes a program for each question. The authors do not assume that execution proves correctness and state that raw synthetic answers are not used directly for training.
• The verification design compares a code-derived solution with an independently generated natural-language solution. Agreement supports acceptance, although both approaches can incorrectly agree, producing a false positive.
• The current semantic verifier uses LLM-as-a-judge. Domain-specific verifiers and reinforcement learning rewards based on verified final-answer agreement are proposed future developments.

3 Experiments

The experiments examine model accuracy across LOONGBENCH domains, the executability and semantic validity of generated solutions, and how prompting strategies change generated-question diversity and difficulty. They evaluate existing models and generated datasets without measuring learning gains after training on the generated data.

3.1 Setup

The benchmark compares GPT4.1-mini, o3-mini, Grok-3, Claude-3.7-Sonnet, DeepSeek-r1, and Qwen3-8B across all 12 domains. Each prompt receives one response without retries, function/tool calling is disabled unless otherwise stated, and the maximum output length is 4096 tokens.

• The authors report consistent temperature, top-k, and top-p settings where applicable: these control sampling variability, restrict candidate tokens by count, and restrict them by cumulative probability, respectively. Their numerical values are not provided in this section. Open-model inference uses a single NVIDIA H100 80GB GPU.
• Accuracy is the percentage of correctly answered problems per domain. GPT4.1-mini judges correctness while accounting for symbolic equivalence, mathematical expressions with the same value or meaning, and differently formatted answers.
• The paper acknowledges that some Mathematical Programming responses exceed the 4096-token window and become incomplete, which complicates interpreting low accuracy as task difficulty alone.
• The implementation uses the CAMEL framework for agent workflows, and the project code is available at https://github.com/camel-ai/loong.

3.2 Benchmarking LOONGBENCH

LOONGBENCH shows substantial variation across domains: Programming accuracy ranges from 81.7% to 100.0%, while Mathematical Programming ranges from 6.4% to 13.2%. Finance and Security also have low scores, with best reported accuracies of 24.3% and 28.7%, respectively.

• DeepSeek-r1 leads Advanced Physics at 77.4%, Game at 97.3%, Graph at 83.7%, and Security at 28.7%, and ties o3-mini in Finance at 24.3%. o3-mini leads Advanced Maths at 97.4% and Programming at 100.0%.
• Leadership depends on the domain: GPT4.1-mini leads Logic at 65.4% and Medicine at 59.6%, Grok-3 leads Computational Biology at 96.1%, and Claude-3.7 leads Chemistry at 80.7% and Mathematical Programming at 13.2%.
• The authors report top-two finishes in 8 of 12 domains for DeepSeek-r1 and 6 of 12 for o3-mini. These counts support broad competitiveness, while several domains are led by other models.
• The narrative describes Qwen3-8B as trailing o3-mini by 50 percentage points in Game and 22 in Logic. The table reports 43.2% versus 96.0% and 39.2% versus 61.6%, giving gaps of 52.8 and 22.4 percentage points. DeepSeek-r1’s results do not support extending Qwen3-8B’s deficits to open-source models generally.

No model leads every domain. Programming scores reach 100.0%, while Mathematical Programming scores remain between 6.4% and 13.2%; DeepSeek-r1 leads several domains, and GPT4.1-mini leads Logic and Medicine. The table supports comparisons by domain more directly than a general ranking of open-source and proprietary model categories.

3.3 Synthesizing Data with LOONGENV

For synthetic generation, separate question and code agents both use GPT-4.1-mini, while DeepSeek-R1 acts as the verification judge. The implementation states that 100 synthetic questions are generated per domain and strategy from randomly selected seed examples.

• Each program runs in a sandboxed Python environment, an isolated environment for executing generated code. Execution success measures whether it runs without error; the judge separately assesses whether the question is meaningful and whether the code solves it.

3.3.1 Execution and Verification Outcomes distinguishes Pass, meaning code executes and the judge approves the answer; Judge-Rejected, meaning code executes but the judge rejects the result; and Not Executable, meaning code fails to run. FewShot achieves 92.6% Pass in Logic and 93.9% in Physics; Self-Instruct has 44.8% judge rejection in Logic and 82.0% Pass in Physics.

• Evol-Instruct produces 55.0% non-executable code in Logic. In Physics, 29.8% of its examples are judge-rejected and 14.0% fail to execute.
• FewShot yields the highest accepted fraction in both reported domains. The separate rejection category also shows that successful execution does not establish that a program solves its question.

The stacked bars separate code that fails to execute from code that executes but is rejected by the judge. FewShot has the largest accepted portion in both domains. Evol-Instruct’s Logic failures are dominated by non-executable code, while its Physics failures include a substantial judge-rejected portion.

3.3.2 Diversity evaluates Advanced Physics through question embeddings, vector representations used to compare meaning, with cosine similarity, a measure of vector alignment, and t-SNE, a two-dimensional projection used to visualize embedding proximity. Across 100 seed-generated pairs per strategy, average/maximum similarities are 0.77/0.88 for FewShot, 0.80/0.91 for Self-Instruct, and 0.90/0.97 for Evol-Instruct.

• FewShot has the lowest measured seed similarity, while Evol-Instruct stays closest to seed questions. The authors interpret Evol-Instruct as preserving core semantics while potentially changing structure and complexity.
• The t-SNE plots and similarity histograms provide qualitative distributional evidence. They do not directly measure reasoning depth or subsequent training benefit.
• The paper describes Self-Instruct as more semantically diverse, although its average seed similarity is higher than FewShot’s. Its qualitative diversity interpretation therefore extends beyond the reported average-similarity comparison.

Average and maximum seed-generated cosine similarity increase from FewShot at 0.77 and 0.88 to Self-Instruct at 0.80 and 0.91 and Evol-Instruct at 0.90 and 0.97. The comparison establishes greater embedding proximity to seeds for Evol-Instruct; solver accuracy provides the separate evidence about difficulty.

The projections compare blue seed examples with orange generated examples under each strategy. Their overlap and separation support the authors’ qualitative discussion of distributional proximity, but the two-dimensional layouts do not directly measure task complexity.

The histograms show the spread of seed-generated similarities beyond the averages. Evol-Instruct concentrates more strongly at high similarity, while Few-shot and Self-Instruct include more lower-similarity pairs, consistent with Evol-Instruct’s greater measured proximity to the seed questions.

3.3.3 Difficulty uses GPT4.1-mini and DeepSeek-r1 accuracy on generated Advanced Physics questions as a proxy for difficulty. Their accuracies are 92.0% and 93.2% on Few-shot questions, 83.0% and 87.4% on Self-Instruct questions, and 62.0% and 70.3% on Evol-Instruct questions, compared with 71.8% and 77.4% on the seed dataset.

• Evol-Instruct questions are harder for both tested models despite their greater embedding similarity to the seeds. This is consistent with the authors’ hypothesis that transformations can preserve question meaning while increasing solution complexity.
• The comparison measures performance on different datasets. Whether training on the harder examples improves generalization remains untested.

4 Related Work

Related Work situates Loong among post-training, further training of an existing language model, with human preferences; model-generated instruction data; and reinforcement learning with automatically verifiable rewards. Self-Instruct and the Evol-Instruct methods used by WizardLM and WizardCoder provide the background for Loong’s generation strategies.

• The post-training discussion covers learning from human feedback, optimizing models to prefer higher-rated responses, and reward-based reasoning methods.
• The synthetic-data discussion connects instruction generation with frameworks that address controllability, diversity, and factuality.
• The reward-learning discussion covers automatically checked supervision across reasoning, instruction-following, and coding tasks. These training methods provide context for Loong’s intended use; they are not training experiments performed in this paper.

5 Conclusion

The conclusion presents LOONGBENCH and LOONGENV as infrastructure for studying synthetic reasoning data and verifiable supervision across domains. The demonstrated contributions are a curated seed dataset, benchmark comparisons, and measurements of generated-data correctness, diversity, and difficulty.

• Future work includes reinforcement learning with synthetic questions, generation assisted by tools, formal abstraction, and expansion to multilingual and multimodal tasks involving multiple languages and input modalities.
• The proposed role in alignment, training models toward desired behavior, remains an intended application without a reported training evaluation.

Appendix

• A System Prompt for Benchmarking Experiments documents standardized solver and judge formats. Solver System Prompt uses the {{DOMAIN}} placeholder and requires the response to end with only the final answer inside \boxed{}, illustrating \boxed{\frac{1}{4}} as the accepted format and \boxed{x = \frac{1}{4}} as the rejected format. It does not prohibit reasoning before the final boxed answer.
• LLM-as-Judge System Prompt compares the final boxed answer with the reference, disregards preceding reasoning, and returns \boxed{1} for mathematical equivalence or \boxed{0} for a missing or non-equivalent answer. Benchmark accuracy therefore evaluates final-answer agreement rather than the correctness of the full reasoning trace.

Brief Thoughts

Loong provides a concrete workflow connecting curated domain examples, executable solutions, and semantic checks. Separate execution and rejection outcomes expose the reliability differences between generation strategies, and the Advanced Physics results show that greater seed similarity can coexist with lower solver accuracy. Verification remains a material limitation: two solutions can incorrectly agree, and the paper does not quantify judge error against an independent human audit of synthetic outputs. Generation evidence is narrower than the 12-domain benchmark: execution outcomes are shown for Logic and Physics, while diversity and difficulty analyses focus on Advanced Physics. The stated 100-question generation setting does not explain the aggregation behind fractional pass rates such as 92.6% and 93.9%. Confidence intervals, measured large-scale generation costs, and downstream training gains are not reported. The 4096-token cap and final-answer-only judging limit what these experiments establish about long CoT synthesis. They do not directly validate complete reasoning traces or evaluate long-form reasoning after training.