TinyGSM: achieving >80% on GSM8k with small language models | Summary

This article explains the key points of TinyGSM: achieving >80% on GSM8k with small language models.

• 2023-12-14 (arXiv)
• Liu, Bingbin, Bubeck, Sebastien, Eldan, Ronen, Kulkarni, Janardhan, Li, Yuanzhi, Nguyen, Anh, Ward, Rachel, Zhang, Yi.
• Carnegie Mellon University, Microsoft Research
• Paper

Read this article in Korean

Summary

• The paper investigates whether small language models can solve multistep grade-school math word problems on GSM8K, a benchmark testing mathematical reasoning and language comprehension. A generator with 1.3 billion parameters paired with a separate, equally sized verifier, a model that scores candidate solutions for correctness, achieves 81.5% accuracy by selecting among 48 Python solutions per question, compared with 68.2% for the generator alone and 77.4% for GPT-3.5-0613 when each produces one Python solution per question.
• TinyGSM contains 12.3M synthetic question–Python-solution pairs, totaling 1.8B tokens, the text units processed by language models; M and B denote million and billion. GPT-3.5-turbo generates these pairs from GSM8K training questions and variants containing irrelevant information. Fine-tuning, further training an existing model on task-specific examples, raises the 1.3B Phi-1.5 generator’s accuracy from 44.6% to 68.2%; verifier training then uses generated solutions labeled against the original GSM8K training answers.
• A 125M generator with a 1.3B verifier reaches 76.1%, versus 71.7% when those component sizes are reversed; diversifying verifier training data across checkpoints, model states saved at different training steps, raises accuracy from 78.9% to 81.5%. These results support allocating more parameters to verification under the tested conditions. The headline comparison uses different candidate budgets, synthetic solutions are not checked for mathematical correctness, and the contamination checks cannot exclude paraphrased test-set overlap.

1 Introduction

The introduction asks whether model scale is necessary for mathematical reasoning or whether better training data and answer selection can compensate for a small generator. GSM8K contains 8.8K word problems involving 2 to 11 reasoning steps, testing both elementary mathematics and interpretation of natural language.

• The teacher–student setup trains a smaller model on examples generated by a larger model, expanding the data available beyond the original training set.
• The verifier follows the approach introduced with GSM8K. This paper combines it with large-scale synthetic training data and examines verifier size and training-data diversity.

Adding verifiers moves the Phi-GSM configurations upward in accuracy while retaining relatively small parameter counts. The parameter axis does not account for generating and scoring multiple candidates, so the plot does not establish an advantage at matched inference cost.

Phi-GSM+V reaches 81.5% using a 1.3B generator, a 1.3B verifier, and verify48@1, while the generator alone reaches 68.2% at pass@1. GPT-3.5-0613 reaches 77.4% using code at pass@1; the table's differing answer formats and candidate-selection methods require care when comparing rows.

2 Related works

Related work connects synthetic-data training, mathematical word-problem datasets, selection among multiple generations, and different forms of supervision. Self-consistency, which selects the most frequent answer among candidates, motivates comparison with learned verification because confident or repeated answers can still be wrong.

• Outcome supervision labels a solution by its final answer; process supervision labels intermediate reasoning steps. This work labels complete programs by their execution outcomes without checking intermediate steps.
• The authors distinguish their separate, fully fine-tuned verifier from methods that ask a generator to critique and revise its own answers.
• GSM-IC adds irrelevant context to GSM8K questions and supplies seeds for part of TinyGSM, addressing a weakness of evaluation on clean problem statements.
• The section also discusses in-context learning, performing tasks from examples supplied in the prompt, and chain-of-thought prompting, requesting intermediate reasoning steps. The authors motivate training small models by describing these prompting approaches as primarily effective for larger models.

3 The TinyGSM dataset

TinyGSM expands the 7473 GSM8K training problems through GPT-3.5-turbo generations conditioned on an original question without its solution. Each generated example pairs a new word problem with a Python solution, allowing an interpreter to perform numerical calculations; approximately one third of the dataset derives from GSM-IC variants containing irrelevant information.

• Temperature sampling, which controls generation randomness, and prompts requesting diverse wording and multiple logical steps encourage variation while the seed-specific prompt remains fixed across calls.
• Most examples are generated jointly as questions and solutions. The remaining examples, including all GSM-IC-derived data, use separate question-generation and solution-generation stages.
• The dataset contains 12.3M question–solution pairs and 1.8B tokens. The paper estimates generation cost at around $3600 under the commercial pricing it used.

The questions appear in function docstrings, text embedded within Python functions, and the function bodies express their calculations as code. This representation lets an interpreter perform arithmetic after the model translates the word problem into operations.

Filtering removes questions that are too short or contain no numbers and solutions whose code cannot execute. It does not verify mathematical correctness, and the paper acknowledges that generated problems can violate common sense even when mathematically valid.

• The authors avoid filtering by majority agreement because they find agreement concentrated on easy questions, which could remove useful challenging examples.
• An early 350M model achieves around 50% on GSM8K but more than 87% on synthetic questions with consistent answers, supporting their concern that agreement filtering could select an overly easy training distribution.

The joint-generation template requests 10 variants of a seed question, varied wording, multiple logical steps, and complete Python programs. Explicit question and program delimiters structure the generated pairs, but these formatting requirements do not verify their mathematical correctness.

4 Solving grade school math with small language models

The solution pipeline has two trained components: a generator fine-tuned on TinyGSM and a verifier trained to distinguish correct from incorrect generated solutions. At inference, the generator produces multiple Python programs, and the verifier selects the candidate whose final-token score is highest.

4.1 Learning from synthetic data

The authors fine-tune pretrained Phi-1.5 variants with 125M, 350M, and 1.3B parameters using Adam, an optimization algorithm, and FP16, 16-bit floating-point training. Training uses linear learning-rate warm-up, a gradual initial increase in update size, a maximum learning rate of 1e-4, weight decay of 0.01 to regularize model weights, an effective batch size of 1024 examples per update, and up to 20k steps.

• Single-generation GSM8K accuracies are 63.1%, 65.9%, and 68.2%, respectively. Table 1 also reports 74.3% for a 2.7B Phi-2-based generator.
• An earlier 1.3B model reaches 94% at pass@32, the fraction of questions with at least one correct answer among 32 generations. This measures the availability of correct candidates, rather than the accuracy of a method that must select one without knowing the answer.

4.2 Improving small models with a verifier

Verification addresses the gap between producing a correct candidate somewhere in a sample set and reliably selecting it. The paper reports diminishing gains from the final one-third of TinyGSM and studies a separate correctness predictor as an additional route beyond increasing generator size or synthetic-data volume.

• At inference, 48 candidates are sampled for each question, and the highest-scoring candidate is submitted as the single final answer.
• The 81.5% result uses verify48@1, selection of one answer from 48 candidates, whereas the reported 77.4% GPT-3.5-0613 comparison uses pass@1, evaluation of one generated solution.

Verifier training samples 48 generations per GSM8K training question and assigns binary labels by comparing executed numerical answers with the known answers. The verifier starts from the generator’s weights, adds a prediction head, an output layer shared across token positions, and updates all network parameters.

• The same whole-program correctness label supervises every token; intermediate calculations receive no separate correctness labels.
• The authors report that this training objective consistently outperforms predicting one label for the entire sequence. Full fine-tuning also outperforms updating only the prediction head or training the verifier from scratch.
• The final token supplies the inference score. Figure 5 illustrates why scores on earlier tokens should not automatically be interpreted as valid assessments of individual reasoning steps.

Verifier training benefits from examples generated at checkpoints of 1k, 6k, and 12k steps; both training and testing candidates mix temperatures 0.5 and 0.7. Using multiple checkpoints improves accuracy from 78.9% to 81.5%, supporting the value of generation diversity even when some contributing checkpoints are less accurate.

• The paper reports that generation accuracy at temperature 1 can be more than 10% worse than at temperature 0, which chooses the most likely next token.
• Some verifier transfer is possible: a 1.3B generator reaches 78.1% using a 350M verifier trained on generations from a 350M generator.

The generator–verifier size grid shows that the best tested configuration uses 1.3B parameters for each component, reaching 81.5%. At the same combined parameter count, the 125M-generator/1.3B-verifier configuration reaches 76.1%, compared with 71.7% for a 1.3B generator and 125M verifier.

• Pairing each generator with an equally sized verifier raises accuracy from 63.1% to 68.9% at 125M, from 65.9% to 71.3% at 350M, and from 68.2% to 81.5% at 1.3B.
• These comparisons support a larger verifier under the tested sampling regime. The paper does not compare configurations at matched inference latency or total computation.

The grid reports 76.1% for a 125M generator with a 1.3B verifier and 71.7% when those sizes are reversed. The bars show improvements for all three generators with equally sized verifiers, including 68.2% to 81.5% at 1.3B; each verifier-assisted result selects one answer from 48 candidates.

All three displayed programs receive correct final classifications, yet one correct program contains intermediate tokens assigned unfavorable scores. This example shows that scores learned from whole-program outcome labels can disagree with the correctness of intermediate calculations.

5 Robustness and decontamination

The robustness analysis examines possible GSM8K test-set overlap inherited through GPT-generated training data and transfer to a different arithmetic word-problem benchmark. These checks assess training-data overlap and performance beyond GSM8K.

5.1 Contamination test

The contamination test removes punctuation and numbers before checking n-grams, contiguous sequences of n words, against GSM8K test questions. Among 11.0M unique synthetic questions, 22 have a nonzero 13-gram match and 38k, around 0.35%, have a nonzero 8-gram match.

• The paper describes 13-gram matching as its decontamination procedure and explicitly acknowledges that exact matching cannot guard against contamination through paraphrases.
• There are fewer unique questions than question–solution pairs because some questions receive multiple solutions in the two-stage generation procedure.
• The paper does not report how much the headline accuracy changes after removing matched examples.

5.2 Evaluation on SVAMP

SVAMP, Simple Variations on Arithmetic Math word Problems, contains 1000 arithmetic word problems formed by varying base questions. Without further fine-tuning, the paired 1.3B generator and 1.3B verifier achieve 75.6%, providing evidence of transfer beyond GSM8K.

• The highest result in Figure 6 is 79.0%, obtained with a 350M generator and a 1.3B verifier. The largest generator therefore does not produce the best transfer result in this grid.
• All three generator sizes achieve their highest reported SVAMP accuracy with the 1.3B verifier: 74.1%, 79.0%, and 75.6% for 125M, 350M, and 1.3B generators, respectively.

The 1.3B verifier gives the best result for every generator size. The highest accuracy is 79.0% with the 350M generator, exceeding the 75.6% obtained when both components have 1.3B parameters, so transfer performance does not increase monotonically with generator size.

6 Discussions

The discussion identifies solution format and verifier size as directions for further study. Python execution avoids some calculation errors, but the authors observe weaker performance on questions requiring equation solving and suggest combining code with natural-language solutions.

• The size-grid results motivate studying how a fixed parameter budget should be divided between generation and verification.
• Combining solution formats is proposed as future work; the paper does not report an evaluated combined-format system.
• The authors state their intention to release TinyGSM. The PDF does not establish a completed release.

Appendix

• A Additional details on TinyGSM, including A.1 Other prompts, documents the two-stage generation procedure. Figures 7 and 8 request new question variants from GSM8K and GSM8K-IC seeds, respectively; Figure 9 requests complete Python solutions and specifies output formatting, symbolic equation solving with sympy.solve(), and rounding upward to an integer with math.ceil().
• A.2 Contamination check: 13-gram collisions reiterates that 22 of 11.0M unique questions overlap test questions under the matching procedure. Figure 10 displays examples of these collisions, although its printed caption repeats the description of the code-solution prompt rather than describing the displayed content.
• B Pretrained vs Random Init compares training on TinyGSM from random weights with starting from pretrained weights. Table 2 reports 53.1%, 55.5%, and 57.3% for random initialization at 125M, 350M, and 1.3B, versus 63.3%, 65.9%, and 68.2% with pretraining; its 125M pretrained value differs from the 63.1% reported in Table 1.

Brief Thoughts

The generator–verifier grid provides a useful comparison at equal combined parameter count: reversing the component sizes changes GSM8K accuracy from 71.7% to 76.1%. Alongside the improvement from 78.9% to 81.5% with checkpoint diversity, this supports studying both verifier capacity and the distribution of its training examples.

The headline result measures a two-model system using 48 candidates and Python execution. The PDF supplies no matched inference-cost comparison, no mathematical-correctness audit of TinyGSM, and no contamination check that excludes paraphrased overlap; Figure 5 also cautions against interpreting outcome-trained token scores as explanations of individual reasoning steps.