Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning | Summary

This article explains the key points of Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning.

• 2025-03-20 (arXiv)
• Liu, Zhaowei, Guo, Xin, Yang, Zhi, Lou, Fangqi, Zeng, Lingfeng, Niu, Jinyi, Li, Mengping, Qi, Qi, Liu, Zhiqiang, Han, Yiyang, et al.
• School of Statistics and Data Science, Shanghai University of Finance and Economics, China, School of Mathematical Sciences, Fudan University, China, School of Economics, Shanghai University of Finance and Economics, China, AI Finance Development and Service Center, Shanghai University of Finance and Economics, China, QuantaAlpha, China, Department of Statistics, Rice University, USA, Department of Statistics, London School of Economics and Political Science, UK, Qinghai Provincial Key Laboratory of Big Data in Finance and Artificial Intelligence Application Technology, Qinghai Institute of Technology, China
• Paper
• Github

Read this article in Korean

Summary

• The paper addresses how to adapt a small language model to financial questions that combine numerical calculation, specialist knowledge, and contextual reasoning. Across five financial benchmarks under the paper’s automated answer-judging protocol, the 7-billion-parameter Fin-R1 achieves a reported average score of 75.2, compared with 65.6 for its Qwen2.5-7B-Instruct base model and 78.2 for DeepSeek-R1-671B. Its five displayed scores average 75.18, which rounds to the reported 75.2.
• Fin-R1-Data contains 60,091 Chinese and English examples with chain-of-thought, sequences of intermediate reasoning steps, generated by DeepSeek-R1 and filtered using Qwen2.5-72B-Instruct. Training combines supervised fine-tuning, learning from example responses, with reinforcement learning, optimizing generated responses using rewards, through Group Relative Policy Optimization, which compares rewards among responses to the same question.
• The combined training procedure raises the reported average from 71.9 for supervised fine-tuning alone to 75.2 and leads the evaluated models on FinQA, numerical reasoning over financial data, and ConvFinQA, financial question answering that requires conversational context, at 76.0 and 85.0. These results support the training procedure on the selected benchmarks, while unclear training–test separation, limited training details, and absent benchmark uncertainty estimates restrict conclusions about generalization and reproducibility.

1 Introduction

The introduction identifies three obstacles to financial language models: fragmented information across economic, legal, and quantitative sources; limited visibility into generated reasoning; and weak transfer across business scenarios. The proposed response is to construct financial reasoning demonstrations and then adapt a small model using supervised learning followed by reward optimization.

• Figure 1 connects data generation and filtering to the two training steps, with applications spanning financial code, professional knowledge, and business knowledge.
• Figure 2 illustrates Chinese government-bond advice and English debt-risk assessment. These selected examples show response structure and bilingual coverage, but do not measure application-level reliability.

The diagram separates two uses of financial questions: filtered reasoning demonstrations supervise the initial adaptation, while question–solution pairs support later reward optimization. This distinction explains why the reinforcement-learning stage can use questions whose generated demonstrations were rejected.

The Chinese bond comparison and English debt-risk example show financial explanations in both languages. Agreement with supplied answers in these selected cases does not quantify reliability across either application.

2 Related Work

The related work covers pre-training, learning general language patterns from large text collections; post-training, adapting an existing model using demonstrations or feedback; and financial language models. It positions Fin-R1 alongside financial reasoning systems such as XuanYuan-FinX1-Preview and Fino1, while motivating group-relative optimization as a way to avoid training a separate value network that predicts expected rewards.

• Proximal policy optimization constrains changes to the model’s response distribution but typically requires a learned value function; direct preference optimization learns from response preferences without that value function.
• The paper does not experimentally compare these alternative optimization algorithms within the Fin-R1 pipeline.

3 Data Construction

Fin-R1-Data combines financial questions from public datasets and proprietary examination materials into a bilingual resource for post-training. Its construction separates the organization of source tasks from the generation and quality filtering of reasoning demonstrations.

3.1 Data Overview

The 60,091 entries cover financial advanced business knowledge, financial basic business knowledge, financial professional knowledge, and financial code. Advanced business tasks account for roughly 25% and include numerical reasoning, sentiment classification, and extraction of causal relationships; basic business knowledge contributes over 50% and emphasizes financial information, text generation, and regulatory compliance.

• Professional knowledge includes terminology, specialized calculations, and FinPEE, a financial postgraduate entrance examination dataset constructed from Shanghai University of Finance and Economics questions. The authors extract examination text using Mineru, a document-content extraction tool, and have financial experts verify structured question–answer pairs.
• Financial code targets scripts and quantitative strategies, but Appendix Table 3 lists only 152 examples, or 0.25% of the dataset. Figure 3 maps sources to categories without evaluating the resulting capabilities.

The concentric layout maps raw datasets onto four financial task categories and indicates Chinese and English coverage. The appendix counts are needed to assess the uneven category sizes.

3.2 Data Processing

DeepSeek-R1-671B generates reasoning paths and answers at temperature 0.6, a setting controlling sampling randomness, with mathematical answers enclosed in \boxed{} and a leading newline. Filtering first checks answer correctness and then assesses reasoning quality; Figure 4 illustrates why a correct answer alone does not guarantee a useful demonstration.

• The main text describes exact answer matching for objective questions and an automated language-model judge for subjective questions. The supplementary answer-check prompt additionally accepts equivalent numerical representations and rounding.
• Qwen2.5-72B-Instruct is selected over GPT-4o after achieving 99.6% agreement with human answer judgments under the selected prompt condition, tested on 100 FinQA examples with five repetitions.
• Reasoning selection checks internal consistency, term overlap, number of reasoning steps, logical coherence, content diversity, task-domain relevance, and instruction alignment. The supplementary prompt requires at least three reasoning steps and acceptance on all seven criteria.

Answer checking removes incorrect outputs before reasoning selection assesses the remaining demonstrations. The examples distinguish final-answer correctness from the quality of accompanying reasoning, explaining the two filtering steps.

4 Model Training

Fin-R1 starts from Qwen2.5-7B-Instruct. Supervised fine-tuning teaches financial response patterns and explicit reasoning structure, after which Group Relative Policy Optimization uses format and answer-accuracy rewards to refine the model.

• Figure 5 separates demonstration-based learning from reward-based learning: the first stage consumes full responses, whereas the second stage consumes questions and reference solutions.

The diagram shows supervised initialization followed by multiple sampled responses, reward calculation, and group-relative comparisons. Its accuracy and format rewards assess final-answer agreement and response structure without validating each reasoning step.

4.1 Supervised Fine-tuning

Supervised examples pair a question with a complete response containing a reasoning trace inside <think>…</think> and an answer inside <answer>…</answer>. Equation (4.1) minimizes negative log-likelihood, a loss that penalizes assigning low probability to the demonstration response, encouraging the model to reproduce both its reasoning text and final answer.

• For objective questions, the answer can include a numerical solution inside \boxed{} together with an explanation. Figure 6 illustrates this structure using financial-news sentiment classification with label 0 for Bearish, the negative-sentiment class.
• The authors characterize this stage as behavior cloning, imitation of demonstrations, and motivate subsequent reinforcement learning as a way to refine responses using feedback.

The sentiment example includes an intermediate explanation and a final answer with label 0. Supervised training therefore imitates explanatory text as well as the classification target.

4.2 Group Relative Policy Optimization

The reinforcement-learning dataset contains objective questions and reference solutions, including difficult questions whose generated demonstrations were unsuitable for supervised training. For each question, the policy, the model’s probability distribution over generated text, samples at least two candidate outputs and receives the sum of two binary rewards: compliance with the required tags and semantic agreement of the extracted answer with the reference.

• The format reward requires <think>…</think> followed by <answer>…</answer>, with no content outside those tags. Qwen2.5-Max supplies the accuracy judgment, so answer verification depends on a language-model evaluator.
• Equation (4.4) centers each reward using the group mean and divides by the group reward standard deviation, with a small denominator floor such as 10⁻⁸. Although the accompanying prose calls this a standard error, the displayed denominator is a standard deviation.
• The policy update uses clipped probability ratios, bounded ratios between current and sampling probabilities, and Kullback–Leibler divergence, a measure of distributional difference, to constrain departure from the supervised policy. Neither reward directly verifies the correctness of intermediate reasoning steps.

The reinforcement-learning version retains the financial-news question and reference label but omits a demonstration response. The model generates its own response for reward evaluation.

5 Experiment

The experiments compare Fin-R1 with eight models across five financial datasets and evaluate whether both training stages are useful. The reported evidence consists of benchmark scores, training-stage ablations, comparisons that remove one training stage, and illustrative model outputs.

5.1 Evaluation Datasets and Methodology

Evaluation uses FinQA, ConvFinQA, Ant-Finance for financial business knowledge, TFNS, Twitter Financial News Sentiment, for sentiment classification, and Finance-Instruct-500k for professional financial questions. Finance-Instruct-500k uses a custom 10% test subset selected through stratified sampling, which samples within dataset groups; each other dataset uses 1,000 randomly sampled questions or all questions when fewer than 1,000 are available.

• An automated language-model judge accepts semantically equivalent answers despite differences in decimal precision or numerical representation, as illustrated in Figure 8. Scores are reported on a 100-point scale representing the percentage judged correct.
• These datasets also appear among the data-construction sources. The PDF does not clearly document split-level exclusion or a duplicate-removal audit sufficient to establish that every evaluation question is separate from post-training data.

The examples contrast 13.1% with 13.12% and a full numerical amount with an expression in billions. They motivate equivalence-aware judging while showing that acceptance depends on rounding and representation rules.

5.2 Baselines

The baselines are DeepSeek-R1-671B; DeepSeek-R1-Distill-Qwen-7B, -14B, and -32B; DeepSeek-R1-Distill-Llama-70B; and Qwen2.5-Instruct models at 7B, 14B, and 32B. Distillation transfers behavior from a larger teacher model to a smaller student, allowing the comparisons to span both model size and training lineage.

• The benchmark table does not include the specialized financial reasoning models discussed in Related Work, limiting direct evidence of superiority over those systems.

5.3 Results

Table 1 reports Fin-R1 scores of 76.0 on FinQA, 85.0 on ConvFinQA, 81.0 on Ant-Finance, 71.0 on TFNS, and 62.9 on Finance-Instruct-500K, with a reported average of 75.2. The reported average places it second behind DeepSeek-R1 at 78.2, and its FinQA and ConvFinQA scores are the highest among the evaluated models.

• The five displayed Fin-R1 scores have an arithmetic mean of 75.18, which rounds to the reported one-decimal average of 75.2.
• Using the reported averages, the improvement is 9.6 points over Qwen2.5-7B-Instruct at 65.6 and 17.2 points over DeepSeek-R1-Distill-Qwen-7B at 58.0. The introduction’s claim of more than 17 points over same-size models therefore applies to the latter comparison, not every 7B baseline.
• Performance varies by task: Fin-R1 trails its base model on Ant-Finance, 81.0 versus 85.0, while improving the other four reported scores.

Fin-R1 ranks second by reported average and leads FinQA and ConvFinQA, but its Ant-Finance score falls below the 7B base model. Its five displayed scores average 75.18, which rounds to the reported 75.2.

Table 2 reports averages of 65.6 for the base model, 67.8 for Fin-R1-Zero trained with Group Relative Policy Optimization alone, 71.9 for Fin-R1-SFT, the supervised-only variant, and 75.2 for the combined procedure. The combined model improves every listed benchmark over Fin-R1-SFT, supporting the usefulness of reward optimization after supervised training under this experimental setup.

• Figure 9 provides a qualitative failure example: Fin-R1-Zero produces incoherent text, while Fin-R1-SFT computes a price-to-earnings ratio, share price divided by earnings per share, as 10 but finishes with the inconsistent answer 100.
• These comparisons test complete training variants without isolating each reward or filtering criterion. The PDF reports no confidence intervals or repeated-run uncertainty for the benchmark scores.

The reported averages are 65.6 for the base model, 67.8 with reward optimization alone, 71.9 with supervised fine-tuning alone, and 75.2 with both stages. The combined procedure improves every displayed score over the supervised-only variant; its five displayed scores average 75.18, which rounds to 75.2.

The reward-only model produces incoherent text, while the supervised-only model reaches the correct intermediate ratio of 10 and then states 100 in its final sentence. These selected failures illustrate why readable reasoning and final-answer correctness require separate scrutiny.

6 Conclusion

The conclusion attributes Fin-R1’s benchmark performance to financial reasoning data and the combination of supervised fine-tuning with group-relative reward optimization. Proposed extensions include financial multimodal inputs, data in multiple forms beyond text, and closer integration with risk management and regulatory compliance; these remain future directions.

• The code URL provided in the paper is https://github.com/SUFE-AIFLM-Lab/Fin-R1.

Appendix

• A Details for raw datasets documents ten sources and their processed contributions: FinanceQT 152, Finance-500K 11,300, FinanceIQ 2,596, FinPEE 179, Ant-Finance 1,548, FinCorpus 29,288, FinQA 2,948, ConvFinQA 7,629, TFNS 2,451, and FinCUGE, a financial dataset covering multiple reasoning tasks, 2,000. These total 60,091 entries; Table 4 identifies FinPEE as closed and the other listed sources as open.
• B The Prompt of Data Construction contains B.1 The Prompt of Data Distillation, B.2 The Prompt of Answer Check, and B.3 The Prompt of Reasoning Selection. The distillation template specifies task inputs, instructions, a boxed final answer, and a leading newline; the answer-check template specifies numerical equivalence, rounding, and a binary decision. The reasoning-selection template applies seven quality criteria and accepts a demonstration only when all seven are satisfied.
• C Model Selection for Data Construction evaluates the automated judges. C.1 Model Selection for Answer Check compares five prompt formats on 100 randomly selected FinQA examples repeated five times, yielding 500 judgments per format: the selected OF, our used format, gives Qwen2.5-72B-Instruct 0.4% classification inaccuracy and 0.0% format irregularity, compared with GPT-4o’s 2.8% and 0.8%. Including the question in the WQ, with-question, format increases Qwen’s inaccuracy to 8.0%, showing sensitivity to prompt design.
• C.2 Model Selection for Reasoning Selection compares Qwen2.5-72B-Instruct and GPT-4o with independent human judgments on 20 randomly selected examples, scoring seven criteria from 0 to 7 in total. The authors report closer agreement for Qwen and use it for reasoning selection, but this small validation sample provides limited coverage of the full dataset.
• D Case Study compares responses to a company with net profit of 2 million yuan, 500,000 issued shares, and a share price of 40 yuan. The base model answers 20 without a reasoning prompt and 8 with one, whereas Fin-R1 correctly computes earnings per share as 4 yuan and the price-to-earnings ratio as 10. This selected example illustrates an arithmetic improvement without systematically testing investment advice or compliance decisions.

Brief Thoughts

The training-stage ablation provides the clearest evidence for the combined procedure: supervised demonstrations improve performance, and subsequent reward optimization improves all five displayed benchmark scores over supervised training alone. The paper reports a 3.3-point average gain; Fin-R1’s five displayed scores average 75.18, consistent with the reported 75.2 after rounding.

Explicit reasoning text makes responses easier to inspect, but the format reward and final-answer judge do not establish that the text faithfully explains the model’s computation. Clearer training–test separation, fuller training and inference settings, broader human validation, and measured deployment costs would strengthen the evidence for generalization, transparency, and practical efficiency.