Video-R1: Reinforcing Video Reasoning in MLLMs | Summary

Contents

Summary
1 Introduction
2 Related Works
- 2.1 Multimodal Large Language Models for Video
- 2.2 Large Language Model Reasoning
3 Methods
4 Experiments
5 Conclusions
Appendix
Brief Thoughts

This article explains the key points of Video-R1: Reinforcing Video Reasoning in MLLMs.

2025-03-27 (arXiv), 39th Conference on Neural Information Processing Systems (NeurIPS 2025)
Feng, Kaituo, Gong, Kaixiong, Li, Bohao, Guo, Zonghao, Wang, Yibing, Peng, Tianshuo, Wu, Junfei, Zhang, Xiaoying, Wang, Benyou, Yue, Xiangyu.
CUHK MMLab, CUHK (SZ), Tsinghua University, UCAS, CUHK HCCL
Paper
Github
Project Page

Summary

Video-R1 addresses how multimodal large language models (MLLMs), which process visual content alongside language, can learn to reason across video frames instead of relying on isolated images. After 1k reinforcement learning (RL) steps, which train responses using rewards, Video-R1-7B reaches 37.1% accuracy on VSI-Bench, a video spatial reasoning benchmark, compared with 31.4% for its Qwen2.5-VL-7B baseline when both use 64 frames and chain-of-thought (CoT), an explicit sequence of reasoning steps.
The method combines image-video training data with Temporal Group Relative Policy Optimization (T-GRPO), which rewards correct responses based on performance with frames in chronological versus shuffled order. Training starts with supervised fine-tuning (SFT), learning from annotated responses, on Video-R1-CoT-165k and continues with RL on Video-R1-260k; removing image data, the temporal reward, or the SFT initialization reduces performance on all six evaluated benchmarks.
At 64 frames, Video-R1-7B exceeds the reported GPT-4o result on VSI-Bench, 37.1% versus 34.0%, but remains below GPT-4o on VideoMMMU, MMVU, and VideoMME, three other video evaluation datasets; GPT-4o’s frame count is unspecified. The results support gains within the tested training pipeline, while the 16-frame training limit, additional shuffled-frame computation, and model-based assessment of temporal reasoning limit broader claims.

1 Introduction

The introduction identifies two obstacles to extending rule-based RL, which computes rewards with predefined scoring rules, from text and images to video: insufficient incentives to use temporal order and limited high-quality video reasoning data. Group Relative Policy Optimization (GRPO), which computes relative advantages from groups of sampled responses, can reward a correct answer without distinguishing temporal reasoning from a shortcut based on one frame.

Figure 1 illustrates this failure mode: a GRPO-trained model incorrectly treats a cyan cube as already present, whereas the T-GRPO model compares the beginning and end of the sequence and identifies its entry.
Video-R1 addresses the data shortage by mixing image reasoning tasks with video tasks, using images to broaden reasoning coverage and videos to supply dependencies across time.

Contrasting GRPO and T-GRPO responses to a question about a cyan cube entering a scene.

The related work connects video-capable multimodal models with rule-based training for language-model reasoning. Video-R1 builds on these approaches through its training data and temporal reward design.

2.1 Multimodal Large Language Models for Video

LLaMA-VID compresses video representations using context and content tokens; VideoLLaMA2 adds spatial-temporal convolution, processing local patterns across space and time, and an audio branch. LongVA extends the language backbone’s context window, the amount of input it can process, to longer videos. VISA combines world knowledge with object tracking for reasoning video object segmentation, which identifies queried objects across frames.

The authors characterize much of this literature as emphasizing perception and input handling, leaving explicit training for video reasoning comparatively underexplored.

2.2 Large Language Model Reasoning

The paper draws on DeepSeek-R1’s use of outcome-based, rule-computed rewards to train reasoning without supervising every intermediate step. Open Reasoner Zero and Kimi k1.5 provide related examples in text and image settings, motivating an investigation of the additional temporal requirements introduced by video.

3 Methods

The method combines mixed image-video dataset construction, a temporal reward added to GRPO for video inputs, and two-stage training with an additional response-length reward. Qwen2.5-VL-7B-Instruct supplies the base model.

3.1 Dataset Construction

Video-R1-260k combines approximately 116k general video examples with image examples covering general understanding, charts, optical character recognition (OCR), which reads text in images, mathematics, knowledge, and spatial reasoning. The corresponding image category counts are 15k, 21k, 16k, 37k, 37k, and 20k; the paper describes the composition as approximate.

Qwen2.5-VL-72B-Instruct generates CoT annotations for the collected examples. Basic rule-based filtering removes low-quality or inconsistent outputs, yielding Video-R1-CoT-165k for the SFT initialization.
Image examples are intended to teach varied reasoning skills, while video examples teach event progression, dependencies between frames, motion, and causal dynamics over time.

Category proportions and source datasets in Video-R1-260k.

Correctness rewards depend on the answer format: multiple-choice answers are checked against the reference, numerical answers require exact matches, and regression, which predicts numerical quantities, uses one minus relative error. OCR rewards use Word Error Rate, an edit-distance measure between predicted and reference text, while free-form question answering uses the average of ROUGE-1, ROUGE-2, and ROUGE-L, reference-overlap scores.

Most examples use readily verifiable multiple-choice or numerical answers. Smaller portions of free-form, OCR, and regression tasks broaden the supported output formats.

3.2 Temporal Group Relative Policy Optimization (T-GRPO)

T-GRPO samples responses to the same question using temporally ordered frames and randomly shuffled frames, then compares their proportions of correct answers, p and p̃. Equation (1) defines the temporal reward as r_t = α if p ≥ p̃ and 0 otherwise, with α = 0.3; this includes ties, although the surrounding prose describes the ordered group as needing to outperform the shuffled group.

The temporal bonus is added only to correct ordered responses: R_i = r_i + r_t when o_i is correct, and R_i = r_i otherwise. The base reward r_i includes correctness and format rewards; a threshold such as 0.5 can classify continuous-reward outputs as correct.
Advantages, the relative reward signals used for optimization, are normalized within the response group: \(A_i=\frac{R_i-\operatorname{mean}(\{R_j\})}{\operatorname{std}(\{R_j\})}\). The policy objective uses a clipped probability ratio, which bounds the update incentive, and a Kullback–Leibler divergence penalty, which limits departure from a reference model.
Only video inputs receive the temporal treatment. Shuffled-frame responses supply the comparison used to determine the temporal bonus.

T-GRPO training flow with ordered and shuffled video inputs.

3.3 Training Strategies

Qwen2.5-VL-7B-Instruct first undergoes SFT on Video-R1-CoT-165k, producing Qwen2.5-VL-7B-SFT, and then RL on Video-R1-260k, producing Video-R1-7B. Both stages mix image and video examples.

A correct response receives an additional length reward of ω = 0.2 when 320 ≤ len(o_i) ≤ 512. This fixed interval is intended to encourage sufficient reasoning while limiting excessive output.
The length reward is separate from the temporal bonus and is conditional on answer correctness.

3.4 Aha Moment in Video Reasoning

The authors describe outputs that revisit interpretations and check alternatives as an “aha moment” in video reasoning. Figure 4 shows Video-R1-7B selecting the correct navigation answer C after discussing the route and rejecting alternatives, whereas the base model selects B.

The example shows verbal self-checking in a generated explanation. It does not independently establish that this explanation faithfully describes the model’s internal computation.

Navigation responses from Qwen2.5-VL-7B-Instruct and Video-R1-7B on VSI-Bench.

4 Experiments

The experiments compare benchmark scores for the base, SFT, and RL models and test variants with individual training components removed. Further analyses examine training dynamics, the frequency of temporal content in responses, and sensitivity to the temporal reward magnitude.

4.1 Setup

Evaluation covers VSI-Bench, VideoMMMU, and MMVU as video reasoning benchmarks, plus MVBench, TempCompass, and VideoMME as general video understanding benchmarks. MMVU uses its multiple-choice subset, VideoMME results are reported without subtitles, and evaluation uses top_p = 0.001, a sampling probability cutoff, and temperature = 0.01, which controls sampling randomness.

Training uses up to 8 NVIDIA A100 (80GB) graphics processing units, one SFT epoch, a pass through the training dataset, and 1k RL steps for the main results. Video inputs contain at most 16 training frames; evaluation uses 16, 32, or 64 frames.
The reported maximum frame pixel budgets are 128 × 28 × 28 during training and 256 × 28 × 28 during inference. The ordered response group has size G = 8, and the shuffled response group has size G̃ = 4.

4.2 Main Results

At 64 frames, Video-R1-7B obtains 37.1 on VSI-Bench, 52.4 on VideoMMMU, 63.8 on MMVU, 64.8 on MVBench, 73.2 on TempCompass, and 61.4 on VideoMME. It exceeds both the base CoT model and the SFT model on every benchmark at each matched frame count.

At 64 frames, the base CoT model scores 31.4, 50.4, 60.0, 59.2, 72.9, and 59.6 in the same benchmark order; the SFT model scores 34.8, 49.4, 61.6, 60.6, 70.0, and 58.8. SFT therefore does not consistently improve over the base model.
GPT-4o scores 34.0 on VSI-Bench, below Video-R1-7B, but scores 61.2 on VideoMMMU, 75.4 on MMVU, and 71.9 on VideoMME, above Video-R1-7B. Its frame count is unspecified in the table.
Increasing Video-R1-7B’s input from 16 to 64 frames improves most results, including VSI-Bench from 34.6 to 37.1 and VideoMME from 57.4 to 61.4. MMVU instead decreases from 64.2 to 63.8.

Benchmark results for external models and the base, SFT, and Video-R1-7B models at different frame counts.

4.3 Ablation Study

The ablation study, which tests components by removing them, evaluates three variants at 16 frames: removing image data, replacing T-GRPO with original GRPO, and skipping the SFT initialization. Each lowers all six benchmark scores relative to the complete model. On VSI-Bench, these variants score 32.3, 32.7, and 31.8 respectively, compared with 34.6 for Video-R1-7B.

Removing image data reduces VideoMMMU from 49.8 to 45.8; removing temporal modeling reduces VideoMME from 57.4 to 54.5. These comparisons support the utility of both components within this pipeline.
The table reports point estimates without repeated-run variation or confidence intervals, so it does not establish the reliability of smaller differences.

Ablations removing image data, temporal training, or the SFT initialization.

4.4 Training Curves

Across 1k RL steps, the accuracy reward and the temporal reward scaled to 0–1 generally increase. Response length initially decreases, then rises and stabilizes.

The authors hypothesize that the length trajectory reflects a transition away from the SFT reasoning style. The curves show the trajectory but do not directly test that explanation.

Accuracy reward, scaled temporal reward, and response length over 1k RL steps.

4.5 Effect of Temporal Reward Analysis

Qwen2.5-VL-72B first identifies questions requiring temporal reasoning across the six benchmarks and then judges whether generated responses incorporate temporal reasoning. It labels 75.0% of Video-R1 responses as doing so, compared with 60.2% for Video-R1-wo-temporal, a difference of 14.8 percentage points.

This evaluation measures temporal content in generated explanations using a model judge. It does not directly measure whether frame order causally determines each answer.
Sensitivity tests compare α = 0.1, 0.2, 0.3, and 0.4 using mean performance across the six benchmarks. The paper reports similar results for 0.2 and 0.3, with slightly lower results at 0.1 and 0.4.

Average benchmark performance for four temporal reward magnitudes.

5 Conclusions

The conclusion summarizes Video-R1’s extension of rule-based RL through temporal reward shaping, which adds a bonus to the training reward, and mixed image-video reasoning data. Code, models, and datasets are reported as released at https://github.com/tulerfeng/Video-R1.

Appendix

A Additional Experiments includes A.1 Scaling Up RL Training and A.2 Effect of Length Reward Analysis. Extending RL from 1k to 10k steps improves several results: at 64 frames, VSI-Bench rises from 37.1 to 37.8, MMVU from 63.8 to 65.0, MVBench from 64.8 to 65.5, TempCompass from 73.2 to 74.2, and VideoMME from 61.4 to 61.8, while VideoMMMU falls from 52.4 to 51.4. Removing the length reward produces shorter responses and lower average benchmark performance; the authors’ explanation that this reflects shallower reasoning remains a hypothesis.
B Reasoning Examples presents an MMVU example about kinetic-energy loss and an MVBench example about a character’s motivation. The outputs compare candidate interpretations and include verbal verification, illustrating the authors’ description of reflective responses.
C Prompt Template contains C.1 Prompt Template for Training and Inference and C.2 Prompt Template for Temporal Reasoning Evaluation. The shared training, inference, and CoT annotation template explicitly encourages self-reflection and verification, with reasoning inside <think>…</think> and final answers inside <answer>…</answer>. The evaluation template asks whether a response engages with time, sequence, progression, duration, speed, or direction, using a broad interpretation of temporal content.
D Additional Implementation Details specifies Adam, the optimizer used to update model parameters, with learning rate 1e-6, weight decay 0.01, maximum gradient norm 5, and maximum response length 768 tokens. The reference-policy divergence coefficient is β = 0.04. The reported durations are approximately 40 hours per SFT epoch and around 15 hours for 1k RL steps.
E Limitations and Future Works identifies the 16-frame training limit, additional computation for ordered-shuffled comparisons, a fixed length-reward interval that ignores question difficulty, straightforward mixing of image data, and task-specific reward rules. Proposed directions include longer-video training, more efficient temporal modeling, adaptive response lengths, more principled image-to-video transfer, and a generalist video reward model.

Brief Thoughts

The strongest evidence is the matched-frame improvement over the same base model and the consistent degradation when individual components are removed. The ordered-shuffled comparison supplies a concrete training intervention, but the discrepancy between p ≥ p̃ in Equation (1) and strict outperformance in the prose leaves the treatment of ties unclear. The common prompt explicitly requests reflection and verification. The reported “aha moment” examples therefore do not isolate self-correction acquired through training. The temporal-content evaluation supports a change in generated explanations. Model judging, absent uncertainty estimates, and unspecified frame counts for external baselines limit stronger claims about temporal reasoning or superiority across models.