ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning | Summary

Contents

Summary
1. Introduction
2. Related Works
- 2.1. Vision-Language-Action Models
- 2.2. Reasoning in Vision-Language-(Action) Models
3. Method
4. Experiment
5. Conclusion
- Limitations
- Broader Impacts
Appendix
Brief Thoughts

This article explains the key points of ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning.

2025-07-22 (arXiv)
Huang, Chi-Pin, Wu, Yueh-Hua, Chen, Min-Hung, Wang, Yu-Chiang Frank, Yang, Fu-En.
NVIDIA, National Taiwan University
Paper
Project Page

Read this article in Korean

Summary

ThinkAct addresses how vision-language-action (VLA) models, which convert visual observations and language instructions into actions, can plan multistep robot tasks before executing them. Under the standard LIBERO simulated manipulation evaluation, it achieves 84.4% overall success versus 76.8% for DiT-Policy, its diffusion Transformer action baseline, and 83.9% for CoT-VLA, a method that predicts visual subgoals before actions.
A multimodal large language model (MLLM), a language model that also processes visual inputs, learns plans through reinforcement learning (RL), optimization using reward signals. Rewards compare predicted gripper trajectories, the paths of the robot’s grasping tool, with demonstrations; a visual plan latent, a compact learned representation of planning intent, then conditions a separate action model trained by imitation learning, learning to reproduce demonstrated actions.
With 10 demonstrations per LIBERO task, Figure 5 reports 39.1% average success for ThinkAct versus 34.7% for Magma, a multimodal agent model, and 14.0% for OpenVLA, a model trained on robot demonstrations. Reward ablations, experiments that remove individual training signals, support contributions from both visual rewards, while self-correction evidence consists of selected execution examples. The appendix reports 17% longer execution time than OpenVLA on an A100 graphics processing unit (GPU), and the authors acknowledge inherited visual and spatial hallucinations, incorrect claims about observed objects or their relationships.

1. Introduction

The introduction identifies a gap between interpreting multimodal inputs and planning actions over extended tasks in changing environments. Direct input-to-action training often omits explicit planning, while supervised chain-of-thought (CoT), intermediate textual reasoning steps, requires costly curated traces and may overfit particular scenes or reasoning patterns.

ThinkAct connects high-level reasoning with low-level control through visual trajectory rewards and a compact plan representation.
The project page is https://jasper0314-huang.github.io/thinkact-vla/.

ThinkAct’s action-aligned visual feedback and examples of adaptation, planning, and self-correction.

The related work separates models trained for visual action understanding and execution from methods that introduce intermediate reasoning. ThinkAct combines these directions by training reasoning with action-related visual feedback and transferring its plan representation to an action policy, the model that selects executable actions.

2.1. Vision-Language-Action Models

RoboPoint, a spatial point prediction model, and LLARVA, a model trained with visual trajectories, use spatial information to improve action understanding. AHA, a manipulation failure detection model, learns from synthetic failures. OpenVLA learns action execution from robot demonstrations, while TraceVLA and HAMSTER use visual traces; the authors identify limited explicit planning for complex instructions and unfamiliar scenarios as a remaining problem.

2.2. Reasoning in Vision-Language-(Action) Models

ECoT, an embodied reasoning method, trains intermediate subgoals with supervised fine-tuning (SFT), training against supplied target outputs. RAD, an action-free reasoning approach, curates reasoning from human videos before connecting it to robot actions. CoT-VLA instead generates visual subgoal frames before action prediction.

The paper identifies curated reasoning annotations and task-specific video generation as scaling constraints.
Existing reasoning RL methods primarily reward question-answering correctness; ThinkAct adds visual trajectory feedback to connect reasoning with executable plans.

3. Method

ThinkAct uses two separately trained components: a reasoning MLLM that produces a visual plan latent and an action model that uses this latent together with the current observation and instruction. Planning updates can occur less frequently than control updates.

3.1. Problem Formulation

At timestep t, the model receives visual observation o_t and instruction l and predicts action a_t. Depending on the embodiment, the action is a textual command or a seven-degree-of-freedom control vector containing translation, rotation, and gripper control.

The reasoning model F_θ produces latent c_t from the observation and instruction; action model π_φ uses it to guide N subsequent interactions with the environment.

Reasoning-model optimization and latent-conditioned action execution in ThinkAct.

3.2. Reinforced Visual Latent Planning for Embodied Reasoning

Under Reward Shaping from Action-Aligned Visual Feedback, the planner generates reasoning and a trajectory of normalized two-dimensional gripper points. The goal reward compares predicted start and end positions with detector-extracted demonstration positions: r_goal = 1/2(f(p_1, p̂_1) + f(p_K, p̂_K)), where f(p, p′) = max(0, 1 − ‖p − p′‖₂²).

This reward measures agreement with demonstrated endpoints; it does not directly measure successful execution in a simulator.
The visual plan latent is decoded into trajectory points and also supplies the representation used to condition the controller.

The trajectory reward is r_traj = max(0, 1 − d(τ, τ̂)), where dynamic time warping, a distance that aligns trajectory sequences, supplies d. The combined reward is r = 0.9r_visual + 0.1r_format, with r_visual = 0.5r_goal + 0.5r_traj and r_format measuring output-format correctness.

Under Reinforced Fine-Tuning for Eliciting Visual Latent Planning, Group Relative Policy Optimization (GRPO), an RL method that compares rewards within sampled response groups, trains the reasoning model.
The objective uses group-standardized rewards and a Kullback–Leibler divergence penalty, a measure of distributional deviation that regularizes changes from the original model. Additional question-answering data provide correctness rewards.

3.3. Reasoning-Enhanced Action Adaptation

A latent projector, a learned mapping into the action model’s input space, connects the frozen reasoning model to the action policy. The state encoder, which represents the observation and instruction, the projector, and the action model learn from annotated action demonstrations while the reasoning model remains frozen.

One latent plan can guide N interactions, permitting asynchronous planning and control: the planner reasons periodically while the action model updates each step.

3.4. Learning Strategy and Inference

Training first initializes the reasoning model with supervised trajectory and question-answering data and independently pretrains the action model on Open X-Embodiment (OXE), a collection of robot demonstrations. GRPO then trains the reasoning model with visual rewards, after which action adaptation trains the controller and its connecting modules using frozen planner outputs.

At inference, the observation and instruction produce a visual plan latent that conditions a sequence of executable actions.

4. Experiment

The experiments test robot execution and embodied reasoning, reasoning about tasks and observations in an agent’s environment. They measure manipulation success and question-answering performance, then examine reward removal, adaptation from limited demonstrations, and selected failure recovery examples.

4.1. Experimental Setup

Implementation Details initialize the planner with Qwen2.5-VL 7B, a multimodal model with 7 billion parameters, and use a 432M-parameter diffusion Transformer (DiT) controller, an action model that generates actions through iterative denoising. SFT uses 20K iterations, batch size 32, and learning rate 1e−5 with DeepSpeed ZeRO-3, the distributed training configuration. GRPO uses 6K iterations, batch size 64, learning rate 1e−6, and rollout size 5, the number of sampled responses per input.

DINOv2, the image encoder, and CLIP, the text encoder, produce 1024-dimensional state embeddings. A Q-Former, the query-based latent projector, uses 32 queries.
Action adaptation uses 100K OXE samples for 120K iterations, batch size 256, and learning rate 2e−5. LIBERO receives another 75K fine-tuning iterations with batch size 128.
All experiments use 16 NVIDIA A100 GPUs with 80 GB memory.

Training Datasets and Evaluation Benchmarks combine OXE trajectories with RoboVQA, a robotic visual question-answering dataset, EgoPlan-IT, a first-person planning dataset, and Video-R1-CoT, a video reasoning dataset with annotated reasoning, for cold-start training. RL additionally uses human manipulation videos from Something-Something v2, EgoPlan-IT/Val training and validation data, Reflect failure data, and LLaVA-Video-178K, a general video instruction dataset.

SimplerEnv, a simulation benchmark with visual scene variations, and LIBERO measure manipulation success.
EgoPlan-Bench2 measures first-person multistep planning with multiple-choice accuracy. RoboVQA uses BLEU, a text-overlap score, while OpenEQA measures understanding of environments with language-model-based answer scoring.

4.2. Quantitative Evaluation

For Robot Manipulation, ThinkAct achieves 71.5%, 65.1%, and 43.8% overall success on Simpler-Google Visual Matching, Simpler-Google Variant Aggregation, and Simpler-Bridge Visual Matching. These settings test performance across visual scene conditions. The corresponding DiT-Policy results are 56.0%, 48.2%, and 32.4%, giving absolute gains of 15.5, 16.9, and 11.4 percentage points.

On LIBERO, ThinkAct reaches 84.4% overall versus 76.8% for DiT-Policy, 76.5% for OpenVLA, and 83.9% for CoT-VLA.
The aggregate advantage does not hold for every subtask: CoT-VLA scores higher on LIBERO-Object and LIBERO-Goal, and Magma scores higher on Simpler-Bridge Stack Blocks.

Robot manipulation success rates on SimplerEnv and LIBERO.

For Embodied Reasoning, ThinkAct scores 48.2% on EgoPlan-Bench2 and 59.8 overall BLEU on RoboVQA, compared with 45.7% and 55.7 for Qwen2.5-VL*, the baseline fine-tuned on EgoPlan-IT and RoboVQA. On OpenEQA, ThinkAct scores 56.2 overall versus 55.5 for InternVL3, the strongest listed overall comparator.

ThinkAct leads all four EgoPlan-Bench2 domains and all four RoboVQA BLEU orders in the table.
OpenEQA results are uneven: other models lead categories including functional reasoning, object recognition, and world knowledge.

Embodied reasoning results on EgoPlan-Bench2, RoboVQA, and OpenEQA.

4.3. Qualitative Results

The manipulation examples pair textual subgoals with planned image-space trajectories and execution frames. For the LIBERO-Long book task, the plan separates grasping the book, moving it from left to right, and placing it in the back compartment.

Before-and-after-RL comparisons show a RoboVQA answer changing to recognize that a lifted sheet of paper can move left and an OpenEQA answer changing from carpet appearance to rectangular shape.
These selected examples illustrate planning and question interpretation; they do not quantify how often the generated reasoning is correct.

Intermediate plans, visual trajectories, and manipulation execution sequences.

Reasoning and answer comparisons before and after reinforcement learning.

4.4. Ablation Study

The full model scores 60.1 on SimplerEnv, 48.2 on EgoPlan-Bench2, and 59.8 on RoboVQA. Removing the trajectory reward yields 59.2, 47.9, and 58.5; removing the goal reward yields 59.1, 47.6, and 58.9.

Removing both visual rewards leaves 56.9, 47.2, and 58.3, compared with 56.4, 46.4, and 57.9 for the SFT cold-start model.
Both visual rewards improve the reported scores over the configuration using only question-answering rewards. The table provides no uncertainty estimates.

Goal and trajectory reward ablations on manipulation and embodied reasoning benchmarks.

4.5. Analysis of ThinkAct

Under Reasoning Enhance Few-Shot Adaptation, adaptation using a small number of demonstrations, the controller is fine-tuned with 10 demonstrations per task and evaluated over 100 trials. LIBERO-Spatial and LIBERO-Object test unseen environments, while LIBERO-Goal tests new skills. ThinkAct achieves 35.5%, 52.0%, and 36.2%, respectively, compared with Magma’s 26.0%, 49.0%, and 29.0%.

Figure 5 reports averages of 39.1% for ThinkAct, 34.7% for Magma, and 14.0% for OpenVLA. The ThinkAct average does not match the arithmetic mean of its three displayed suite scores, approximately 41.2%; the plotted Goal values also imply a 7.2-percentage-point gain over Magma, whereas the prose states 7.3%.
Under Reasoning Elicit Self-Correction, the planner receives a short video segment instead of one image. The illustrated robot detects a dropped box, revises its plan, regrasps it, and completes the task.
The recovery example uses added temporal context; the section reports no aggregate recovery success rate.

LIBERO success rates after adaptation with 10 demonstrations per task.

A dropped-object recovery example using revised planning.

5. Conclusion

The conclusion attributes ThinkAct’s manipulation and reasoning results to visually rewarded planning combined with latent-conditioned action adaptation. The benchmark comparisons support improved aggregate task performance, while the execution examples illustrate multistep decomposition and failure recovery.

Limitations

The authors acknowledge that pretrained MLLMs can hallucinate object attributes and spatial relationships, producing incorrect plans that affect downstream execution. They identify training for better agreement with visual observations and hallucination suppression as future work.

Broader Impacts

The paper identifies assistive robotics, home automation, and industrial systems as potential applications. It also notes that ambiguous instructions, hallucinated visual reasoning, and overconfidence in generated reasoning could produce unintended actions, particularly in safety-critical settings.

Appendix

A. Additional Experimental Setup and A.1. Implementation Details specify a GRPO regularization weight β of 1e−2, maximum response length 1024, temperature 1.0, and top-p sampling, sampling from a set of likely outputs with cumulative probability p = 0.99. The planner uses up to 16 video frames, with maximum resolutions stated as 128 × 28 × 28 pixels for video data and 256 × 28 × 28 pixels for image data, and generates K = 8 trajectory points. Additional question-answering rewards use accuracy for multiple-choice answers and average ROUGE-1/2/L scores, text-overlap measures, for free-form answers.
Reasoning-Enhanced Action Adaptation caches planner latents offline during controller training. The diffusion controller uses a DDPM noise scheduler, its training noise schedule, with 1000 timesteps and 20 DDIM sampling steps, its inference denoising procedure. Standard controller observations are single 224 × 224 RGB, red-green-blue, third-person images; planning updates occur every N = 15 interactions on SimplerEnv and N = 75 on LIBERO.
A.2. Training Data Preparation and A.2.1. Training Datasets describe extracting gripper tracks from OXE’s fractal20220817_data and bridge subsets, sampling three starting frames per video, and simplifying trajectories with the Ramer–Douglas–Peucker algorithm, a keypoint reduction procedure. Human trajectories use hand detection, select the hand with the largest movement when two appear, and apply camera stabilization. The datasets include approximately 5K long-horizon and 92K medium-horizon RoboVQA videos, 100 simulated and 30 real-world RoboFail cases reformatted into 300 question-answer pairs, 50K EgoPlan-IT and 5K EgoPlan-Val instances, 165K Video-R1-CoT samples, and LLaVA-Video-178K’s 178K videos, 960K open-ended questions, and 196K multiple-choice questions.
A.2.2. Training Data Construction specifies 30K trajectories, 50K RoboVQA samples, 50K EgoPlan-IT samples, and 165K Video-R1-CoT samples for SFT. RL uses 12.5K trajectories, 10K RoboVQA samples, 10K EgoPlan-IT/Val samples, 0.5K RoboFail samples, and 10K LLaVA-Video-178K samples. Table A4 documents trajectory and question-answering prompt formats, including eight normalized trajectory points and separate reasoning and answer tags.
A.3. Evaluation Benchmarks describes SimplerEnv’s variations in lighting, textures, backgrounds, distractors, and camera poses, and specifies 500 trials per LIBERO suite using three random seeds. EgoPlan-Bench2 contains 1,321 multiple-choice questions across 24 scenarios and four daily-life domains; RoboVQA evaluation uses 1,893 validation video–text pairs. OpenEQA contains over 1,600 human-authored questions from more than 180 real-world environments and uses language-model-based scoring.
B. Additional Experiment Results and B.1. Qualitative Comparisons of Robot Execution Results refer comparisons with DiT-Policy and OpenVLA to ThinkAct.mp4. B.2. Additional Qualitative Results adds an EgoPlan-Bench2 example in which RL changes an incorrect next-action answer to the correct one, while B.3. More Self-Correction Samples shows recovery from an unsuccessful mug grasp and a failed box pickup. B.4. Results of Smaller Model Size reports ThinkAct-3B scores of 44.0 on EgoPlan-Bench2, 55.3 on RoboVQA, and 48.9 on OpenEQA, compared with 43.0, 53.6, and 44.6 for Qwen2.5-VL-3B*.
B.5. Results of 5-Shot Adaptation uses five demonstrations per task and 100 evaluation trials. ThinkAct achieves 24.0%, 29.0%, and 32.5% on LIBERO-Spatial, LIBERO-Object, and LIBERO-Goal, with a reported 28.5% average versus 21.3% for Magma and 8.3% for OpenVLA.
B.6. Ablation Study reports LIBERO/OpenEQA scores of 84.4/56.2 for the full model, 82.1/55.9 without trajectory reward, 81.7/55.6 without goal reward, 81.6/55.7 without both, and 79.1/53.3 for SFT cold-start. Setting N to 25, 50, 75, and 100 gives LIBERO success rates of 84.0%, 84.6%, 84.4%, and 83.7%; the authors choose 75 to reduce planning frequency while retaining similar performance. B.7. Inference Speed reports 17% longer average execution time than OpenVLA on LIBERO using an A100 GPU.

Brief Thoughts

The controller-baseline comparisons and reward ablations provide the clearest evidence for ThinkAct: demonstration-based visual feedback improves the reported results beyond question-answering RL alone. The 0.5-percentage-point overall LIBERO advantage over CoT-VLA is small, and the comparison provides no uncertainty estimates. The manipulation evaluations use simulation benchmarks; results on real-world videos do not establish reliability during physical deployment. Self-correction is illustrated by selected examples, and the reported 17% execution-time overhead provides a measured cost for the added planning.