WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning | Summary

Contents

Summary
1 Introduction
2 Agentic Framework
3 SailorFog-QA-V2
4 Agentic Post-training
- 4.1 SFT Cold Start
- 4.2 Agentic Reinforcement Learning
5 Experiments
6 Conclusion
Appendix
Brief Thoughts

This article explains the key points of WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning.

2025-09-16 (arXiv)
Li, Kuan, Zhang, Zhongwang, Yin, Huifeng, Ye, Rui, Zhao, Yida, Zhang, Liwen, Ou, Litu, Zhang, Dingchu, Wu, Xixi, Wu, Jialong, et al.
Tongyi Lab, Alibaba Group
Paper
Project Page

Read this article in Korean

Summary

WebSailor-V2 addresses the difficulty of training open-source agents to answer research questions through repeated web searches and evidence synthesis. In the paper’s reported evaluations, its Qwen3-30B-A3B agent scores 35.3 on BrowseComp-EN, a benchmark for finding difficult information online, compared with 30.0 for DeepSeek-V3.1-671B, although it trails that baseline on the Chinese counterpart BrowseComp-ZH, 44.1 versus 49.2.
SailorFog-QA-V2 generates question-answer (QA) pairs from a knowledge graph, a network of entities and their relationships, with deliberately dense connections and cycles. Random walks, which sample paths through connected nodes, and coverage of different structural node roles broaden the generated tasks; the paper also proposes uncertainty beyond obfuscation, which replaces specific entities, dates, or values with ambiguous descriptions, but does not enumerate the additional uncertainty types.
Training combines supervised fine-tuning (SFT), learning from generated solution trajectories, with reinforcement learning (RL), updating the agent’s policy, or rule for choosing outputs and actions, from rewards. An offline Wikipedia simulator supports algorithm development, while a managed live-web environment supports final training; automated task synthesis and filtering connect data construction to the evolving policy.
RL improves all five reported Table 1 scores over the SFT checkpoint, reaching 73.7 on xbench-DeepSearch, a complex-search benchmark, 74.1 on the 103-case text-only subset of GAIA, a benchmark for general assistants requiring tool use, and 30.6 on Humanity’s Last Exam (HLE), an expert-question benchmark for academic and logical reasoning. Evaluation uses temperature 0.85, which controls output sampling randomness, top-p 0.95, which restricts sampling to a cumulative probability threshold, and a large language model (LLM) judge to assess correctness, while some baseline scores come from website evaluations or external reports.
The reported results support the complete training pipeline and show improved accuracy with larger inference budgets, meaning more context and tool calls available at evaluation time. They do not isolate the contributions of graph density, uncertainty diversity, or environment management; further qualifications include an inconsistent DeepResearch Bench score, incomplete baseline coverage, and an insufficiently documented experiment that trained directly on the BrowseComp testing set.

1 Introduction

The introduction identifies two obstacles to capable open-source research agents: training tasks cover too few logical structures and uncertainty patterns, and large-scale web interaction introduces costly, unreliable tool responses. WebSailor-V2 responds with richer synthetic data and separate simulated and real training environments, connected through automated data curation.

The data-policy feedback loop synthesizes and filters tasks according to training dynamics, allowing the training distribution to change as the policy develops.
The authors choose a simple agent framework to reduce dependence on specialized planning structures and prompt engineering when evaluating their post-training pipeline.

BrowseComp-EN and xbench-DeepSearch scores for selected agents.

2 Agentic Framework

The agent uses ReAct, a framework that alternates thoughts, actions, and observations. Its trajectory, the sequence of these interactions, is H_T = (τ_0, a_0, o_0, …, τ_i, a_i, o_i, …, τ_T, a_T), where τ_i, a_i, and o_i denote the thought, action, and observation in round i; the next thought and action are sampled from π(a_t, τ_t | H_{t−1}), the policy conditioned on previous context.

search returns result titles, snippets, and URLs; visit retrieves a webpage and produces a summary tailored to the current goal.
Google Scholar supports academic searches, and the Python interpreter executes generated code in a sandbox, an isolated execution environment. A final answer action terminates the trajectory.

3 SailorFog-QA-V2

SailorFog-QA-V2 constructs synthetic research tasks from real internet information. Its pipeline builds a dense entity graph, extracts connected subgraphs, or smaller graphs drawn from it, and generates questions intended to require different patterns of information gathering and reasoning.

3.1 Graph Construction

Graph construction starts from a seed entity and uses web tools to discover related entities and facts. The expansion deliberately adds connections between nodes to create cycles, extending beyond the predominantly tree-like structures that the authors associate with easy-to-hard generation, which progressively expands a simple seed question.

The graph retains discovery queries and source URLs, providing procedural information alongside extracted facts.
Statistical features are stored for each entity to support question generation, but the paper does not specify those features or quantify the resulting graph’s size and density.

3.2 Subgraph Extraction

Subgraph extraction uses random walks because enumerating all substructures with a fixed number of edges becomes infeasible in a denser graph. The authors describe collecting connected, non-isomorphic subgraphs, meaning structurally distinct graphs, and using the Weisfeiler-Leman algorithm, a graph-structure comparison procedure, for verification.

The intended benefit is broad structural coverage without exhaustive enumeration. Sampling parameters, coverage measurements, and runtime comparisons are not reported.

3.3 QA Generation

Before generating questions, the pipeline analyzes node orbits, groups of nodes with equivalent structural roles, to distribute question targets across different roles. Obfuscation creates uncertainty by replacing key facts with less specific descriptions, requiring disambiguation, hypothesis testing, and evidence synthesis.

The paper states that V2 introduces additional uncertainty definitions, but provides neither a taxonomy nor worked examples of those additional types.
Without those details or component comparisons, the effects of graph structure and uncertainty design cannot be assessed separately.

4 Agentic Post-training

Agentic post-training begins with an SFT cold start, an initial policy learned from example trajectories, and then applies RL using simulated and real environments. Figure 2 links asynchronous rollout collection, in which agent interactions are gathered without requiring all workers to finish together, with reward processing, policy updates, and automatic synthetic data construction.

Training architecture connecting synthetic data, rollouts, rewards, and simulated or real environments.

4.1 SFT Cold Start

The SFT stage uses synthetic SailorFog-QA-V2 tasks and trajectories generated by open-source models through rejection sampling, which filters candidate solutions before training. This section identifies Qwen3-30B-A3B-Thinking-2507 as the base model and increases its context length, the amount of interaction history it can process, to 128k.

The purpose is to establish an initial policy that can complete enough difficult tasks to obtain useful positive rewards during RL.
The paper does not report the SFT dataset size, trajectory-generating model identities, or detailed rejection criteria.

4.2 Agentic Reinforcement Learning

The simulated environment combines an offline Wikipedia database with corresponding web tools and synthetic training and testing tasks. The real environment wraps search sources, webpage parsers, and code execution behind a unified interface with queries-per-second limits, caching, timeout retries, fallback sources, and reduced service functionality for non-critical failures.

Automated synthesis and filtering dynamically adjust the training set according to training dynamics. The paper describes faster iteration and reduced operational costs without reporting measured speedups, costs, or simulation-to-real fidelity tests.
The RL algorithm adapts GRPO, a policy optimization method that compares rewards within groups of sampled outputs. It uses on-policy trajectories sampled from the current policy, a token-level clipped objective that limits changes in output probabilities, and a stated leave-one-out strategy that excludes a sample from its comparison baseline.
Certain failed trajectories, including outputs truncated before a final answer, are excluded from the loss to mitigate format collapse, or failure to maintain the required output format. Dynamic sampling, which changes the sampled training examples during optimization, is omitted in favor of larger batch and group sizes.

Equation (3) defines the importance ratio, which compares current and sampling-policy token probabilities, as r_{i,t}(θ) = π_θ(o_{i,t} | context) / π_{θ_old}(o_{i,t} | context), and the advantage, a reward relative to a comparison baseline, as $\hat{A}_{i,t}=R_i-\operatorname{mean}(\{R_i\}_{i=1}^{G})$. The displayed mean-centered advantage does not explicitly express the leave-one-out estimator described in the prose.

The authors also report poorer results when training directly on the BrowseComp testing set than when using synthetic data. They hypothesize that synthetic tasks have a more consistent learnable distribution, but provide no numerical results or sufficient details to establish how this experiment relates to the reported final checkpoint.

5 Experiments

Experiments evaluate answer correctness, research-report performance, training dynamics, and sensitivity to context and tool-call budgets. The main quantitative comparison includes both SFT and RL checkpoints, showing the additional improvement after the RL stage.

5.1 Setup

The setup lists six benchmarks: BrowseComp-EN, BrowseComp-ZH, GAIA, xbench-DeepSearch, HLE, and DeepResearch Bench, which evaluates research reports and information retrieval. GAIA evaluation is restricted to 103 text-only validation cases and therefore does not cover its full set of tasks involving multiple input modalities.

Training data primarily combines SailorFog-QA and SailorFog-QA-V2, supplemented by IterBench for mathematical and academic reasoning. The IterBench reference is the placeholder “authors. title, 2025,” limiting source traceability.
Evaluation reports pass@1, the fraction of correct single-attempt responses, at temperature 0.85 and top-p 0.95. For pass@k with k > 1, the paper generates k attempts; an LLM judge assesses correctness, but its identity and judging procedure are not specified.
Some proprietary agents are only accessible through web interfaces, and cost constraints leave missing baseline results. Figure 1 also combines official release scores with independent tests using the authors’ browsing tools.

5.2 Main Results

Table 1 reports SFT-to-RL improvements from 24.4 to 35.3 on BrowseComp-EN, 28.3 to 44.1 on BrowseComp-ZH, 61.7 to 73.7 on xbench-DeepSearch, 66.0 to 74.1 on GAIA, and 23.9 to 30.6 on HLE. These correspond to gains of 10.9, 15.8, 12.0, 8.1, and 6.7 percentage points, respectively.

WebSailor-V2 has the highest reported Table 1 scores on xbench-DeepSearch, GAIA, and HLE. Its BrowseComp-EN score exceeds the listed open-source baselines but remains below OpenAI-o3 at 49.7 and OpenAI DeepResearch at 51.5.
On BrowseComp-ZH, DeepSeek-V3.1-671B scores 49.2 and OpenAI-o3 scores 58.1, both above WebSailor-V2’s 44.1. The table therefore contradicts the prose claim of superiority over every open-source agent on both BrowseComp benchmarks.
The SFT checkpoint is already competitive with several trained agents. The authors interpret this as support for a strong cold start under sparse rewards, meaning successful outcomes are infrequent, but do not provide an experiment that removes SFT before RL.

Benchmark scores for proprietary agents, open-source agents, and WebSailor-V2 SFT and RL checkpoints.

5.3 More Comparison with Proprietary Agents in Deep-research Task

DeepResearch Bench compares research-report quality and information retrieval against proprietary research agents. Figure 3 labels WebSailor-V2 at 47.7, below Gemini-2.5-pro-DeepResearch at 49.7 and above OpenAI-Deepresearch at 46.5; the accompanying prose instead reports 48.9.

The disagreement leaves the exact WebSailor-V2 score unresolved, although both values place it second among the displayed agents.
The authors attribute the remaining gap to less emphasis on report style during training. No component-level report scores or controlled analysis establish that explanation.

Overall DeepResearch Bench scores for WebSailor-V2 and proprietary research agents.

5.4 Detailed Analyses

Figure 4 shows rising training reward and improving, initially fluctuating BrowseComp validation accuracy. Figure 6 reports pass@1/pass@3 gains, where pass@3 measures success with three attempts, of 10.9/7.5 points on BrowseComp-EN, 15.8/12.2 on BrowseComp-ZH, 12.0/3.0 on xbench-DeepSearch, and 8.1/2.9 on GAIA.

The authors interpret gains in both one-attempt and three-attempt success on BrowseComp as broader problem-solving improvement, and the smaller pass@3 gains on xbench-DeepSearch and GAIA as improved first-attempt reliability. These patterns show different effects across tasks but do not directly measure reasoning capacity.

BrowseComp validation accuracy and training reward over RL steps.

Figure 5 shows that BrowseComp-EN accuracy rises with context length and tool-call budget before flattening. The agent increases context from WebSailor-V1’s 32k to 128k and permits up to 100 ReAct iterations; the text reports around 16 accuracy at 32k and nearly 90% of correctly solved instances completed within 64k.

The budget curves show that the headline accuracy depends on substantial inference resources. The paper does not provide matched end-to-end cost or latency comparisons with competing agents.

BrowseComp-EN accuracy as a function of context length and tool-call budget.

Pass@1 and pass@3 accuracy improvements from RL across four benchmarks.

Figure 7 shows policy entropy, a measure of uncertainty in the policy’s output distribution, remaining relatively high and fluctuating during training, with a decline near the end. The implementation omits an explicit entropy bonus, a reward term encouraging output diversity, because the authors judge exploration sufficient.

The authors hypothesize that changing web observations sustain exploration. The paper does not test this explanation by comparing fixed and changing observation distributions under otherwise matched conditions.

Raw and smoothed policy entropy over RL training steps.

6 Conclusion

The conclusion presents WebSailor-V2 as a systems approach combining synthetic task construction, SFT initialization, and RL in controlled and managed environments. The authors argue that data quality and environment stability matter more than the particular RL algorithm; the reported results establish the complete pipeline’s performance without isolating the relative importance of those components.

Appendix

A Related Work organizes prior research around data construction, agent training strategies, and inference paradigms. It contrasts graph-based task generation with easy-to-hard expansion, discusses SFT followed by GRPO-style RL, and positions standard ReAct as a way to reduce confounding from elaborate context management.
B Experimental Details specifies that search returns the top-10 Google results per query, while visit uses Jina, a webpage retrieval service, to retrieve pages and Qwen3-30B-A3B to summarize information for a stated goal. SFT uses Megatron, a model-training framework, batch size 64, learning rate 5e-6 with minimum 1e-10, warmup plus cosine decay, a schedule that first raises and then lowers the learning rate, and weight decay 0.1, a parameter regularization setting; RL uses rLLM, a framework for post-training language agents, temperature 1.0, top-p 1.0, batch size 128, and learning rate 1e-6.
C Case Study presents a 29-step BrowseComp trajectory identifying FormFactor, Inc. after initial unsuccessful searches, a targeted leadership-change query, and checks of incorporation, customer concentration, and litigation. The displayed answer matches April 15, 1993 incorporation, a fiscal third-quarter 2008 leadership transition, three customers contributing 47.3% of 2006 revenue, four customers contributing 64.8% in 2004 and 72.8% in 2005, and a $1.5 million settlement; many underlying tool responses are omitted, so the example illustrates the workflow without exposing the complete evidence returned to the agent.

Brief Thoughts

For an ML engineer, the practical contribution is the integration of task generation, trajectory filtering, and reliable tool execution into agent training. The SFT-to-RL gains document improvement across all five Table 1 benchmarks, while the budget curves show how evaluation resources affect accuracy.

The competitive results are benchmark-specific: Table 1 contains a stronger BrowseComp-ZH open-source baseline, and proprietary agents remain ahead on BrowseComp-EN. Component ablations, experiments that remove or vary individual design choices, along with uncertainty definitions, dataset statistics, judge details, clarification of test-set training, and reconciliation of the DeepResearch Bench score, would strengthen reproducibility and causal interpretation.