Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale | Summary

Contents

• Summary
• 1 Introduction
• 2 Related work
• 3 Problem setting: environments, tasks and grounded grading
  • 3.1 Environments, tasks and worlds
  • 3.2 Grounded grading
  • 3.3 Depth, defined operationally
• 4 Echoverse: the factory and the co-evolution loop
  • 4.1 An agent workforce
  • 4.2 Phase 1: building the environment
  • 4.3 Phase 2: growing the corpus
  • 4.4 Seeding
  • 4.5 Determinism, isolation and reset
  • 4.6 One rollout, two signals
  • 4.7 Repair before training, and why one fix clears many tasks
  • 4.8 From tasks to trajectories
• 5 The environment suite
  • 5.1 Ten deep domains
  • 5.2 Two capability worlds
  • 5.3 Twelve training worlds, fourteen evaluation splits
• 6 Supervised experiments
  • 6.1 Setup
  • 6.2 Deep worlds transfer; shallow worlds set the model back
  • 6.3 The full scorecard: base, our model and a frontier reference
  • 6.4 Precision about one skill
  • 6.5 Transfer to the live web
  • 6.6 What scaling buys, and what it does not
  • 6.7 The model is not the only thing that learns
• 7 Reinforcement learning on the worlds
  • 7.1 Imitation carries a ceiling
  • 7.2 Why the live web is not an RL environment
  • 7.3 Method
  • 7.4 Setup
  • 7.5 Results
• 8 Conclusion
• Appendix
• Brief Thoughts

This article explains the key points of Echoverse: Deep, Evolving Environments for Training Computer-Use Agents at Scale.

• 2026-07-30 (arXiv), arXiv
• Pandya, Yash, Gupta, Sahil, Harne, Sarthak, Yadav, Archana, Chourasia, Kavyansh, Mozannar, Hussein, Vineet, Vibhav, Abdali, Sara, Rosset, Corby, Lara, Yash, et al.
• Microsoft Research
• Paper

Read this article in Korean

Summary

• Echoverse builds stateful synthetic applications for training computer-use agents, defining a trainable world as an environment, task corpus, and database-grounded verifier. It argues that behavioural depth, targeted capability variation, and co-evolution of the world with the model matter more than raw environment count. A Qwen3.5-9B model trained on 21,009 verified trajectories from twelve worlds improves from 36.5% to 67.1% across fourteen splits; RL over five worlds raises held-out grounded score from 58.8% to 68.0%.

1 Introduction

The paper argues that consequential computer use occurs in login-gated, stateful applications that cannot safely support large-scale training interaction. It therefore proposes controlled clones whose databases can be reset and inspected, with a loop that treats each graded rollout as evidence of either model failure or defects in the environment, task, or verifier.

The diagram separates two consequences of one graded rollout: world defects feed repair, while failures surviving triage feed model training. This is the central co-evolution mechanism.

2 Related work

Compared with live-web benchmarks, Echoverse provides writable state, exact reset, and outcome checks derived from backend data rather than screenshot-based judgments. It complements synthetic suites that scale environment count by varying interior quality, particularly workflow completeness, and uses verifier-filtered rejection-sampled distillation before RL.

3 Problem setting: environments, tasks and grounded grading

The problem setting distinguishes an application environment from a world, the latter adding a task corpus and verifier. Database state is the source of truth, allowing reproducible reset, grounded grading, and reward construction.

3.1 Environments, tasks and worlds

An environment is specified as E = (S, A, T, O, D0): database state, browser actions, backend transition function, observation mapping, and seeded database. A task is τ = (g, r, κ), with natural-language goal g, database-derived reference r, and kind κ ∈ {READ, WRITE, READ_WRITE}; a world is W = (E, T, V).

3.2 Grounded grading

WRITE tasks compare the required state change with the sqldiff from pristine D0 to final DT; READ tasks compare the reported answer with a database-derived reference; READ_WRITE uses the lower of the two scores. An LLM performs only a constrained semantic comparison of observed output or SQL diff against a reference, without seeing screenshots or trajectories.

3.3 Depth, defined operationally

Depth is defined operationally as support for every workflow demanded by a task corpus, not as feature count or visual realism. The required properties are behavioural fidelity, coherent state, workflow dependency, authoritative state-based verification, and domain value; support is established through machine-checkable claims and interface-executed tasks.

The table turns depth into five concrete requirements rather than a visual or feature-count judgment. The first four concern structural support and verification of workflows; domain value determines which workflows justify that work.

4 Echoverse: the factory and the co-evolution loop

Echoverse has two connected phases: compile hand-written seeds into a validated application, then generate and repair a grounded task corpus over its live database. Failures remain part of the loop after shipment because a graded rollout may expose a model capability gap or a defect in the coupled world stack.

The factory illustration shows that corpus construction can trigger environment fixes, not merely task edits. Both phases repeatedly verify against database-grounded checks before progressing.

4.1 An agent workforce

GitHub Copilot SDK agents act as builders, verifiers, triagers, and layer-specific fixers, with terminal, Playwright, and database tools. Separating construction from verification reduces self-judging, while triage identifies shared causes so one repair can unblock multiple tasks; human supervision and QA provide further oversight.

4.2 Phase 1: building the environment

Phase 1 expands seed scenarios into specifications and machine-checkable claims, then generates a FastAPI and SQLite backend with a React interface. A world advances when at least 95% of claims pass and no hard blocker remains; claims cover the routes, state, and behaviour required by target workflows.

4.3 Phase 2: growing the corpus

Phase 2 regrounds task goals on entities in the live database and checks entity existence, plausibility, difficulty, and interface feasibility. White-box verifier agents inspect source and database state while driving the UI; failures are assigned to the database, backend, frontend, task text, or verifier and repaired until at least 95% of the surviving corpus passes.

4.4 Seeding

ECHOSTAY uses InsideAirbnb data, while ECHOFORUM uses a public corpus of 2.55 million comments. Domains without suitable public data, including mail, calendar, banking, and health records, are seeded under constraints intended to create dense, internally consistent state rather than placeholder rows.

4.5 Determinism, isolation and reset

Every rollout uses an isolated per-task database copy, preventing cross-task contamination and enabling exact reproduction. Seed-data changes trigger regeneration, revalidation, and republication of the grounding database so that the corpus and released artefact do not drift.

4.6 One rollout, two signals

Each failure is attributed either to the model or to the world stack. The system repairs ambiguous cases rather than treating them as supervision, since unreachable goals, broken interfaces, or incorrect verifiers would otherwise turn environment noise into a capability target.

4.7 Repair before training, and why one fix clears many tasks

Repairs precede training, and the process forbids weakening goals merely to improve pass rates. Triage prioritizes shared causes: a broken interaction control can block many superficially distinct tasks, so repairing it can improve corpus validity and supervision quality together.

4.8 From tasks to trajectories

After validation, GPT-5.4 solves tasks in the live world and only trajectories passing the grounded verifier become supervised data. This is verifier-filtered, rejection-sampled distillation: factory feasibility checks determine task existence rather than teacher success, but the resulting model remains bounded by the teacher’s successful demonstrations.

5 The environment suite

The suite contains ten deep full-domain worlds and two capability worlds. Full domains model stateful workflows in communication, technical operations, regulated transactions, and community, media, and travel; capability worlds isolate controls whose variation is difficult to obtain in a single application.

5.1 Ten deep domains

The ten domains are ECHOMAIL, ECHOCALENDAR, ECHOCHAT, ECHOML, ECHOFORGE, ECHOBANK, ECHOCARE, ECHOFORUM, ECHOTUNES, and ECHOSTAY. Their workflows often span five to thirty actions and require persistent effects across screens, users, permissions, histories, and backend state.

The suite covers four workflow categories and identifies the stateful structure each category must preserve. It supports the claim that domains are organized around workflows rather than product replicas.

The first ten panels depict full-domain applications, while the last two are held-out renderings of capability worlds. The juxtaposition illustrates the intended distinction between workflow depth and control-level variation.

The shipped-database counts evidence nontrivial grounded state, ranging from thousands to millions of rows. ECHOFORUM has 4.0M rows and ECHOSTAY has 180K, while the table records domain-specific entities.

5.2 Two capability worlds

The datepicker world uses six core widgets across ten contexts, 100 frontends, and six visual styles; it holds out ten unseen widget types across 36 scenarios and 80 frontends in nine styles. The nested-filter world uses twenty training widget families across 200 frontends and holds out nine compound panel families across 100 frontends, grading whether returned results satisfy requested conditions.

The distributions show that capability-world controls are re-themed across multiple verticals, reducing reliance on a single familiar layout. Nested filters cover six verticals and date pickers ten contexts.

5.3 Twelve training worlds, fourteen evaluation splits

Training uses twelve worlds: the ten full domains and two capability worlds. Evaluation uses fourteen splits, adding in-distribution and held-out splits for each capability world; reported averages are unweighted over these fourteen splits.

6 Supervised experiments

Supervised experiments compare a browser-action-aligned Qwen3.5-9B base policy, the same 9B model fine-tuned on 21,009 verified trajectories, and GPT-5.4, which generated the accepted demonstrations. Rollouts are capped at 100 actions; synthetic tasks use database-grounded scoring, while live-web transfer uses the published WebVoyager and Online-Mind2Web judges.

6.1 Setup

The supervised corpus covers twelve worlds and 404,814 total actions. Policies are served with vLLM, and the paper reports both hosted-browser and datacentre live-web conditions because bot blocking and rate limits depress raw live-site performance.

6.2 Deep worlds transfer; shallow worlds set the model back

On matched-domain, comparable-scale ablations, shallow-world training reduces Allrecipes success from 80.0% to 75.0% and leaves Hugging Face at 48.0%, whereas deep-world training reaches 85.0% and 65.0%. The deep model also reduces Hugging Face tasks exhausting the step budget from 15 to 9 of 37, but the authors frame this two-domain result as evidence that shallow worlds can harm transfer rather than a general estimate of depth’s value.

The bars provide the direct deep-versus-shallow contrast: deep training improves both live WebVoyager domains, while shallow training harms Allrecipes and does not improve Hugging Face.

6.3 The full scorecard: base, our model and a frontier reference

Across fourteen synthetic splits, πSFT improves the 9B base average from 36.5% to 67.1%, versus 80.7% for GPT-5.4. It matches or exceeds the frontier reference on ECHOBANK, ECHOMAIL, and both nested-filter splits, but remains substantially weaker on held-out datepickers (52.0% versus 94.7%) and ECHOSTAY (37.6% versus 50.4%).

This main supervised scorecard shows broad gains for πSFT over the base model and the remaining gap to GPT-5.4. It also identifies splits where the 9B model matches or exceeds the teacher reference.

The plot decomposes each split into progress from the base model to πSFT and remaining distance to GPT-5.4. It makes the uneven residual gap visible, especially for held-out datepickers and several deep-domain worlds.

6.4 Precision about one skill

Capability-only training transfers across layouts and between skills: joint datepicker and nested-filter training yields 83.5% in-distribution and 57.3% held-out datepicker success, plus 88.0% and 84.8% on nested filters. The held-out gains support control-level generalization, though the held-out datepicker gap to GPT-5.4 remains substantial.

The ablation shows that each capability-world training regime improves both control types, with joint training best on all four splits. The held-out values quantify generalization to widget families omitted from training.

The grouped bars reinforce that gains are not limited to trained layouts or controls. Joint training has the highest score for every datepicker and nested-filter split.

6.5 Transfer to the live web

Without live-web training data, πSFT improves WebVoyager from 66.5% to 71.5% in hosted conditions and from 50.9% to 55.6% in datacentre conditions; Online-Mind2Web rises from 40.5% to 43.4% hosted and 29.5% to 37.2% datacentre. The paper characterizes these as modest transfer gains because the synthetic worlds emphasize login-gated, write-heavy workflows while those benchmarks are largely public and read-oriented.

6.6 What scaling buys, and what it does not

Adding trajectories within a fixed world mixture continues to improve synthetic performance but has little live-web transfer benefit: from 6,400 to 20,000 trajectories, WebVoyager changes only from 54.8% to 55.6% and Online-Mind2Web falls from 40.1% to 37.2%. Adding distinct environments instead continues to raise both synthetic averages and WebVoyager, supporting breadth rather than raw trajectory volume as the transfer lever.

6.7 The model is not the only thing that learns

Co-evolution identifies environment-side failures that ordinary QA can miss. In ECHOSTAY, repairing a guest-count control increased completable affected bookings from 48% to 78% and, across two loop rounds, raised a model trained on that world’s corpus from 16.2% to 38.5%; because environment, corpus, and verifier changed together, the paper does not claim an isolated causal contribution for an individual repair.

The chart reports the end-to-end effect of two ECHOSTAY co-evolution rounds: the world-specific model score rises by 22.3 points, from 16.2% to 38.5%. It also shows the remaining gap to GPT-5.4 at 50.4%.

7 Reinforcement learning on the worlds

The RL stage addresses the limitation of imitation: accepted teacher trajectories do not show how the student should notice, recover from, or stop after its own mistakes. Controlled worlds provide resettable, high-throughput, state-grounded episodes that live websites cannot safely provide.

7.1 Imitation carries a ceiling

A verifier-filtered demonstration corpus is clean but can teach only the teacher’s successful behavior and recovery patterns. RL instead evaluates the policy’s own trajectories, making its recurrent errors available for optimization.

7.2 Why the live web is not an RL environment

The paper argues that the live web fails RL requirements for exact reset, throughput, trustworthy reward, stationarity, and safe destructive actions. Echoverse instead provides per-task snapshots, parallel self-contained applications, fixed seed data, and grounded outcome checks.

The comparison explains why the authors treat synthetic worlds, rather than the live web, as RL substrates. Echoverse satisfies reset, episode-volume, reward, stationarity, and safety requirements through database ownership and isolation.

7.3 Method

Starting from πSFT, the method uses group-relative policy gradient with a low-variance KL penalty to a fixed reference. Each row receives Rrow = Rtraj + λRstep with λ = 1 and Rstep ∈ {0, 1}; database-grounded trajectory outcome is combined with a GPT-4.1 vision per-step judge that rewards actions demonstrably taking effect. Advantages are centered over all step rows sharing a prompt, Arow = Rrow − mean(Rgroup), without division by group standard deviation.

7.4 Setup

RL trains on ECHOBANK, ECHOFORGE, ECHOFORUM, ECHOSTAY, and ECHOTUNES using 32 GPUs. It retains tasks solved one to three times in four starting-policy samples, yielding 353 unique training tasks and 125 disjoint validation tasks; each update uses 16 prompts with group size 8, and rollout length is capped at 50 turns.

The pool is balanced across five worlds and has zero train-validation overlap. Upsampling is used only for ECHOTUNES and ECHOBANK to reach the target range of training rows.

The configuration specifies group-relative policy-gradient training with 16 prompts, group size 8, 50-turn rollouts, a 1 × 10−7 learning rate, and trajectory and step rewards. It makes the evaluation distinction important: dense judged shaping is used in training only.

7.5 Results

At step 50, a little over two epochs, held-out grounded trajectory score rises from 58.8% to 68.0% across 25 validation tasks per world. Training reward also trends upward, but it includes dense per-step judgment and is therefore not directly comparable to the held-out outcome score.

The held-out grounded validation score rises to 68.0% by step 50, while training reward is noisier because it includes the dense step signal. The separate panels prevent treating the two quantities as the same metric.

8 Conclusion

The conclusion positions deep, repairable, state-grounded worlds as both training infrastructure and benchmark substrate. The release includes code and graded evaluation tasks for ECHOSTAY, ECHOFORGE, and the two capability worlds, while withholding training tasks to reduce benchmark contamination; code is available at https://aka.ms/echoverse.

The catalogue identifies twenty nested-filter training widget families and nine held-out compound families, making the held-out generalization setting concrete. Training and evaluation differ by widget family, not just task instances.

The datepicker catalogue specifies six training families and ten unusual held-out types, including heatmaps, scroll wheels, ISO-week pickers, and fiscal-year pickers. This supports interaction transfer across visibly and structurally distinct controls.

Appendix

• Appendix B reports 31 task sets, 21,009 passing GPT-5.4 trajectories, and 404,814 steps; ECHOSTAY trajectories are longest at 33.80 actions on average, whereas nested-filter trajectories average 4.70. The verifier prompts permit semantic equivalence and benign extra writes, while the widget catalogue specifies that held-out families are evaluation-only.

Brief Thoughts

The paper’s strongest evidence is the controlled contrast between deep and shallow worlds, the grounded evaluation design, and the explicit acknowledgement that co-evolution changes multiple coupled components at once. Its external-validity evidence is encouraging but limited: live-web gains are modest, supervised data inherit a GPT-5.4 ceiling, and RL combines reliable terminal verification with a model-judged shaping signal during training.