Monthly AI Papers: September 2026

Read this article in Korean

Trends

September 2026’s theme is testing whether measured capability survives practical use: the three selected papers examine whether improvements in search scores, mixed-input predictions, or written problem solving translate into unseen-task performance, separate text generation, or interactive execution. RRSI studies an agent harness, the prompts, tools, memory, and control flow surrounding a language model. The superposition study averages token embeddings, vectors representing input tokens, to test how much information survives from two texts. VHD-Play generates environments with persistent state from solved mathematical decision models. Across these studies, evaluation separates intermediate measurements from the outcomes those measurements are intended to support. 1, 2, 3 For agent harness, see Summary explanation: Agent Safety Should Be a Runtime Contract. For language model, see DuPO: Enabling Reliable LLM Self-Verification via Dual Preference Optimization | Summary.

Agent improvement depends on how feedback is constructed and reused. RRSI, Regularized Recursive Self-Improvement of Agent Harnesses, constrains candidate edits and their acceptance to reduce overfitting to repeatedly evaluated tasks. Removing its controls raises search-task scores while reducing performance on unseen benchmarks. VHD-Play, Verifiable Hidden Dynamics Play, derives environment behavior and scoring references from the same solved model before generating the player-facing interface. Its comparison of written and interactive presentations shows a large execution gap even when all parameters are disclosed. Together, these results support checking both the reliability of feedback and transfer beyond the conditions supplying it. For Recursive Self-Improvement, see The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement | Summary.

Efficiency claims also require a clear comparison baseline. RRSI’s final workspace harness uses fewer language-model execution tokens than unregularized evolution, but more than the initial harness; those measurements exclude total search cost. In the superposition study, preserving the average prediction distribution does not establish reliable generation of two separate continuations. The strongest guided decoding results remain below the independent small guide models’ accuracy, and guidance adds computation. The two-head variant, which uses separate output mappings for the streams, approximately matches ordinary two-input batched throughput. These findings make execution cost and task quality necessary companions to intermediate fidelity metrics.

Top Papers in September 2026

1. RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

• https://arxiv.org/abs/2609.24972
• RRSI: Regularized Recursive Self-Improvement of Agent Harnesses | Summary
• RRSI keeps model weights fixed while evolving the harness. It gradually reduces the number of edits bundled into each candidate, retains earlier evaluation evidence, explores underused components, screens benchmark-specific logic, and applies performance and token-cost checks before accepting changes. For harness, see Meta-Harness: End-to-End Optimization of Model Harnesses | Summary.
• The main experiment improves all six held-out evaluation splits relative to the initial harness. SWE-bench Verified, a repository bug-fixing benchmark, rises from 82.0 to 83.8. Three unseen professional-work benchmarks gain 3.5–4.7 points. Frontier-Eng, an engineering optimization benchmark, rises from 17.7 to 22.0 Medal points, credit for reaching ranked performance thresholds.
• On unseen workspace benchmarks, full RRSI averages 43.6, while removing both proposal and acceptance controls yields 40.3 despite a higher search-task score of 92.8 versus 90.5. On the evolve split, full RRSI uses 2.42 million policy tokens per trial, tokens consumed by the fixed language model, compared with 3.80 million without both controls. The tabulated costs imply approximately 36% lower execution-token use; the initial harness remains cheaper at 1.56 million tokens on the same evolve split.

2. Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

• https://arxiv.org/abs/2609.29845
• Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs | Summary
• Your Transformer Can Hold Two Thoughts at Once tests linear superposition: whether averaging two texts’ input representations approximately preserves the average of their independent next-token probability distributions. In illustrated unmodified models, approximately 30–40% of independent highest-probability predictions remain among the mixed output’s top ten. This measures agreement with independent predictions, rather than correctness against observed text.
• Training against a frozen copy’s averaged independent predictions reduces Pythia-2.8B’s reported Kullback–Leibler divergence, a measure of distribution mismatch, from 1.86 to 0.27. For Qwen2.5-3B, content-position median rank improves from 284 to 5. Ordinary single-stream quality declines: its LAMBADA accuracy, measured on word prediction requiring broad narrative context, falls from 0.602 to 0.460.
• Joint Contrastive decoding adds a small model’s guidance for one stream and subtracts guidance for the other. Llama-3.2-3B mixed LAMBADA accuracy rises from 0.182 to 0.430, below its small guide model’s independent accuracy of 0.540. Deployment efficiency remains unproven: at prompt and generation lengths of 128, Pythia-2.8B’s two-head variant reaches 109.6 ± 0.8 tokens/s versus 113.1 ± 0.6 for ordinary batch-size-2 inference, while guided decoding reaches 79.4 ± 0.5; intervals are 95% confidence intervals over 100 runs.

3. Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanisms

• https://arxiv.org/abs/2609.27321
• Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanisms | Summary
• VHD-Play solves a sampled mathematical decision model before generating its executable environment and interface. The same model supplies state transitions and scoring references, avoiding a language-model judge. The normalized reward places a fixed default strategy at 0 and a full-information upper reference at 1; that upper reference may be unattainable when the player must act with partially hidden information.
• The corpus contains 2,200 training environments and 1,100 evaluation environments across eleven mechanism families, groups of decision problems sharing mathematical structure. After training Qwen3.6-35B-A3B on three families using reinforcement learning, learning from outcome rewards, mean interactive score across five optimization families rises from 0.204 to 0.815. With parameters revealed upfront, interactive scores rise from 0.231 to 0.875, while written-out problem scores move from 0.962 to 0.992, supporting stateful execution as a substantial learning target.
• External transfer includes function-calling scores rising from 61.25 to 64.08 and travel-plan quality rising from 0.700 to 0.794 on 20 English tasks. In a 365-day simulated business, mean ending balance increases from 54,294 to 182,844 across five runs per model. The main generated-family results use one training run and one evaluation seed. Reference replay, executing reference strategies to check generated outcomes, covers eight of eleven families, limiting the scope of verification. For function-calling, see Towards General Agentic Intelligence via Environment Scaling | Summary.

Sources

[1] RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

[2] Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

[3] Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanisms