Monthly AI Papers: May 2026

Contents

Trends
Top Papers in May 2026
Sources

Trends

The three papers selected for May 2026 examine adaptation through persistent state: δ-mem, SkillOS, and AutoScientists all carry information from earlier interactions or experiments into later decisions. They store different kinds of experience. δ-mem maintains associative memory, a compact matrix of learned key–value relationships; SkillOS edits procedural skills, reusable instructions stored as Markdown files; AutoScientists shares experimental results, rejected directions, and a champion, the current best accepted program. These designs make the representation and revision of accumulated experience central parts of the system. 1, 2, 3

The papers also separate accumulated knowledge from the model or agent that uses it. δ-mem trains a memory interface while keeping the backbone, the pretrained language model, frozen; its memory content then changes during inference. SkillOS trains a curator to manage an external repository for a frozen executor, the agent solving tasks. AutoScientists coordinates successive agent sessions through shared experimental records and reorganizes teams when progress stalls. Across these approaches, adaptation depends on how stored information influences subsequent computation, as well as what gets retained.

Their efficiency results require different interpretations. δ-mem’s small trainable interface accompanies slower token-by-token decoding. SkillOS reduces executor actions, but those counts exclude curation, judging, and training. AutoScientists improves experimental search under matched hardware and time budgets while spending more on language-model calls. Together, the evidence supports useful reuse of experience in the evaluated settings; it does not establish lower total operating cost or uniformly better behavior across tasks.

Top Papers in May 2026

1. $\delta$-mem: Efficient Online Memory for Large Language Models

https://arxiv.org/abs/2605.12357
$\delta$-mem: Efficient Online Memory for Large Language Models | Summary
δ-mem: Efficient Online Memory for Large Language Models adds a persistent matrix to a frozen full-attention Transformer, retaining the backbone’s attention over available tokens. At each position, it reads the previous memory, corrects the attention query and output, and then writes new information. Its gated delta rule combines controlled forgetting with correction of associations the existing state predicts incorrectly.
Writing granularity changes the memory behavior. Token-State Write (TSW) updates after every token; Sequence-State Write (SSW) updates once per message using its mean hidden representation; Multi-State Write (MSW) maintains parallel sub-states. At rank r = 8, the single-state variants use an 8 × 8 matrix per insertion layer and 4.87M trainable parameters. MSW uses four such sub-states and 19.47M trainable parameters; these parameter counts are separate from runtime memory-state storage.
On Qwen3-4B-Instruct, TSW raises the reported five-benchmark average from 46.79 to 51.66. That average uses HotpotQA exact-match scoring and combines different benchmark metrics. The strongest variant also improves the aggregate from 47.20 to 50.86 on Qwen3-8B and from 26.08 to 36.96 on SmolLM3-3B, but the best writing strategy changes across backbones.

δ-mem architecture with associative-state reading, low-rank attention steering, delta-rule writing, and three writing strategies.

No-context baseline scores and gains from compressed-memory state on HotpotQA and LoCoMo.

2. SkillOS: Learning Skill Curation for Self-Evolving Agents

https://arxiv.org/abs/2605.06614
SkillOS: Learning Skill Curation for Self-Evolving Agents | Summary
SkillOS: Learning Skill Curation for Self-Evolving Agents separates task execution from memory management. A frozen executor retrieves relevant Markdown skills through BM25, a keyword-based retrieval method. After execution, a trainable curator inserts, updates, or deletes skills using the trajectory and a self-judged correctness signal, changing the repository available to subsequent tasks.
Curator training processes groups of related tasks so that later outcomes evaluate earlier repository edits. Its reinforcement-learning reward combines downstream success, valid repository operations, judged content quality, and compression. On ALFWorld, a benchmark of household tasks, replacing related groups with random task sequences lowers success from 61.2% to 57.3%, the largest drop among the reported grouping and auxiliary-reward ablations.
With Qwen3-8B as executor, reported ALFWorld success rises from 47.9% without memory to 61.2 ± 4.6%, while average actions fall from 21.1 to 18.9. The trained curator also exceeds the untrained SkillOS-base result of 53.1%, supporting a benefit from learning curation under the same repository interface. With Gemini-2.5-Pro as an unseen executor, success increases from 66.4% without memory to 80.2 ± 3.1%. Reported means and standard deviations come from three runs.

Streaming execution–curation loop and the Markdown skill representation used by SkillOS.

Grouped-task training pipeline with repository evolution and composite-reward optimization.

3. AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation

https://arxiv.org/abs/2605.28655
AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation | Summary
AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation coordinates computational research without a central planner agent. Analysts critique and queue proposals; experiment agents modify and evaluate the shared champion. Persistent logs and dead-end registries carry evidence between sessions, while renewed discussion can merge, split, or retire teams as research directions change.
Across 24 BioML-Bench biomedical machine-learning tasks, mean leaderboard percentile reaches 74.40% with a standard error of 6.20%, versus 66.07% with a standard error of 7.38% for Autoresearch, an 8.33 percentile-point gain. AutoScientists, Autoresearch, and rerun Biomni receive one H100 GPU, 16 CPUs, and 48 GB memory for four hours per non-imaging task and 16 hours per imaging task. Other published non-imaging baselines use a different, CPU-only protocol.
In GPT nanochat training optimization, AutoScientists reportedly reaches validation bits-per-byte, a prediction-loss measure where lower is better, of approximately 0.978 in 34 experiments versus 65 for Autoresearch. Each experiment trains for five minutes on one H100. Starting from the same stronger champion at 0.9777 and the same rejected-direction record, AutoScientists accepts seven updates over 93 experiments and reaches 0.9730; Autoresearch accepts none over 100 experiments. These comparisons measure experimental search efficiency, without establishing a wall-clock speedup.

Shared-state coordination of self-organizing research teams, with stagnation-triggered reorganization and outputs comprising a champion model, model card, and research insights.