Monthly AI Papers: June 2025

Read this article in Korean

Trends

This retrospective covers papers first released in June 2025. The Sekai summary uses the archived v3 revision dated November 9, 2025; its results are not a snapshot of what was available in June.

June 2025’s theme across these three selected papers is improving how models organize work and information: SwarmAgentic searches over teams of language-model agents, each with assigned responsibilities and instructions; Sekai pairs first-person video with environmental descriptions and camera trajectories, the camera’s positions and orientations over time; and Autoregressive U-Net learns text representations at several scales before predicting successive bytes. Together, they examine system structure, training supervision, and representation granularity as distinct ways to improve task performance. 1, 2, 3

The strongest results depend on the comparison conditions. SwarmAgentic improves complete travel-plan success, but its search budget is not matched to baseline token, call, or monetary costs. Sekai improves camera following through fine-tuning, additional training of an existing model, while ordinary video-generation results compare training checkpoints without an unfine-tuned baseline. Autoregressive U-Net broadly tracks a strong fixed-vocabulary baseline in compute-controlled experiments, although some equal-data comparisons use larger models or more computation. These findings support specific engineering choices; they leave search efficiency, sustained interactive exploration, and gains beyond measured training scales unresolved.

Top Papers in June 2025

1. SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence

• https://arxiv.org/abs/2506.15672v1
• SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence | Summary
• SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence jointly searches agent instructions and workflows, the execution steps and information flow connecting agents. It adapts Particle Swarm Optimization, a population search guided by each candidate’s best previous solution and the population’s best solution, to structured language descriptions. A large language model diagnoses failures and produces textual modifications that revise agents and their coordination.
• With GPT-4o execution, TravelPlanner, a benchmark for travel planning under user constraints, reaches a 32.2% final pass rate versus 8.9% for Automated Design of Agentic Systems (ADAS), a baseline that searches agent code. This is a 23.3-percentage-point increase; the paper reports a 261.8% relative improvement. SwarmAgentic uses 5 candidate systems and 10 optimization iterations, while ADAS receives at most 30 iterations. These settings do not establish equal search costs.
• The clearest component evidence favors optimizing behavior and coordination together. On 20 Creative Writing instances, removing workflow reconfiguration lowers the full system’s score from 8.8 to 6.7; removing agent-level adaptation lowers it to 7.3. Cross-model transfer is tested on Creative Writing. Missing uncertainty estimates, inconsistent scores for a duplicated search setting, and conflicting descriptions of one Gemini condition limit stronger efficiency and robustness claims. The disclosed prompts also impose modular workflow conventions.

SwarmAgentic initialization, language-based particle updates, and best-system selection.

TravelPlanner search trajectory with annotated agent and verification changes.

2. Sekai: A Video Dataset towards World Exploration

• https://arxiv.org/abs/2506.15675v3
• Sekai: A Video Dataset towards World Exploration | Summary
• Sekai: A Video Dataset towards World Exploration supplies walking and drone footage with location, environmental labels, captions, and camera trajectories. Figure 2 reports 5,064 hours of Sekai-Real across 101 countries and regions and more than 750 cities, plus 36 hours of Sekai-Game from Lushfoil Photography Sim. Real-world camera trajectories are estimated from video; game camera positions and orientations are captured directly. Quality and diversity sampling produce the 400-hour Sekai-Real-HQ training subset.
• Fine-tuning Wan2.1-Fun-V1.1-1.3B-Control-Camera, a camera-conditioned video generator, on Sekai-Real drone footage reduces translation error from 28.32 to 17.19 and rotation error from 27.2 to 19.89. Sekai-Game fine-tuning reduces the corresponding errors from 7.64 to 4.22 and 8.36 to 6.22. Lower errors indicate closer agreement between requested and estimated camera motion; physical units are unspecified. MegaSaM, the camera-motion estimation system, supplies both real-world training annotations and generated-video evaluation trajectories, limiting independent claims about geometric accuracy.
• Annotation checks report location issues in less than 5% of 500 sampled videos and category agreement above 90% on 500 examples, excluding indoor scenes from the latter analysis. Generation evaluations use 50 selected test clips. Ordinary video-generation experiments use 49-frame clips at 16 frames per second and compare fine-tuning checkpoints without an unfine-tuned baseline. Persistent navigation and revisiting locations are not evaluated, and inconsistent duration statistics leave aspects of dataset accounting unclear.

Real-world and game video sources with an example of camera, category, and caption annotations.

Geographic coverage, sample sequences, and collection statistics for Sekai-Real and Sekai-Game.

3. From Bytes to Ideas: Language Modeling with Autoregressive U-Nets

• https://arxiv.org/abs/2506.14761v1
• From Bytes to Ideas: Language Modeling with Autoregressive U-Nets | Summary
• From Bytes to Ideas: Language Modeling with Autoregressive U-Nets introduces Autoregressive U-Net (AU-Net), which contracts byte sequences into coarser contextual representations and expands them for next-byte prediction. Skip connections carry fine-grained representations directly into later layers. The hierarchy can process byte, word, two-word, and four-word scales, but its boundaries follow predefined text-pattern rules. The learned representations therefore retain manually specified segmentation, with space-based rules that suit Latin scripts.
• At 370B LLaMa 3 token-equivalent training data, AU-Net 3 scores 72.9 ± 0.9 on Hellaswag, a multiple-choice benchmark, versus 70.2 ± 0.9 for Byte-Pair Encoding (BPE), whose vocabulary is built by repeatedly merging frequent byte sequences. Table 2 rounds both training costs to 4e21 floating-point operations (FLOPs), a measure of computation; Table 10 gives 4.0e21 for AU-Net 3 and 3.6e21 for BPE. Compute-scaling experiments broadly track BPE on several tasks, while knowledge and mathematical reasoning results remain weaker at the upper end of the tested range.
• Practical gains vary by task and hierarchy depth. At 60B token-equivalent data, AU-Net 3 uses 2.5B parameters versus BPE’s 1.8B and trains at 180k versus 210k bytes per second per device on H100 80GB hardware. On CUTE, a word and character manipulation benchmark, AU-Net 2 raises character-level exact-match accuracy, the fraction of answers matching the target exactly, from 39.3 to 42.7 while word-level accuracy falls from 36.9 to 33.1. Languages without clear space-based boundaries remain a limitation, and measured generation-speed results are absent.

Three-stage AU-Net with byte, word, and two-word representations, contracting and expanding paths, and skip connections.

Boundary-based pooling and position-specific expansion into the following segment.

Sources

[1] SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence

[2] Sekai: A Video Dataset towards World Exploration

[3] From Bytes to Ideas: Language Modeling with Autoregressive U-Nets