SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence | Summary

This article explains the key points of SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence.

• 2025-06-18 (arXiv)
• Zhang, Yao, Lin, Chenyang, Tang, Shijie, Chen, Haokun, Zhou, Shijie, Ma, Yunpu, Tresp, Volker.
• LMU Munich, Technical University of Munich, Munich Center for Machine Learning
• Paper

Read this article in Korean

Summary

• Designing systems of collaborating language-model agents, each assigned a role and instructions, usually requires manually chosen roles and workflows that can limit adaptation to new tasks. With GPT-4o executing the generated systems, SwarmAgentic achieves a 32.2% final pass rate on TravelPlanner, a benchmark for travel planning under user constraints, compared with 8.9% for Automated Design of Agentic Systems (ADAS), a baseline that searches over agent code; the paper reports a 261.8% relative improvement.
• The method adapts Particle Swarm Optimization (PSO), a population search guided by each candidate’s best previous solution and the population’s best solution, to structured language descriptions of agents and workflows. A large language model (LLM) diagnoses execution failures, proposes changes using failed modifications and successful designs, and applies those changes to agent policies, the instructions governing their behavior, and coordination structures.
• With 5 particles, each representing a candidate system, and 10 optimization iterations, SwarmAgentic leads the reported final-score comparisons across six tasks spanning planning, creative writing, and mathematical reasoning. Component removal experiments support joint optimization of agents and workflows, but the comparisons do not establish equal-cost search efficiency, and the framework retains language-model factual errors and text-only interaction limits.

1 Introduction

The introduction defines three requirements for autonomous agentic system generation: From-Scratch Agent Generation creates task-specific agents without predefined functional modules; Self-Optimizing Agent Functionality revises agent behavior from feedback; and Self-Optimizing Agent Collaboration revises coordination and execution structure. Table 1 classifies SwarmAgentic as satisfying all three, whereas the compared frameworks retain templates, seed agents, or fixed interactions.

• The proposed contribution is a gradient-free search, requiring no derivatives of the task score, over complete systems using structured language representations, failure diagnosis, and population guidance.
• The abstract displays github.com/SwarmAgentic; its PDF hyperlink points to https://yaoz720.github.io/SwarmAgentic/.

The authors classify SwarmAgentic as the only listed framework meeting all three autonomy properties. This is a definition-based capability assessment supported by Appendix A rather than an experimental performance comparison.

2 Related Work

The related work separates the construction of agent teams from the optimization of their behavior and coordination. This distinction motivates searching jointly over agent functionality and collaborative structure.

2.1 Agentic System Generation

Prior generation methods include SPP, which uses structured persona prompting to simulate collaboration among multiple personas; AgentVerse, which assembles expert teams; AutoAgents, which refines agents through discussion; and EvoAgent, which evolves agent configurations. The paper argues that predefined collaboration patterns constrain these methods and positions SwarmAgentic as generating task-specific roles and collaboration strategies together.

2.2 Agentic System Optimization

The optimization literature includes agent-policy refinement from execution records, inter-agent coordination search, workflow optimization using reinforcement learning, which improves decisions through reward feedback, and searches over combinations of predefined modules. Other methods use textual feedback as a direction for revising prompts or pipelines. The paper’s proposed distinction is to treat agent functionality and collaboration as interdependent search variables, both revised through language-driven population search.

3 Preliminary

The preliminary section defines an agentic system as a set of agents coupled to a collaborative execution structure. It then introduces the numerical PSO formulation that motivates the language-based update procedure.

3.1 Agentic System Optimization

Each candidate system contains agents described by an identifier, responsibility, and execution policy, together with workflow steps assigning agents to tasks. A task-specific fitness function, the scoring rule used to evaluate candidates, scores the complete system. Optimization can change both the agent set and the collaborative structure.

3.2 Particle Swarm Optimization

In numerical PSO, a particle’s position represents a candidate solution, while its velocity specifies an update combining the previous velocity with attraction toward personal and global best positions. Random factors vary the influence of those best positions. SwarmAgentic reinterprets position as a structured system description and velocity as a textual modification plan, allowing language-model transformations to update discrete configurations.

4 SwarmAgentic

SwarmAgentic initializes a population, then repeatedly evaluates, diagnoses, and modifies its candidate systems while retaining the best-performing configuration. Figure 1 shows how failure-driven adjustments, personal best guidance, and global best guidance produce an update plan that can add, delete, modify, or reorder system components.

Each particle represents a complete agent team and workflow. Three guidance sources produce textual modifications that change roles and execution structure, while the search retains the best evaluated candidate.

4.1 Particle Initialization

Initialization samples candidate systems at different language-model temperatures, sampling settings that control output variability, to encourage structural diversity. Lower temperatures favor stable configurations, while higher temperatures encourage exploration; initial modification plans use the evaluated systems and diagnosed flaws. Each particle starts with its initial system as its personal best, and the global best is selected after the initial population has been evaluated.

4.2 Flaw Identification

An LLM analyzes objective-function errors alongside the current system to identify agent and workflow flaws. Agent flaws include missing or redundant roles and ambiguous policies; workflow flaws include missing or redundant steps, insufficient contextual inputs, and unsuitable outputs that propagate errors downstream.

4.3 Failure-Aware Velocity Update

The Failure-Aware Velocity Update combines three textual sources of improvement. Failure-Driven Adjustments compare previous and current flaws with the previous modification plan, Personal Best Guidance extracts relevant improvements from the particle’s best configuration, and Global Best Guidance extracts improvements from the best system in the population.

• Separate LLM transformations produce these three recommendations, which another LLM transformation integrates into the next update plan.
• The numerical-looking differences between configurations represent semantic comparisons rather than subtraction of numerical system encodings.

4.4 Position Update

The position update applies the textual modification plan to the current system. Agent-Level Adaptation revises individual agents and can introduce or remove roles; Collaborative Structures Reconfiguration revises execution order, dependencies, inputs, outputs, and the presence of workflow steps. Although the main text describes changes to roles, responsibilities, and policies, the disclosed position-update prompt restricts modification of an existing role to its policy.

5 Experiments

The experiments evaluate whether automatically generated systems improve task performance across planning, generation, and reasoning. They compare SwarmAgentic with direct prompting, iterative answer refinement, and automated agent design baselines.

5.1 Experimental Setup

The six tasks are TravelPlanner (TP); Trip Planning, Meeting Planning, and Calendar Scheduling from Natural Plan (NP), a natural-language planning benchmark; Creative Writing (CW), which generates coherent paragraphs from unordered key points; and MGSM, a mathematical reasoning benchmark. Baselines are Direct, which answers with a fixed prompt; Chain-of-Thought (CoT), which prompts step-by-step reasoning; Self-Refine, which revises answers using self-feedback; SPP; EvoAgent; and ADAS.

• GPT-4o-mini-0718 is the optimizer, the model that proposes system designs and revisions. Executor models, which run the discovered systems, include GPT-3.5-turbo-0125, GPT-4o-0806, Claude-3.5-sonnet-0620, DeepSeek-V3, and Gemini-1.5-Pro.
• SwarmAgentic uses 5 particles and 10 optimization iterations; ADAS receives a maximum of 30 iterations. These settings do not establish equal token, call, or monetary budgets.

5.2 Results

Tables 2 and 3 show the highest reported final task scores for SwarmAgentic with both GPT-3.5 and GPT-4o execution. On TravelPlanner, final pass rates are 3.3% / 32.2%, compared with ADAS at 1.1% / 8.9%; some intermediate constraint metrics tie, so the advantage is not strict in every table cell.

• For GPT-4o, SwarmAgentic obtains 13.1 on Trip Planning, 56.0 on Meeting Planning, 82.0 on Calendar Scheduling, 8.5 on Creative Writing, and 88.4 on MGSM. Planning and mathematics use exact-match evaluation, which checks agreement with reference answers; writing uses an LLM-assigned score from 1–10.
• The corresponding GPT-3.5 scores are 13.1, 23.0, 28.0, 8.2, and 65.6. Gains extend to the structured mathematics task, although their magnitude varies substantially across tasks and executors.

With GPT-4o execution, the final pass rate increases from ADAS’s 8.9% to 32.2%, while delivery remains 100.0% for both. The improvement concerns satisfying the complete planning requirements; some GPT-3.5 intermediate constraint metrics tie.

SwarmAgentic leads every reported task column under both executors. Its GPT-4o Calendar Scheduling score is 82.0 versus the strongest baseline’s 66.0, while the MGSM margin is smaller, at 88.4 versus 87.3.

6 Analysis

The analysis examines transfer of discovered systems across executor models, ablations that remove individual optimization mechanisms, and a TravelPlanner search trajectory. These experiments assess portability, component contribution, and the concrete changes made during search.

6.1 Cross-Model Transferability Analysis

On Creative Writing, the system discovered using GPT-4o-mini achieves scores of 8.5 with GPT-4o, 8.3 with Claude-3.5-sonnet, 9.0 with DeepSeek-V3, and 7.5 with Gemini-1.5, exceeding the listed baselines in each column. This supports transfer across the tested models on this task.

• The Gemini-1.5* score is 7.8. Table 4 specifies optimization on Gemini-1.5-flash and testing on Gemini-1.5-Pro, whereas the surrounding prose describes direct optimization and evaluation on Gemini-1.5-Pro; the exact condition is inconsistent within the paper.

The discovered Creative Writing workflow exceeds the listed baselines across four executor models. The starred Gemini condition increases SwarmAgentic’s score from 7.5 to 7.8; the caption specifies Gemini-1.5-flash optimization and Gemini-1.5-Pro evaluation, differing from the accompanying prose.

6.2 Ablation Study

The ablation study uses 20 Creative Writing instances. The full 5-particle, 10-iteration system scores 8.8; removing Collaborative Structures Reconfiguration lowers the score to 6.7, removing Agent-Level Adaptation to 7.3, and removing Failure-Driven Adjustments to 8.4.

• Workflow reconfiguration has the largest observed component removal penalty, while removing failure-driven adjustments produces a smaller 0.4-point difference in this experiment.
• At 3 particles, increasing iterations from 1 to 5 to 10 gives 5.9, 6.4, and 7.0. The separate particle-count comparison gives 6.3, 6.7, and 6.9 for 1, 3, and 5 particles at 5 iterations.
• Table 5 reports the same 3-particle, 5-iteration setting as both 6.4 and 6.7 in different blocks without explaining the difference, limiting precise interpretation of the scaling comparison.

Removing workflow reconfiguration lowers the full system’s score from 8.8 to 6.7, the largest listed component penalty. Each search-size block improves with additional iterations or particles, but the duplicated 3-particle, 5-iteration setting has inconsistent scores across blocks.

6.3 Case Study: Search Trajectory on TP

The illustrated TravelPlanner trajectory reaches an 11% success rate after introducing a Quality Assurance Specialist, a role that reviews plan compliance, and accommodation verification; 22% after further checking budget and preferences; and 33% after explicitly enforcing budget compliance. Figure 2 associates these increases with concrete role and workflow changes, but presents an illustrative trajectory without variation across repeated searches.

The stepwise trajectory associates success-rate increases to 11%, 22%, and 33% with verification responsibilities and stronger budget enforcement. It illustrates the changes accompanying improvement without quantifying variation across independent searches.

7 Conclusion

The conclusion presents SwarmAgentic as a language-driven PSO framework that dynamically refines agent sets and collaboration structures. The experiments support improved task performance under the reported benchmark conditions and demonstrate discovery of task-specific systems without manually supplied task-specific seed agents.

Limitations

The authors identify three limitations: omitting domain-specific templates may slow convergence in structured environments; plausible but incorrect language-model outputs can propagate through optimization; and the text-only framework lacks multimodal perception and physical action. They suggest structural priors, predefined assumptions that guide search, external knowledge, and multimodal or embodied extensions as future directions without evaluating those extensions.

Appendix

• A Agentic System Generation Autonomy, including A.1 Defining Autonomy: Three Core Properties and A.2 Evaluation of Existing Agentic Frameworks, supplies the definitions and framework-by-framework reasoning behind Table 1. From-Scratch Agent Generation permits minimal task-agnostic scaffolding such as input/output wrappers and abstract interfaces, while requiring task-specific reasoning and coordination to be newly generated. The assessment assigns no properties to SPP, functionality optimization alone to EvoAgent and AgentSquare, and functionality plus collaboration optimization to AutoAgents, AFlow, Agent Symbolic Learning, and ADAS; none receives the from-scratch designation.
• B Comparison with MODEL SWARMS: From Model Fusion to Agentic System Generation distinguishes searching over model weights or token probabilities from searching over executable agent systems. MODEL SWARMS adapts an existing expert pool into a single model, whereas SwarmAgentic rewrites roles, policies, and collaboration flows. The shared PSO inspiration therefore applies to different search spaces and outputs.
• C Experimental Setup, through C.1 Dataset Statistics and Evaluation and C.2 Baselines and Configurations, specifies 128 training and 800 test questions for MGSM, 5 training and 95 evaluation tasks for Creative Writing, and 9 training queries with 180 evaluation queries for TravelPlanner. Natural Plan uses 8 Trip Planning training examples and 10 training examples each for Meeting Planning and Calendar Scheduling, with evaluation on a disjoint, difficulty-matched 10% of the full dataset. Trip Planning covers 3–10 cities, Meeting Planning covers 1–10 friends, and Calendar Scheduling covers one-day schedules with 3–7 meetings and two-day schedules 1–5 days apart.
• C.1 evaluates TravelPlanner using delivery, constraint, and final pass rates; Natural Plan uses exact match after extracting plan details, and MGSM compares integer answers directly with references. Creative Writing uses zero-shot LLM scoring, meaning scoring without task demonstrations, from 1–10 and averages five samples per output. C.2 follows official baseline implementations with task-specific prompt wording adjustments; ADAS retains its 7 seed agents and Meta Agent Search procedure, which generates and revises candidate system code.
• D SwarmAgentic Implementation includes D.1 Basic Structure of Agentic System, D.2 Pseudo Code for SwarmAgentic, and D.3 Prompt Repository. Role and Team classes represent agents and their coordination, generated Python functions execute workflows through team.call, Algorithm 1 specifies population evaluation and updates, and the prompt repository covers initialization, diagnosis, failed-adjustment identification, best-system guidance, and application of changes. The prompts require modular information flow, one responsible role per workflow step, and final integration; they also preserve an existing role’s name and responsibility when modifying its policy, imposing general structural conventions on the search.
• E Case Study and E.1 Illustrative Optimization Process illustrate personal-best accommodation verification, global-best introduction of a Quality Assurance Specialist, and failure-driven strengthening of budget checks. F Best-Discovered Agentic System provides workflows in F.1 MGSM, with analysis, calculation planning, review, execution, and formatting; F.2 Creative Writing, with narrative construction and review stages; F.3 Meeting Scheduling, with travel-time verification and schedule validation; and F.4 TravelPlanner, with transportation, accommodation, dining, attractions, quality assurance, and integration. F.5 Comparison with ADAS-Discovered Agentic System attributes accommodation failures in the ADAS example to missing dedicated responsibility and weak integration, and argues that its growing workflow archive complicates search; these are the authors’ diagnoses rather than controlled tests of those explanations.

Brief Thoughts

The clearest component evidence supports optimizing agent behavior and information flow together: removing workflow reconfiguration produces the largest ablation drop, and the TravelPlanner examples show specific verification dependencies. Figures 3 and 4 make the discovered architectures inspectable, with SwarmAgentic assigning accommodation and quality-assurance responsibilities and the ADAS example combining specialist proposals through negotiation and a meta-agent. The headline 261.8% relative gain corresponds to a 23.3-percentage-point increase, from 8.9% to 32.2%; most TravelPlanner queries remain unsolved. The reported comparisons lack uncertainty estimates and matched search-cost measurements. Transfer is tested on Creative Writing, the ablation uses 20 instances, and the duplicate Table 5 setting and inconsistent Gemini condition limit stronger claims about efficiency or robustness. The disclosed prompts make the search procedure easier to inspect, while their modular and sequential conventions qualify the paper’s descriptions of structurally unconstrained generation.

The ADAS example begins with itinerary, budget, dining, and activity proposals, passes them through collaborative negotiation, and uses a meta-agent for final integration. The absence of a dedicated accommodation role is the architectural issue discussed in Appendix F.5; the diagram alone does not establish its causal effect on performance.