EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms (2406.14228v3)

Abstract: The rise of powerful LLMs has spurred a new trend in building LLM-based autonomous agents for solving complex tasks, especially multi-agent systems. Despite the remarkable progress, we notice that existing works are heavily dependent on human-designed frameworks, which greatly limits the functional scope and scalability of agent systems. How to automatically extend the specialized agent to multi-agent systems to improve task-solving capability still remains a significant challenge. In this paper, we introduce EvoAgent, a generic method to automatically extend specialized agents to multi-agent systems via the evolutionary algorithm, thereby improving the effectiveness of LLM-based agents in solving tasks. Specifically, we consider the existing agent frameworks as the initial individual and then apply a series of evolutionary operators (e.g., mutation, crossover, selection, etc.) to generate multiple agents with diverse settings. Experimental results across various tasks show that EvoAgent can significantly enhance the task-solving capability of LLM-based agents, and can be generalized to any LLM-based agent framework to extend them into multi-agent systems. Resources are available at https://evo-agent.github.io/.

• The paper presents a novel method that uses evolutionary algorithms to automatically generate and optimize multi-agent systems.
• It integrates crossover and mutation with an LLM-based quality check to iteratively refine agent configurations.
• Experimental results on NLP and real-world planning tasks demonstrate enhanced scalability and performance compared to traditional frameworks.

EvoAgent: Automatic Multi-Agent Generation using Evolutionary Algorithms

The paper "EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms" introduces a generic method for automatically extending expert agents into multi-agent systems using evolutionary algorithms. This paper addresses the limitations of existing LLM-based agent frameworks that rely heavily on human-designed settings. By leveraging evolutionary algorithms, EvoAgent enhances the scalability and effectiveness of multi-agent systems, allowing them to tackle more complex tasks.

Introduction and Background

The advent of LLMs, such as GPT-4 and Gemini, has driven advancements in autonomous agents capable of solving diverse tasks. Multi-agent systems combine the expertise of various agents to emulate human-like collaboration in addressing complex scenarios. Traditionally, these systems require extensive human intervention to design agent roles, task scopes, and framework settings. EvoAgent aims to alleviate these constraints by introducing a process analogous to biological evolution.

Evolutionary algorithms simulate the natural processes of mutation, crossover, and selection to optimize and generate diverse individuals. EvoAgent applies these principles to evolve agent settings automatically, creating specialized agents that collectively enhance task-solving capabilities. This approach transforms agent generation into an iterative procedure that begins with existing frameworks and evolves them without additional human labor.

Figure 1: The adaption of EvoAgent on MetaGPT framework. With the EA, we can extend the original role in the debate scenario to different expert agents to enrich the opinions.

Methodology

EvoAgent's methodology consists of several steps:

1. Initialization: EvoAgent commences with existing agent frameworks as the initial generation of agents. The framework design serves as the initial "individual" in evolutionary terms.
2. Evolutionary Operations: EvoAgent uses crossover and mutation operators to derive new agents from the existing ones. During crossover, EvoAgent combines settings from successful agents to generate offspring agents. Mutation introduces variability by altering specific aspects of an agent's settings to discover optimal configurations.
3. Selection Process: The system includes a quality-check module employing LLMs to validate the fitness of generated agents. The module ensures diversity among agents and the effectiveness of generated solutions, mimicking natural selection.
4. Iterative Process: Agents undergo multiple evolutionary cycles, iteratively refining their settings through newly generated agents until high-quality, specialized agents are consistently produced.

   Figure 2: The adaption of EvoAgent on Camel and AutoGen frameworks.

Experimental Evaluation

Experiments conducted on various NLP and multi-modal benchmarks demonstrate EvoAgent's ability to improve LLM-based agents:

• NLP Tasks: EvoAgent was tested on Logic Grid Puzzle, Trivia Creative Writing, and Codenames Collaborative tasks, demonstrating its ability to produce superior results compared to previous frameworks like Self-Refine and SSP.
• Multi-Modal Applications: In the MMMU benchmark, EvoAgent significantly elevated performance across difficulty levels by generating domain-specific agents that collectively tackle complex scientific questions.

  Figure 3: Overall results of GPT-4V and Gemini-Pro with different methods on the MMMU validation set. We also compare the performance of GPT-4V and Gemini-Pro across three difficulty levels.

• Interactive Scenarios: In ScienceWorld, EvoAgent extended single agents into multi-agent systems, achieving better task completion rates.
• Real-World Planning: EvoAgent's effectiveness in generating high-quality travel plans in TravelPlanner illustrated its potential for real-world applications requiring complex planning under constraints.

Conclusion

EvoAgent represents a pioneering approach in automating the generation of multi-agent systems using evolutionary algorithms. By dispensing with human-designed frameworks and settings, EvoAgent enhances scalability and effectiveness, optimizing LLM-based agents for diverse real-world applications. The research establishes EvoAgent as a versatile method capable of extending any agent framework, highlighting its broad potential for future developments in AI.