Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence (2407.07061v2)

Abstract: The rapid advancement of LLMs has paved the way for the development of highly capable autonomous agents. However, existing multi-agent frameworks often struggle with integrating diverse capable third-party agents due to reliance on agents defined within their own ecosystems. They also face challenges in simulating distributed environments, as most frameworks are limited to single-device setups. Furthermore, these frameworks often rely on hard-coded communication pipelines, limiting their adaptability to dynamic task requirements. Inspired by the concept of the Internet, we propose the Internet of Agents (IoA), a novel framework that addresses these limitations by providing a flexible and scalable platform for LLM-based multi-agent collaboration. IoA introduces an agent integration protocol, an instant-messaging-like architecture design, and dynamic mechanisms for agent teaming and conversation flow control. Through extensive experiments on general assistant tasks, embodied AI tasks, and retrieval-augmented generation benchmarks, we demonstrate that IoA consistently outperforms state-of-the-art baselines, showcasing its ability to facilitate effective collaboration among heterogeneous agents. IoA represents a step towards linking diverse agents in an Internet-like environment, where agents can seamlessly collaborate to achieve greater intelligence and capabilities. Our codebase has been released at \url{https://github.com/OpenBMB/IoA}.

• The paper introduces the Internet of Agents (IoA), a robust framework for integrating diverse agents through a unified communication protocol.
• The framework employs a layered architecture and autonomous conversation flow, using techniques like finite-state machines and LLMs to coordinate tasks dynamically.
• Empirical evaluations on benchmarks like GAIA demonstrate IoA's superior collaborative performance and scalability in real-world applications.

Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence

The paper "Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence" details a framework named Internet of Agents (IoA), designed to create a flexible and scalable platform for collaboration among diverse autonomous agents. This essay explores the architectural design, implementation strategies, and empirical evaluations presented for IoA, elucidating its advantages and implications for the development of intelligent systems.

Conceptual Framework and Architectural Design

Core Framework Elements

The IoA concept draws inspiration from the Internet as a medium for interconnected nodes to communicate seamlessly. The framework introduces several innovative elements to overcome the limitations of existing multi-agent systems, such as ecosystem isolation, lack of support for distributed environments, and rigid communication structures.

1. Agent Integration Protocol: Enables the inclusion of a wide array of third-party agents by utilizing a common protocol for defining agent capabilities.
2. Instant-Messaging-Like Architecture: Facilitates dynamic discovery and teaming of agents, resembling the functionality of messaging applications to enable real-time updates and communication.
3. Conversation Flow Control: Employs a finite-state machine inspired by Speech Act Theory to guide agents through structured dialogues for team collaboration and task execution.

   Figure 1: The illustration on the conceptual layered architecture on the design of IoA.

Layered Architecture

IoA's design is compartmentalized into three layers:

• Interaction Layer: Manages dynamic interactions among agents, overseeing team formation and communication.
• Data Layer: Stores information pertinent to ongoing dialogues, agent capabilities, and task status.
• Foundation Layer: Provides infrastructure for integration, data management, and security between agents and their respective environments.

Implementation and Mechanisms

Key Mechanisms in IoA

The deployment of IoA involves several mechanisms that ensure efficient operation and scalability.

• Agent Registration and Discovery: Agents register their capabilities upon joining the framework. This enables the querying and discovery of agents based on task-specific requirements.
• Nested Team Formation: Allows the dynamic creation of teams within teams for intricate tasks, using a coordinated sub-task assignment strategy.
• Autonomous Conversation Flow: Utilizes a predefined set of conversation states managed through LLMs to transition between discussion phases and task assignments dynamically.

  Figure 2: An example walkthrough of the major components of IoA.

Empirical Evaluation and Performance

Evaluation on Diverse Benchmarks

IoA's performance was rigorously tested across different domains:

• GAIA Benchmark: IoA demonstrated superior task-solving capabilities, particularly in tool integration, by outperforming other state-of-the-art frameworks.
• Open-Ended Instruction Benchmark: IoA's ability to facilitate agent collaboration resulted in high success rates, showcasing its capability to handle diverse and unstructured tasks.

  Figure 3: Comparison of win rates on the open-ended instruction benchmark between IoA, AutoGPT, and Open Interpreter.

Application in Real-World Tasks

IoA has showcased its versatility in managing agents with heterogeneous architectures, observation and action spaces, and retrieval-augmented generation tasks. Its ability to handle complexities inherent in these tasks validates IoA as a powerful framework for orchestrating collaborative problem-solving efforts.

Conclusion

IoA represents a significant step towards scalable, internet-like interconnections of autonomous agents, enabling seamless integration of varied functionalities. The architectural design and empirical evaluations underscore IoA's potential to revolutionize multi-agent systems by providing a flexible robust foundation for collaborative intelligence. As smaller and more capable models emerge, the deployment of IoA can contribute to the development of sophisticated multi-agent ecosystems capable of real-time collaboration and dynamic task handling.