Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization (2404.02183v1)
Abstract: Recent advancements in automatic code generation using LLM agent have brought us closer to the future of automated software development. However, existing single-agent approaches face limitations in generating and improving large-scale, complex codebases due to constraints in context length. To tackle this challenge, we propose Self-Organized multi-Agent framework (SoA), a novel multi-agent framework that enables the scalable and efficient generation and optimization of large-scale code. In SoA, self-organized agents operate independently to generate and modify code components while seamlessly collaborating to construct the overall codebase. A key feature of our framework is the automatic multiplication of agents based on problem complexity, allowing for dynamic scalability. This enables the overall code volume to be increased indefinitely according to the number of agents, while the amount of code managed by each agent remains constant. We evaluate SoA on the HumanEval benchmark and demonstrate that, compared to a single-agent system, each agent in SoA handles significantly less code, yet the overall generated code is substantially greater. Moreover, SoA surpasses the powerful single-agent baseline by 5% in terms of Pass@1 accuracy.
- Gemini: A family of highly capable multimodal models. CoRR, abs/2312.11805.
- W Ross Ashby. 1947. Principles of the self-organizing dynamic system. The Journal of general psychology, 37(2):125–128.
- Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
- Evaluating large language models trained on code. CoRR, abs/2107.03374.
- Self-collaboration code generation via chatgpt. CoRR, abs/2304.07590.
- Incoder: A generative model for code infilling and synthesis. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
- Metagpt: Meta programming for multi-agent collaborative framework. CoRR, abs/2308.00352.
- Agentcoder: Multi-agent-based code generation with iterative testing and optimisation. CoRR, abs/2312.13010.
- Same task, more tokens: the impact of input length on the reasoning performance of large language models. CoRR, abs/2402.14848.
- Loogle: Can long-context language models understand long contexts? CoRR, abs/2311.04939.
- Starcoder: may the source be with you! CoRR, abs/2305.06161.
- Competition-level code generation with alphacode. CoRR, abs/2203.07814.
- Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
- Octopack: Instruction tuning code large language models. CoRR, abs/2308.07124.
- Skeleton-of-thought: Large language models can do parallel decoding. CoRR, abs/2307.15337.
- OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
- Communicative agents for software development. CoRR, abs/2307.07924.
- Toolformer: Language models can teach themselves to use tools. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
- Zeroscrolls: A zero-shot benchmark for long text understanding. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 7977–7989. Association for Computational Linguistics.
- Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
- Llama 2: Open foundation and fine-tuned chat models. CoRR, abs/2307.09288.
- Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
- Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
- React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
- Self-edit: Fault-aware code editor for code generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 769–787. Association for Computational Linguistics.
- LDB: A large language model debugger via verifying runtime execution step-by-step. CoRR, abs/2402.16906.
- Language agent tree search unifies reasoning acting and planning in language models. CoRR, abs/2310.04406.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
