If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents

Overview of Integrating Code into LLM Training

Training data plays a pivotal role in developing LLMs. The inclusion of formal language or code in the training materials greatly broadens the scope of what these models can do. Code, by nature, is structured and logically consistent, qualities that when learned can significantly enhance the capabilities of an LLM. The paper explores how code not only aids in improving LLMs' core programming abilities but also in boosting their problem-solving skills and connecting them to a variety of functional ends.

Enhancements in LLM Abilities

Programming Proficiency

One of the most noticeable improvements with code integration is in programming competence. LLMs excel in both writing and assessing code, surpassing previous benchmarks. They can manage multiple programming languages and have been applied to an array of tasks from software development to data analysis. This proficiency has also led to growth in LLMs' ability to evaluate code for errors and performance, indicating a comprehensive understanding of code beyond mere generation.

Complex Reasoning Capabilities

Another critical enhancement is in sophisticated reasoning skills. LLMs show improved performance in tasks that require methodical, step-by-step processing—key elements of programming logic. This has been evident in models showing improved 'chain of thought' capabilities, which allow them to break down complex tasks into smaller, more manageable steps, resulting in better decision-making processes.

Structured Knowledge Capture

LLMs trained with code also show advanced abilities to understand and generate content that involves structured information. The models can handle diverse forms of structured data, such as HTML or tabular representations. This ability is particularly important for interpreting multimedia information and enhances the LLMs' interaction with graph-structured data.

LLMs as Intelligent Agents

Decision Making and Execution

Through code training, LLMs have developed into more sophisticated IAs. They can perceive environments, plan, and execute actions more effectively, thanks to their improved ability to process structured information. The models can also use code to interact with multiple tools and APIs, allowing them to perform a wider range of tasks, including those that involve complex interplays between different modalities and physical-world interactions.

Self-improvement through Feedback

An important aspect of intelligent behavior is the ability to learn from feedback and adapt. Code-driven environments allow LLMs to collect highly reliable and precise feedback—essential for refining their performance. This is a self-reinforcing loop as better performance opens up avenues for receiving more feedback, which in turn leads to further improvements.

Challenges and Future Directions

Despite the progress, there are still some key questions and challenges. For one, it is not clear exactly how much of the enhancement in reasoning abilities comes directly from code pre-training. The nature of reasoning beyond what code provides is another area that invites further exploration. Lastly, perfecting methods to fully leverage feedback for performance improvement in multi-turn interactions is still ongoing, with potential avenues like reinforcement learning yet to be thoroughly explored.

Conclusion

The survey presents a robust argument for the value of including code in LLM training. It underscores the significant impact this inclusion has on the models' capabilities and their application as IAs. Nevertheless, it also offers a perspective on the challenges that remain, providing a catalyst for further research in this domain.