Human-Centered Planning (2311.04403v1)

Abstract: LLMs have recently made impressive inroads on tasks whose output is structured, such as coding, robotic planning and querying databases. The vision of creating AI-powered personal assistants also involves creating structured outputs, such as a plan for one's day, or for an overseas trip. Here, since the plan is executed by a human, the output doesn't have to satisfy strict syntactic constraints. A useful assistant should also be able to incorporate vague constraints specified by the user in natural language. This makes LLMs an attractive option for planning. We consider the problem of planning one's day. We develop an LLM-based planner (LLMPlan) extended with the ability to self-reflect on its output and a symbolic planner (SymPlan) with the ability to translate text constraints into a symbolic representation. Despite no formal specification of constraints, we find that LLMPlan performs explicit constraint satisfaction akin to the traditional symbolic planners on average (2% performance difference), while retaining the reasoning of implicit requirements. Consequently, LLM-based planners outperform their symbolic counterparts in user satisfaction (70.5% vs. 40.4%) during interactive evaluation with 40 users.

• The paper demonstrates that an LLM-based planner with self-reflection improves handling vague constraints compared to traditional symbolic planners.
• It leverages natural language interpretation to convert user inputs into structured daily schedules with iterative adjustments.
• Experimental evaluations reveal high user satisfaction and performance near parity with symbolic methods, highlighting practical applications in personal assistance.

Human-Centered Planning

The paper "Human-Centered Planning" introduces an innovative approach to personal day planning utilizing LLMs. This research extends beyond conventional planning algorithms by integrating LLMs with self-reflective capabilities, thus enhancing the system's ability to handle vague and implicit user constraints. The paper positions its contributions as a comparative analysis between LLM-based planners and traditional symbolic planning methods.

Introduction to LLM-Based Day Planning

The research is motivated by the increasing success of LLMs in generating structured outputs across various domains, such as programming, robotic planning, and data querying. Unlike traditional methods that require explicit, syntactic inputs, LLMs allow for the interpretation of human language that often includes ambiguities. The paper focuses on the task of day planning, which involves arranging activities with varying constraints on time and order.

The novel planner, denoted as $\text{LLM}_{\text{planner}}$, leverages LLMs to generate a daily agenda by translating user-input natural language constraints into coherent schedules. Notably, the system includes a self-reflection mechanism allowing it to adjust plans iteratively by identifying and correcting constraint violations upon user interaction.

Figure 1: The LLM-based planner processing user-defined events and adjusting iteratively based on feedback, demonstrating near parity with traditional symbolic planners.

LLM-Based vs. Symbolic Planning

The paper compares two planning paradigms: the LLM-based planner and a symbolic planner. The LLM planner gains an advantage in handling non-explicit constraints due to its inherent language understanding derived from extensive pre-training. However, symbolic planners offer deterministic constraint satisfaction with optimization goals such as minimizing completion time. The researchers extend the symbolic planner by incorporating a capability for translating text-based constraints into symbolic representations, attempting to emulate the commonsense reasoning seen in LLMs.

Figure 2: An example simple temporal network (STN) showing events with constraints and scheduling nodes.

Experimental Evaluation

The robustness of the proposed planners was assessed using both synthetic and real-user datasets. In synthetic tests, LLM-based planners approached the performance of symbolic planners in constraint satisfaction but excelled in the adaptability to unspecific constraints. User studies demonstrated higher satisfaction for the LLM planner in terms of flexibility and user-interactive refinement capabilities.

Figure 3: Comparison of planner outputs highlighting differences in handling explicit and commonsense constraints.

Implications and Future Directions

The paper highlights significant implications for the future application of AI planning systems in personal assistance tools. By leveraging LLMs, planners can offer more personalized and adaptable solutions, accommodating the nuances of human preferences and needs. While the results are promising, the paper suggests further investigation in enhancing LLMs' understanding of complex constraints and embedding deeper contextual awareness for more sophisticated application scenarios.

Conclusion

In conclusion, the LLM-based planner with its self-reflective mechanism marks a significant step forward in AI-driven personal day planning. This hybrid approach demonstrates a compelling balance of accuracy in constraint handling and flexibility inherent to natural language processing. Future research endeavors can focus on refining these systems to manage finer complexities inherent in human-centered planning, bringing us closer to truly intelligent personal assistants capable of nuanced day-to-day management.