Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants (2402.06229v1)
Abstract: The widespread availability of LLMs within Integrated Development Environments (IDEs) has led to their speedy adoption. Conversational interactions with LLMs enable programmers to obtain natural language explanations for various software development tasks. However, LLMs often leap to action without sufficient context, giving rise to implicit assumptions and inaccurate responses. Conversations between developers and LLMs are primarily structured as question-answer pairs, where the developer is responsible for asking the the right questions and sustaining conversations across multiple turns. In this paper, we draw inspiration from interaction patterns and conversation analysis -- to design Robin, an enhanced conversational AI-assistant for debugging. Through a within-subjects user study with 12 industry professionals, we find that equipping the LLM to -- (1) leverage the insert expansion interaction pattern, (2) facilitate turn-taking, and (3) utilize debugging workflows -- leads to lowered conversation barriers, effective fault localization, and 5x improvement in bug resolution rates.
- An Exploratory Study of Sharing Strategic Programming Knowledge. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 66, 15 pages. https://doi.org/10.1145/3491102.3502070
- Chapter Two - The Self-Organization of Human Interaction. Psychology of Learning and Motivation, Vol. 59. Academic Press, 43–95. https://doi.org/10.1016/B978-0-12-407187-2.00002-2
- Forbes. 2023. The Age of Copilots. https://www.forbes.com/sites/forbestechcouncil/2023/09/08/the-age-of-co-pilots/
- GitHub. 2023. GitHub Copilot Chat. https://docs.github.com/en/copilot/github-copilot-chat/using-github-copilot-chat-in-your-ide
- Andreas Göldi and Roman Rietsche. 2023. Insert-expansions for Tool-enabled Conversational Agents. arXiv:2307.01644 [cs.HC]
- Towards Designing Conversational Agents for Pair Programming: Accounting for Creativity Strategies and Conversational Styles. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 1–11. https://doi.org/10.1109/VL/HCC50065.2020.9127276
- A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). IEEE Computer Society, Los Alamitos, CA, USA, 605–617. https://doi.ieeecomputersociety.org/
- Amanda Liu and Michael Coblenz. 2023. Debugging Techniques in Professional Programming. (3 2023). https://doi.org/10.1184/R1/22277365.v1
- Microsoft. 2023. Visual Studio. https://visualstudio.microsoft.com/#vs-section
- Robert J. Moore and Raphael Arar. 2019. Conversational UX Design: A Practitioner’s Guide to the Natural Conversation Framework. Association for Computing Machinery, New York, NY, USA.
- OpenAI. 2022. ChatGPT. https://chat.openai.com
- Charles Rich and Richard C. Waters. 1990. The programmer’s apprentice. Association for Computing Machinery, New York, NY, USA.
- The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23). Association for Computing Machinery, New York, NY, USA, 491–514. https://doi.org/10.1145/3581641.3584037
- A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language 50, 4 Pt 1 (Dec 1974), 696–735. ERIC Number: EJ109512.
- Itamar Shatz. [n. d.]. Grice’s Maxims of Conversation: The Principles of Effective Communication. https://effectiviology.com/principles-of-effective-communication/
- Jack Sidnell. 2016. Conversation Analysis. https://doi.org/10.1093/acrefore/9780199384655.013.40
- Tabnine. 2023. Tabnine AI Assistant. https://marketplace.eclipse.org/content/tabnine-ai-assistant-chat-software-developers
- Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 332, 7 pages. https://doi.org/10.1145/3491101.3519665
- Wikipedia contributors. 2023. Conversation analysis — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Conversation_analysis&oldid=1185118449 [Online; accessed 26-January-2024].
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.