Ever-Evolving Memory by Blending and Refining the Past (2403.04787v2)
Abstract: For a human-like chatbot, constructing a long-term memory is crucial. However, current LLMs often lack this capability, leading to instances of missing important user information or redundantly asking for the same information, thereby diminishing conversation quality. To effectively construct memory, it is crucial to seamlessly connect past and present information, while also possessing the ability to forget obstructive information. To address these challenges, we propose CREEM, a novel memory system for long-term conversation. Improving upon existing approaches that construct memory based solely on current sessions, CREEM blends past memories during memory formation. Additionally, we introduce a refining process to handle redundant or outdated information. Unlike traditional paradigms, we view responding and memory construction as inseparable tasks. The blending process, which creates new memories, also serves as a reasoning step for response generation by informing the connection between past and present. Through evaluation, we demonstrate that CREEM enhances both memory and response qualities in multi-session personalized dialogues.
- Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977.
- Keep me updated! memory management in long-term conversations. arXiv preprint arXiv:2210.08750.
- Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118.
- Conversation chronicles: Towards diverse temporal and relational dynamics in multi-session conversations. arXiv preprint arXiv:2310.13420.
- Persona expansion with commonsense knowledge for diverse and consistent response generation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1131–1141.
- Soda: Million-scale dialogue distillation with social commonsense contextualization. arXiv preprint arXiv:2212.10465.
- Gpteval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634.
- Like hiking? you probably enjoy nature: Persona-grounded dialog with commonsense expansions. arXiv preprint arXiv:2010.03205.
- OpenAI. 2023. Chatgpt.
- Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744.
- Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22.
- Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551.
- Beyond goldfish memory: Long-term open-domain conversation. arXiv preprint arXiv:2107.07567.
- Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
