A Survey of the Evolution of Language Model-Based Dialogue Systems

Dialogue systems, including task-orienteddialoguesystem (TOD) and open-domaindialoguesystem (ODD), have undergone significant transformations, with languagemodels (LM) playing a central role. This survey explore the historical trajectory of dialogue systems, elucidating their intricate relationship with advancements in language models by categorizing this evolution into four distinct stages, each marked by pivotal LM breakthroughs: 1) EarlyStage: characterized by statistical LMs, resulting in rule-based or machine-learning-driven dialoguesystems; 2) Independent development of TOD and ODD based on neurallanguagemodels (NLM; e.g., LSTM and GRU), since NLMs lack intrinsic knowledge in their parameters; 3) fusion between different types of dialogue systems with the advert of pre-trainedlanguagemodels (PLMs), starting from the fusion between foursub-taskswithinTOD, and then TODwithODD; and 4) current LLM-baseddialoguesystem, wherein LLMs can be used to conduct TOD and ODD seamlessly. Thus, our survey provides a chronological perspective aligned with LM breakthroughs, offering a comprehensive review of state-of-the-art research outcomes. What's more, we focus on emerging topics and discuss open challenges, providing valuable insights into future directions for LLM-baseddialoguesystems. Through this exploration, we pave the way for a deepercomprehension of the evolution, guiding future developments in LM-based dialoguesystems.