Large language models converge toward human-like concept organization (2308.15047v1)

Abstract: LLMs show human-like performance in knowledge extraction, reasoning and dialogue, but it remains controversial whether this performance is best explained by memorization and pattern matching, or whether it reflects human-like inferential semantics and world knowledge. Knowledge bases such as WikiData provide large-scale, high-quality representations of inferential semantics and world knowledge. We show that LLMs learn to organize concepts in ways that are strikingly similar to how concepts are organized in such knowledge bases. Knowledge bases model collective, institutional knowledge, and LLMs seem to induce such knowledge from raw text. We show that bigger and better models exhibit more human-like concept organization, across four families of LLMs and three knowledge graph embeddings.