Understanding Language Model Circuits through Knowledge Editing (2406.17241v3)

Abstract: Recent advances in LLM interpretability have identified circuits, critical subnetworks that replicate model behaviors, yet how knowledge is structured within these crucial subnetworks remains opaque. To gain an understanding toward the knowledge in the circuits, we conduct systematic knowledge editing experiments on the circuits of the GPT-2 LLM. Our analysis reveals intriguing patterns in how circuits respond to editing attempts, the extent of knowledge distribution across network components, and the architectural composition of knowledge-bearing circuits. These findings offer insights into the complex relationship between model circuits and knowledge representation, deepening the understanding of how information is organized within LLMs. Our findings offer novel insights into the ``meanings'' of the circuits, and introduce directions for further interpretability and safety research of LLMs.