What Changed? Investigating Debiasing Methods using Causal Mediation Analysis (2206.00701v1)

Abstract: Previous work has examined how debiasing LLMs affect downstream tasks, specifically, how debiasing techniques influence task performance and whether debiased models also make impartial predictions in downstream tasks or not. However, what we don't understand well yet is why debiasing methods have varying impacts on downstream tasks and how debiasing techniques affect internal components of LLMs, i.e., neurons, layers, and attentions. In this paper, we decompose the internal mechanisms of debiasing LLMs with respect to gender by applying causal mediation analysis to understand the influence of debiasing methods on toxicity detection as a downstream task. Our findings suggest a need to test the effectiveness of debiasing methods with different bias metrics, and to focus on changes in the behavior of certain components of the models, e.g.,first two layers of LLMs, and attention heads.