Machine Unlearning Fails to Remove Data Poisoning Attacks

Machine Unlearning Fails to Remove Data Poisoning Attacks

The paper "Machine Unlearning Fails to Remove Data Poisoning Attacks" by Martin Pawelczyk, Jimmy Z. Di, Yiwei Lu, Gautam Kamath, Ayush Sekhari, and Seth Neel presents a comprehensive reevaluation of the efficacy of current machine unlearning algorithms in handling data poisoning attacks. The authors perform extensive experimental analyses to gauge whether these unlearning methods can effectively mitigate the impact of several types of data poisoning attacks, including indiscriminate, targeted, and a newly introduced Gaussian poisoning attack.

Key Findings

1. Broad Failure Across Methods and Metrics:

   • The study demonstrates that state-of-the-art unlearning algorithms generally fail to remove the effects of data poisoning under various settings. Even when granted a generous compute budget (relative to retraining), none of the methods could reliably mitigate the adverse impacts of the poisoning attacks.

2. Implementation and Evaluation:

   • Seven unlearning algorithms, including Gradient Descent (GD), Noisy Gradient Descent (NGD), Gradient Ascent (GA), Exact Unlearning the last k layers (EUk), Catastrophically forgetting the last k layers (CFk), SCRUB, and NegGrad+, were examined across standard language and vision classification tasks.
   • Evaluation involved metrics tailored to each poisoning type. For Gaussian data poisoning, a new measure called the Gaussian Unlearning Score (GUS) was introduced, focusing on the correlation between added noise and the model's gradients.

3. Diverse Challenges in Unlearning:

   • Different types of data poisoning attacks present unique challenges:
     • Targeted Data Poisoning: The success of unlearning algorithms varied, with many failing to revert the effects on specified target samples.
     • Indiscriminate Data Poisoning: Methods like GD showed some improvement in model performance, yet failed to provide substantial benefits over retraining.
     • Gaussian Data Poisoning: Techniques such as NGD on Resnet-18 showed a decrease in the Gaussian Unlearning Score after unlearning, but not to the extent achieved by retraining, highlighting a gap in efficacy.
   • The success of unlearning algorithms was highly dependent on the underlying task, with some methods showing partial success in text classification but failing in image classification, and vice versa.

Hypotheses and Failures

The failure of unlearning methods was attributed to two chief hypotheses:

1. Large Model Shift Induced by Poisons:

   • The authors hypothesize that poison samples induce a larger model shift than random clean samples. This increased shift necessitates more update steps for effective unlearning, which the tested algorithms could not achieve within the practical computational budget.
   • Experiments using logistic regression on Resnet-18 features confirmed this hypothesis, showing significant (_1) norm distances for models trained with poisoned versus clean data.

2. Orthogonal Model Shifts:

   • Poison samples shift the model in a subspace orthogonal to that spanned by clean training samples. Gradient-based unlearning updates using only clean samples fail to correct shifts within this orthogonal subspace.
   • Linear regression experiments demonstrated that the desired update direction for mitigating poisoning was orthogonal to the gradient descent updates using clean data.

Implications and Future Directions

The findings suggested that heuristic methods for machine unlearning might convey a false sense of security. The results advocate for a more comprehensive evaluation involving diverse attack vectors and stress the necessity for either provable guarantees or thorough empirical evaluations for unlearning algorithms. Particularly, the study underscores that:

• Current heuristic unlearning methods are not sufficiently reliable for deployment in real-world scenarios.
• Future research should prioritize developing new unlearning techniques that can effectively handle the varied effects of data poisoning without the prohibitive costs associated with complete retraining.

The results also pose practical recommendations for improving existing methods. Enhancing the alignment of unlearning updates with the specific directions induced by poisons, leveraging additional structural information about the model, and combining multiple unlearning strategies might provide more robust solutions. The paper sets a valuable benchmark and guidepost for future unlearning research to achieve more dependable outcomes.