Machine Unlearning in Large Language Models (2405.15152v1)
Abstract: Machine unlearning, a novel area within artificial intelligence, focuses on addressing the challenge of selectively forgetting or reducing undesirable knowledge or behaviors in machine learning models, particularly in the context of LLMs. This paper introduces a methodology to align LLMs, such as Open Pre-trained Transformer LLMs, with ethical, privacy, and safety standards by leveraging the gradient ascent algorithm for knowledge unlearning. Our approach aims to selectively erase or modify learned information in LLMs, targeting harmful responses and copyrighted content. This paper presents a dual-pronged approach to enhance the ethical and safe behavior of LLMs by addressing the issues of harmful responses and copyrighted content. To mitigate harmful responses, we applied gradient ascent on the PKU dataset, achieving a 75\% reduction in harmful responses for Open Pre-trained Transformer LLMs (OPT1.3b and OPT2.7b) \citet{zhang2022opt} while retaining previous knowledge using the TruthfulQA dataset \citet{DBLP:journals/corr/abs-2109-07958}. For handling copyrighted content, we constructed a custom dataset based on the Lord of the Rings corpus and aligned LLMs (OPT1.3b and OPT2.7b) \citet{zhang2022opt} through LoRA: Low-Rank Adaptation of LLMs \citet{DBLP:journals/corr/abs-2106-09685} finetuning. Subsequently, we employed gradient ascent to unlearn the Lord of the Rings content, resulting in a remarkable reduction in the presence of copyrighted material. To maintain a diverse knowledge base, we utilized the Book Corpus dataset. Additionally, we propose a new evaluation technique for assessing the effectiveness of harmful unlearning.
- Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.
- Safe rlhf: Safe reinforcement learning from human feedback, 2023a.
- Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXiv:2310.12773, 2023b.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
- Lora: Low-rank adaptation of large language models. CoRR, abs/2106.09685, 2021. URL https://arxiv.org/abs/2106.09685.
- Truthfulqa: Measuring how models mimic human falsehoods. CoRR, abs/2109.07958, 2021. URL https://arxiv.org/abs/2109.07958.
- A survey on machine unlearning and model updates. IEEE Transactions on Computational Social Systems, 7(3):664–675, 2020.
- Roberta: A robustly optimized bert approach. arXiv preprint arXiv:1907.11692, 2019.
- Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, page 311–318, USA, 2002. Association for Computational Linguistics. doi: 10.3115/1073083.1073135. URL https://doi.org/10.3115/1073083.1073135.
- Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019.
- Privacy-preserving deep learning. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1310–1321, 2015.
- Fast yet effective machine unlearning. CoRR, abs/2111.08947, 2021. URL https://arxiv.org/abs/2111.08947.
- Unrolling SGD: understanding factors influencing machine unlearning. CoRR, abs/2109.13398, 2021. URL https://arxiv.org/abs/2109.13398.
- Machine unlearning of features and labels. CoRR, abs/2108.11577, 2021. URL https://arxiv.org/abs/2108.11577.
- Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237, 2019.
- Large language model unlearning. arXiv preprint arXiv:2310.10683, 2023.
- Opt: Open pre-trained transformer language models, 2022.
- Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. CoRR, abs/1506.06724, 2015. URL http://arxiv.org/abs/1506.06724.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
