Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models (2403.02271v2)

Published 4 Mar 2024 in cs.CL and cs.LG

Abstract: Pre-trained LLMs (PLMs) can be accurately fine-tuned for downstream text processing tasks. Recently, researchers have introduced several parameter-efficient fine-tuning methods that optimize input prompts or adjust a small number of model parameters (e.g LoRA). In this study, we explore the impact of altering the input text of the original task in conjunction with parameter-efficient fine-tuning methods. To most effectively rewrite the input text, we train a few-shot paraphrase model with a Maximum-Marginal Likelihood objective. Using six few-shot text classification datasets, we show that enriching data with paraphrases at train and test time enhances the performance beyond what can be achieved with parameter-efficient fine-tuning alone. The code used for our experiments can be found at https://github.com/SaeedNajafi/RIFF.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. LM-CPPF: Paraphrasing-guided data augmentation for contrastive prompt-based few-shot fine-tuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 670–681, Toronto, Canada. Association for Computational Linguistics.
  2. Palm 2 technical report.
  3. David Broughton. 1995. The assumptions and theory of public opinion polling, pages 15–33. Macmillan Education UK, London.
  4. Language models are few-shot learners.
  5. Language models are few-shot learners. CoRR, abs/2005.14165.
  6. An empirical survey of data augmentation for limited data learning in NLP. CoRR, abs/2106.07499.
  7. Decoder tuning: Efficient language understanding as decoding. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15072–15087, Toronto, Canada. Association for Computational Linguistics.
  8. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3369–3391, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  9. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  10. Margaret C. Donaldson. 1978. Children’s Minds.
  11. Wanyu Du and Yangfeng Ji. 2019. An empirical comparison on imitation learning and reinforcement learning for paraphrase generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6012–6018, Hong Kong, China. Association for Computational Linguistics.
  12. A survey of data augmentation approaches for NLP. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 968–988, Online. Association for Computational Linguistics.
  13. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3816–3830, Online. Association for Computational Linguistics.
  14. Unsupervised contextual paraphrase generation using lexical control and reinforcement learning. CoRR, abs/2103.12777.
  15. Efficient (soft) Q-learning for text generation with limited good data. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6969–6991, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  16. Efficient (soft) q-learning for text generation with limited good data.
  17. From language to programs: Bridging reinforcement learning and maximum marginal likelihood. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1051–1062, Vancouver, Canada. Association for Computational Linguistics.
  18. Dan Hendrycks and Kevin Gimpel. 2016. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. CoRR, abs/1606.08415.
  19. The curious case of neural text degeneration.
  20. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2790–2799. PMLR.
  21. Parameter-efficient transfer learning for NLP. CoRR, abs/1902.00751.
  22. Lora: Low-rank adaptation of large language models. CoRR, abs/2106.09685.
  23. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, page 168–177, New York, NY, USA. Association for Computing Machinery.
  24. Averaging weights leads to wider optima and better generalization.
  25. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems, volume 35, pages 22199–22213. Curran Associates, Inc.
  26. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  27. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–4597, Online. Association for Computational Linguistics.
  28. Making language models better reasoners with step-aware verifier. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5315–5333, Toronto, Canada. Association for Computational Linguistics.
  29. Paraphrase generation with deep reinforcement learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3865–3878, Brussels, Belgium. Association for Computational Linguistics.
  30. Memory augmented policy optimization for program synthesis and semantic parsing. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.
  31. Towards Understanding and Mitigating Social Biases in Language Models. In International Conference on Machine Learning, pages 6565–6576. PMLR.
  32. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  33. Exploring versatile generative language model via parameter-efficient transfer learning. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 441–459, Online. Association for Computational Linguistics.
  34. A learning-exploring method to generate diverse paraphrases with multi-objective deep reinforcement learning. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2310–2321, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  35. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. CoRR, abs/2107.13586.
  36. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.
  37. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61–68, Dublin, Ireland. Association for Computational Linguistics.
  38. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
  39. Ilya Loshchilov and Frank Hutter. 2017. Fixing weight decay regularization in adam. CoRR, abs/1711.05101.
  40. Asynchronous methods for deep reinforcement learning. CoRR, abs/1602.01783.
  41. Saeed Najafi and Alona Fyshe. 2023. Weakly-supervised questions for zero-shot relation extraction. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3075–3087, Dubrovnik, Croatia. Association for Computational Linguistics.
  42. Training language models to follow instructions with human feedback.
  43. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 115–124, Ann Arbor, Michigan. Association for Computational Linguistics.
  44. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463–2473, Hong Kong, China. Association for Computational Linguistics.
  45. Exploring diverse expressions for paraphrase generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3173–3182, Hong Kong, China. Association for Computational Linguistics.
  46. Language models are unsupervised multitask learners.
  47. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  48. Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR, abs/1910.10683.
  49. On the convergence of adam and beyond. CoRR, abs/1904.09237.
  50. Self-critical sequence training for image captioning. CoRR, abs/1612.00563.
  51. Entailment relation aware paraphrase generation. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):11258–11266.
  52. Proximal policy optimization algorithms. CoRR, abs/1707.06347.
  53. Toward human readable prompt tuning: Kubrick’s the shining is a good movie, and a good prompt too?
  54. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Online. Association for Computational Linguistics.
  55. Unsupervised paraphrasing via deep reinforcement learning. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 1800–1809, New York, NY, USA. Association for Computing Machinery.
  56. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.
  57. Black-box tuning for language-model-as-a-service. CoRR, abs/2201.03514.
  58. Policy gradient methods for reinforcement learning with function approximation. In Proceedings of the 12th International Conference on Neural Information Processing Systems, NIPS’99, page 1057–1063, Cambridge, MA, USA. MIT Press.
  59. Llama 2: Open foundation and fine-tuned chat models.
  60. DyLoRA: Parameter-efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3274–3287, Dubrovnik, Croatia. Association for Computational Linguistics.
  61. Diverse beam search for improved description of complex scenes. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).
  62. Maxim Kuznetsov Vladimir Vorobev. 2023. Chatgpt paraphrases dataset.
  63. Ellen M. Voorhees and Dawn M. Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’00, page 200–207, New York, NY, USA. Association for Computing Machinery.
  64. Chain of thought prompting elicits reasoning in large language models. CoRR, abs/2201.11903.
  65. Jason Wei and Kai Zou. 2019. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6382–6388, Hong Kong, China. Association for Computational Linguistics.
  66. Adaptive budget allocation for parameter-efficient fine-tuning.
  67. Opt: Open pre-trained transformer language models.
  68. Tempera: Test-time prompting via reinforcement learning.
  69. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations.
  70. Character-level convolutional networks for text classification. CoRR, abs/1509.01626.
  71. Automatic chain of thought prompting in large language models.
  72. Jianing Zhou and Suma Bhat. 2021. Paraphrase generation: A survey of the state of the art. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5075–5086, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  73. Large language models are human-level prompt engineers.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now