· Trending Papers
Emergent Mind

Inverse Scaling: When Bigger Isn't Better

(2306.09479)
Published Jun 15, 2023 in cs.CL , cs.AI , and cs.CY

Abstract

Work on scaling laws has found that LLMs (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, we present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale, e.g., due to flaws in the training objective and data. We present empirical evidence of inverse scaling on 11 datasets collected by running a public contest, the Inverse Scaling Prize, with a substantial prize pool. Through analysis of the datasets, along with other examples found in the literature, we identify four potential causes of inverse scaling: (i) preference to repeat memorized sequences over following in-context instructions, (ii) imitation of undesirable patterns in the training data, (iii) tasks containing an easy distractor task which LMs could focus on, rather than the harder real task, and (iv) correct but misleading few-shot demonstrations of the task. We release the winning datasets at https://inversescaling.com/data to allow for further investigation of inverse scaling. Our tasks have helped drive the discovery of U-shaped and inverted-U scaling trends, where an initial trend reverses, suggesting that scaling trends are less reliable at predicting the behavior of larger-scale models than previously understood. Overall, our results suggest that there are tasks for which increased model scale alone may not lead to progress, and that more careful thought needs to go into the data and objectives for training language models.

We're not able to analyze this paper right now due to high demand.

Please check back later (sorry!).

Generate a detailed summary of this paper with a premium account.

We ran into a problem analyzing this paper.

Subscribe by Email

Get summaries of trending comp sci papers delivered straight to your inbox:

Unsubscribe anytime.

References

  1. Anthropic. Introducing Claude. https://www.anthropic.com/index/introducing-claude, 2023. Accessed: 2023-03-15.

  2. A General Language Assistant as a Laboratory for Alignment
  3. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
  4. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pp.  610–623, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383097. doi: 10.1145/3442188.3445922. https://doi.org/10.1145/3442188.3445922.
  5. On the Opportunities and Risks of Foundation Models

  6. Language Models are Few-Shot Learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp.  1877–1901. Curran Associates, Inc., 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.

  7. Quantifying Memorization Across Neural Language Models
  8. Evaluating Large Language Models Trained on Code
  9. PaLM: Scaling Language Modeling with Pathways

  10. Deep Reinforcement Learning from Human Preferences. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. https://proceedings.neurips.cc/paper/2017/file/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf.

  11. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp.  4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1423. https://aclanthology.org/N19-1423.

  12. Behavioral Biases in the NFL Gambling Market: Overreaction to News and the Recency Bias. SSRN Electronic Journal, 2021. doi: 10.2139/ssrn.3861231. https://doi.org/10.2139/ssrn.3861231.

  13. Predictability and Surprise in Large Generative Models. In 2022 ACM Conference on Fairness, Accountability, and Transparency. ACM, jun 2022. doi: 10.1145/3531146.3533229. https://doi.org/10.1145%2F3531146.3533229.
  14. The Capacity for Moral Self-Correction in Large Language Models
  15. Scaling Laws for Reward Model Overoptimization
  16. Measuring Massive Multitask Language Understanding
  17. Training Compute-Optimal Large Language Models
  18. Risks from Learned Optimization in Advanced Machine Learning Systems

  19. On the Psychology of Prediction. Psychological Review, 80(4):237–251, July 1973. doi: 10.1037/h0034747. https://doi.org/10.1037/h0034747.

  20. Deduplicating Training Data Mitigates Privacy Risks in Language Models. In ICML, 2022. https://proceedings.mlr.press/v162/kandpal22a/kandpal22a.pdf.

  21. Scaling Laws for Neural Language Models
  22. Alignment of Language Agents
  23. Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models
  24. Pretraining Language Models with Human Preferences
  25. Holistic Evaluation of Language Models

  26. TruthfulQA: Measuring How Models Mimic Human Falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.  3214–3252, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.229. https://aclanthology.org/2022.acl-long.229.

  27. Inverse Scaling Prize Ideas, Oct 2022. https://ethanperez.net/inverse-scaling-prize-ideas/.

  28. The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python

  29. Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp.  2381–2391, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1260. https://aclanthology.org/D18-1260.

  30. COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models
  31. The Alignment Problem from a Deep Learning Perspective
  32. Show Your Work: Scratchpads for Intermediate Computation with Language Models
  33. In-context Learning and Induction Heads

  34. OpenAI. Model index for researchers. https://platform.openai.com/docs/model-index-for-researchers, 2022. Accessed: 2023-04-06.

  35. GPT-4 Technical Report
  36. Training language models to follow instructions with human feedback

  37. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.  1525–1534, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653/v1/P16-1144. https://aclanthology.org/P16-1144.

  38. BBQ: A Hand-Built Bias Benchmark for Question Answering. In Findings of the Association for Computational Linguistics: ACL 2022, pp.  2086–2105, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.findings-acl.165. https://aclanthology.org/2022.findings-acl.165.

  39. True Few-Shot Learning with Language Models
  40. Discovering Language Model Behaviors with Model-Written Evaluations

  41. Sundar Pichai. An important next step on our AI journey, 2023. https://blog.google/technology/ai/bard-google-ai-search-updates/.

  42. Counterfactual Story Reasoning and Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp.  5043–5053, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1509. https://aclanthology.org/D19-1509.

  43. Language Models are Unsupervised Multitask Learners. Technical report, OpenAI
  44. Scaling Language Models: Methods, Analysis & Insights from Training Gopher

  45. Introducing ChatGPT. https://openai.com/blog/chatgpt, 2022. Accessed: 2023-03-15.

  46. Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals

  47. Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp.  5861–5873. Curran Associates, Inc., 2021. https://proceedings.neurips.cc/paper/2021/file/2e855f9489df0712b4bd8ea9e2848c5a-Paper.pdf.

  48. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

  49. Learning to Summarize with Human Feedback. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp.  3008–3021. Curran Associates, Inc., 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/1f89885d556929e98d3ef9b86448f951-Paper.pdf.

  50. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353–355, Brussels, Belgium, 2018. Association for Computational Linguistics. doi: 10.18653/v1/W18-5446. http://aclweb.org/anthology/W18-5446.

  51. SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. https://papers.nips.cc/paper_files/paper/2019/hash/4496bf24afe7fab6f046bf4923da8de6-Abstract.html.

  52. Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models. In Findings of the Association for Computational Linguistics: NAACL 2022, pp.  1719–1729, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.findings-naacl.130. https://aclanthology.org/2022.findings-naacl.130.

  53. Peter Cathcart Wason. Reasoning about a Rule. Quarterly Journal of Experimental Psychology, 20(3):273–281, 1968. doi: 10.1080/14640746808400161. https://journals.sagepub.com/doi/10.1080/14640746808400161.

  54. Inverse scaling can become U-shaped
  55. Emergent Abilities of Large Language Models

  56. Chain of Thought Prompting Elicits Reasoning in LLMs. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022c. https://openreview.net/forum?id=_VjQlMeSB_J.

  57. OPT: Open Pre-trained Transformer Language Models
  58. Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models. In Findings of the Association for Computational Linguistics (ACL Findings)

Show All 58