Pre-trained Large Language Models for Financial Sentiment Analysis (2401.05215v1)
Abstract: Financial sentiment analysis refers to classifying financial text contents into sentiment categories (e.g. positive, negative, and neutral). In this paper, we focus on the classification of financial news title, which is a challenging task due to a lack of large amount of training samples. To overcome this difficulty, we propose to adapt the pretrained LLMs [1, 2, 3] to solve this problem. The LLMs, which are trained from huge amount of text corpora,have an advantage in text understanding and can be effectively adapted to domain-specific task while requiring very few amount of training samples. In particular, we adapt the open-source Llama2-7B model (2023) with the supervised fine-tuning (SFT) technique [4]. Experimental evaluation shows that even with the 7B model (which is relatively small for LLMs), our approach significantly outperforms the previous state-of-the-art algorithms.
- Improving language understanding by generative pre-training. 2018.
- Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9, 2019.
- Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
- Llama: Open and efficient foundation language models, 2023.
- Bing Liu. Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Studies in Natural Language Processing. Cambridge University Press, 2 edition, 2020.
- Bo Pang and Lillian Lee. Opinion Mining and Sentiment Analysis, volume 2. 2008.
- Sentiment analysis of twitter data. In Proceedings of the Workshop on Languages in Social Media, LSM ’11, page 30–38, USA, 2011. Association for Computational Linguistics.
- Introduction, pages 1–21. Springer International Publishing, Cham, 2015.
- Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65(4):782–796, 2014.
- Decision support from financial disclosures with deep neural networks and transfer learning. Decision Support Systems, 104:38–48, December 2017.
- Big data: Deep learning for financial sentiment analysis. Journal of Big Data, 5, 2018.
- Sentence-level sentiment analysis of financial news using distributed text representations and multi-instance learning, 2018.
- Finsslx: A sentiment analysis model for the financial domain using text simplification. In 2018 IEEE 12th International Conference on Semantic Computing (ICSC), pages 318–319, 2018.
- Dogu Araci. Finbert: Financial sentiment analysis with pre-trained language models, 2019.
- Finbert: A pretrained language model for financial communications, 2020.
- Recent progress on text summarisation based on bert and gpt. In Zhi Jin, Yuncheng Jiang, Robert Andrei Buchmann, Yaxin Bi, Ana-Maria Ghiran, and Wenjun Ma, editors, Knowledge Science, Engineering and Management, pages 225–241, Cham, 2023. Springer Nature Switzerland.
- Sentimentgpt: Exploiting gpt for advanced sentiment analysis and its departure from current machine learning, 2023.
- Long short-term memory. Neural Comput, 9(8):1735–1780, 1997.
- Deep contextualized word representations. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
- Universal language model fine-tuning for text classification, 2018.
- Srikumar Krishnamoorthy. Sentiment analysis of financial news articles using performance indicators. Knowledge and Information Systems, 56(2):373–394, 2018.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.