Transfer Learning and Transformer Architecture for Financial Sentiment Analysis (2405.01586v1)
Abstract: Financial sentiment analysis allows financial institutions like Banks and Insurance Companies to better manage the credit scoring of their customers in a better way. Financial domain uses specialized mechanisms which makes sentiment analysis difficult. In this paper, we propose a pre-trained LLM which can help to solve this problem with fewer labelled data. We extend on the principles of Transfer learning and Transformation architecture principles and also take into consideration recent outbreak of pandemics like COVID. We apply the sentiment analysis to two different sets of data. We also take smaller training set and fine tune the same as part of the model.
- B. M. Tim Loughran, “Textual analysis in finance,” Annual Review of Financial Economics, vol. 12, pp. 357–375, 2020.
- A. S. e. a. Pekka Malo, “Good debt or bad debt: Detecting semantic orientations in economic texts,” Journal of the Association for Information Science and Technology, vol. 65, no. 4, pp. 782–796, 2014.
- Y. Z. Ryan Kiros, Ruslan Salakhutdinov, “Skip-thought vectors,” Sanja Fidler, NIPS’15: Proceedings of the 28th International Conference on Neural Information Processing Systems, vol. 2, pp. 3294–3302, 2015.
- J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pretraining of deep bidirectional transformers for language understanding,” Proceedings of NAACL-HLT 2019, pp. 4171–4186, 2019.
- M. I. Matthew E Peters, “Deep contextualized word representations,” 2018.
- J. V. Robin Brochier, Adrien Guille, “Global vectors for node representations,” The World Wide Web Conference on - WWW ’19, 2019.
- K. C. e. a. Tomas Mikolov, “Efficient estimation of word representations in vector space,” 2013.
- F. S. e. a. Chuanqi Tan, “A survey on deep transfer learning,” 2018.
- I. G. Nils Reimers, “Sentence-bert: Sentence embeddings using siamese bert-networks,” 2019.
- Y. Y. e. a. David D. Lewis, “Rcv1: A new benchmark collection for text categorization research,” J. Mach. Learn. Res., vol. 5, pp. 361–397, 2004.
- S. R. Jeremy Howard, “Universal language model fine-tuning for text classification,” 2018.
- J. L. e. a. Yang You, “Large batch optimization for deep learning: Training bert in 76 minutes,” 2020.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.