Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting (2306.11025v1)
Abstract: This paper presents a novel study on harnessing LLMs' (LLMs) outstanding knowledge and reasoning abilities for explainable financial time series forecasting. The application of machine learning models to financial time series comes with several challenges, including the difficulty in cross-sequence reasoning and inference, the hurdle of incorporating multi-modal signals from historical news, financial knowledge graphs, etc., and the issue of interpreting and explaining the model results. In this paper, we focus on NASDAQ-100 stocks, making use of publicly accessible historical stock price data, company metadata, and historical economic/financial news. We conduct experiments to illustrate the potential of LLMs in offering a unified solution to the aforementioned challenges. Our experiments include trying zero-shot/few-shot inference with GPT-4 and instruction-based fine-tuning with a public LLM model Open LLaMA. We demonstrate our approach outperforms a few baselines, including the widely applied classic ARMA-GARCH model and a gradient-boosting tree model. Through the performance comparison results and a few examples, we find LLMs can make a well-thought decision by reasoning over information from both textual news and price time series and extracting insights, leveraging cross-sequence information, and utilizing the inherent knowledge embedded within the LLM. Additionally, we show that a publicly available LLM such as Open-LLaMA, after fine-tuning, can comprehend the instruction to generate explainable forecasts and achieve reasonable performance, albeit relatively inferior in comparison to GPT-4.
- Time-series clustering–a decade review. Information systems 53 (2015), 16–38.
- Handbook of financial time series. Springer Science & Business Media.
- Clustering approach to stock market prediction. International Journal of Advanced Networking and Applications 3, 4 (2012), 1281.
- Arash Bahrammirzaee. 2010. A comparative survey of artificial intelligence applications in finance: artificial neural networks, expert system and hybrid intelligent systems. Neural Computing and Applications 19, 8 (2010), 1165–1195.
- Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
- Language Models are Few-Shot Learners. ArXiv abs/2005.14165 (2020).
- A review of threshold time series models in finance. Statistics and its Interface 4, 2 (2011), 167–181.
- Correlated anomaly detection from large streaming data. In 2018 IEEE International Conference on Big Data (Big Data). IEEE, 982–992.
- Financial time series forecasting with multi-modality graph neural network. Pattern Recognition 121 (2022), 108218.
- Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality. https://vicuna.lmsys.org
- Cross-correlation dynamics in financial time series. Physica A: Statistical Mechanics and its Applications 388, 5 (2009), 705–714.
- Piet De Jong and Ben Zehnwirth. 1983. Claims reserving, state-space models and the Kalman filter. Journal of the Institute of Actuaries 110, 1 (1983), 157–181.
- Alexiei Dingli and Karl Sant Fournier. 2017. Financial time series forecasting-a deep learning approach. International Journal of Machine Learning and Computing 7, 5 (2017), 118–122.
- Feike C Drost and Theo E Nijman. 1993. Temporal aggregation of GARCH processes. Econometrica: Journal of the Econometric Society (1993), 909–927.
- Vijay Prakash Dwivedi and Xavier Bresson. 2020. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699 (2020).
- Jianqing Fan. 2005. A selective overview of nonparametric methods in financial econometrics. Statist. Sci. (2005), 317–337.
- Christian Francq and Jean-Michel Zakoian. 2004. Maximum likelihood estimation of pure GARCH and ARMA-GARCH processes. Bernoulli 10, 4 (2004), 605–637.
- Xinyang Geng and Hao Liu. 2023. OpenLLaMA: An Open Reproduction of LLaMA. https://github.com/openlm-research/open_llama
- M Ghahramani and A Thavaneswaran. 2006. Financial applications of ARMA models with GARCH errors. The Journal of Risk Finance 7, 5 (2006), 525–543.
- Scaling and correlation in financial time series. Physica A: Statistical Mechanics and its Applications 287, 3-4 (2000), 362–373.
- A review of nonparametric time series analysis. International statistical review 65, 1 (1997), 49–72.
- MCMC-based estimation of Markov Switching ARMA–GARCH models. Applied Economics 43, 3 (2011), 259–271.
- Altaf Hossain and Mohammed Nasser. 2011. Comparison of the finite mixture of ARMA-GARCH, back propagation neural networks and support-vector machines in forecasting financial returns. Journal of Applied Statistics 38, 3 (2011), 533–551.
- Explainable multivariate time series classification: a deep neural network which learns to attend to important variables as well as time intervals. In Proceedings of the 14th ACM international conference on web search and data mining. 607–615.
- Deep learning with long short-term memory for time series prediction. IEEE Communications Magazine 57, 6 (2019), 114–119.
- Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.05398 (2023).
- Søren Johansen. 1995. Likelihood-based inference in cointegrated vector autoregressive models. OUP Oxford.
- Norberto Ritzmann Júnior and Julio Cesar Nievola. 2018. A generalized financial time series forecasting model based on automatic feature engineering using genetic algorithms and support vector machine. In 2018 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
- Nont Kanungsukkasem and Teerapong Leelanupab. 2019. Financial latent Dirichlet allocation (FinLDA): Feature extraction in text and data mining for financial time series prediction. IEEE Access 7 (2019), 71645–71664.
- Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems 30 (2017).
- Kyoung-jae Kim. 2003. Financial time series forecasting using support vector machines. Neurocomputing 55, 1-2 (2003), 307–319.
- Boris Kovalerchuk and Evgenii Vityaev. 2005. Data mining for financial applications. Data Mining and Knowledge Discovery Handbook (2005), 1203–1224.
- Financial time series forecasting with machine learning techniques: a survey.. In ESANN.
- An experimental review on deep learning architectures for time series forecasting. International Journal of Neural Systems 31, 03 (2021), 2130001.
- Tim Leung and Theodore Zhao. 2021. Financial time series analysis and forecasting with Hilbert–Huang transform feature generation and machine learning. Applied Stochastic Models in Business and Industry 37, 6 (2021), 993–1016.
- Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858 (2022).
- Wei Li and Jian Liao. 2017. A comparative study on trend forecasting approach for stock price time series. In 2017 11th IEEE International Conference on Anti-counterfeiting, Security, and Identification (ASID). IEEE, 74–78.
- Let’s Verify Step by Step. arXiv preprint arXiv:2305.20050 (2023).
- Junmei Ma and Xinli Yu. 2013. Research on the Pricing of the Basket Credit Default Swap. Mathematical Computation 2, 4 (2013).
- Time-series learning of latent-space dynamics for reduced-order model closure. Physica D: Nonlinear Phenomena 405 (2020), 132368.
- Alexey Natekin and Alois Knoll. 2013. Gradient boosting machines, a tutorial. Frontiers in neurorobotics 7 (2013), 21.
- Nam Nguyen and Brian Quanz. 2021. Temporal latent auto-encoder: A method for probabilistic multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 9117–9125.
- OpenAI. 2023a. GPT-4 Technical Report. ArXiv abs/2303.08774 (2023).
- OpenAI. 2023b. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]
- Ping-Feng Pai and Chih-Sheng Lin. 2005. A hybrid ARIMA and support vector machines model in stock price forecasting. Omega 33, 6 (2005), 497–505.
- Andrew J Patton. 2012. A review of copula models for economic time series. Journal of Multivariate Analysis 110 (2012), 4–18.
- Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277 (2023).
- Universal and nonuniversal properties of cross correlations in financial time series. Physical review letters 83, 7 (1999), 1471.
- Yuan Qi and Jing Xiao. 2018. Fintech: AI powers financial services to improve people’s lives. Commun. ACM 61, 11 (2018), 65–69.
- Sangeeta Rani and Geeta Sikka. 2012. Recent techniques of clustering of time series data: a survey. International Journal of Computer Applications 52, 15 (2012).
- Self-supervised graph transformer on large-scale molecular data. Advances in Neural Information Processing Systems 33 (2020), 12559–12571.
- Financial time series forecasting with deep learning: A systematic literature review: 2005–2019. Applied soft computing 90 (2020), 106181.
- On Efficient Training of Large-Scale Deep Learning Models: A Literature Review. arXiv preprint arXiv:2304.03589 (2023).
- Frank Smets and Raf Wouters. 2003. An estimated dynamic stochastic general equilibrium model of the euro area. Journal of the European economic association 1, 5 (2003), 1123–1175.
- Wen Song and Shigeru Fujimura. 2021. Capturing combination patterns of long-and short-term dependencies in multivariate time series forecasting. Neurocomputing 464 (2021), 72–82.
- Finite mixture of ARMA-GARCH model for stock price prediction. In Proceedings of the Third International Workshop on Computational Intelligence in Economics and Finance (CIEF’2003), North Carolina, USA. 1112–1119.
- Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/tatsu-lab/stanford_alpaca.
- Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).
- LLaMA: Open and Efficient Foundation Language Models. ArXiv abs/2302.13971 (2023).
- Theodore B Trafalis and Huseyin Ince. 2000. Support vector machine for regression and applications to financial forecasting. In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, Vol. 6. IEEE, 348–353.
- Chih F Tsai and Sammy P Wang. 2009. Stock price forecasting by hybrid machine learning techniques. In Proceedings of the international multiconference of engineers and computer scientists, Vol. 1. 60.
- Deep learning for stock market prediction from financial news articles. In 2017 IEEE international conference on computational intelligence and virtual environments for measurement systems and applications (CIVEMSA). IEEE, 60–65.
- Jar-Long Wang and Shu-Hui Chan. 2006. Stock market trading rule discovery using two-layer bias decision tree. Expert Systems with Applications 30, 4 (2006), 605–611.
- Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022).
- Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564 (2023).
- Support vector machine regression for volatile stock market prediction. In Intelligent Data Engineering and Automated Learning—IDEAL 2002: Third International Conference Manchester, UK, August 12–14, 2002 Proceedings 3. Springer, 391–396.
- Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems 34 (2021), 28877–28888.
- Machine learning techniques and use of event information for stock market prediction: A survey and evaluation. In International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC’06), Vol. 2. IEEE, 835–841.
- Graph transformer networks. Advances in neural information processing systems 32 (2019).
- Graph-bert: Only attention is needed for learning graph representations. arXiv preprint arXiv:2001.05140 (2020).
- Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923 (2023).
- A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).
- Eric Zivot and Jiahui Wang. 2006. Vector autoregressive models for multivariate time series. Modeling financial time series with S-PLUS® (2006), 385–429.