Benchmarking News Recommendation in the Era of Green AI (2403.04736v2)
Abstract: Over recent years, news recommender systems have gained significant attention in both academia and industry, emphasizing the need for a standardized benchmark to evaluate and compare the performance of these systems. Concurrently, Green AI advocates for reducing the energy consumption and environmental impact of machine learning. To address these concerns, we introduce the first Green AI benchmarking framework for news recommendation, known as GreenRec, and propose a metric for assessing the tradeoff between recommendation accuracy and efficiency. Our benchmark encompasses 30 base models and their variants, covering traditional end-to-end training paradigms as well as our proposed efficient only-encode-once (OLEO) paradigm. Through experiments consuming 2000 GPU hours, we observe that the OLEO paradigm achieves competitive accuracy compared to state-of-the-art end-to-end paradigms and delivers up to a 2992\% improvement in sustainability metrics.
- Neural news recommendation with long-and short-term user representations. In ACL. 336–345.
- Behavior sequence transformer for e-commerce recommendation in alibaba. In DLP4Rec.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv (2018).
- Torchrec: a pytorch domain library for recommendation systems. In RecSys.
- Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In EMNLP. Association for Computational Linguistics, Doha, Qatar.
- Quantifying the Carbon Emissions of Machine Learning. arXiv (2019).
- Boosting Deep CTR Prediction with a Plug-and-Play Pre-trainer for News Recommendation. In COLING. International Committee on Computational Linguistics.
- Piotr Przybyła and Matthew Shardlow. 2022. Using NLP to quantify the environmental cost and diversity benefits of in-person NLP conferences. In ACL Findings.
- Green ai. Commun. ACM (2020).
- Weichen Shen. 2017. DeepCTR: Easy-to-use,Modular and Extendible package of deep-learning based CTR models.
- Towards Sustainability-aware Recommender Systems: Analyzing the Trade-off Between Algorithms Performance and Carbon Footprint. In RecSys.
- Attention is all you need. arXiv (2017).
- Deep & cross network for ad click predictions. In ADKDD.
- Urban dictionary embeddings for slang NLP applications. In LREC.
- Neural news recommendation with attentive multi-view learning. IJCAI (2019).
- NPA: neural news recommendation with personalized attention. In SIGKDD.
- Neural news recommendation with multi-head self-attention. In EMNLP-IJCNLP.
- Empowering News Recommendation with Pre-trained Language Models. SIGIR (2021).
- Mind: A large-scale dataset for news recommendation. In ACL.
- Where to go next for recommender systems? id-vs. modality-based recommender models revisited. arXiv (2023).
- UNBERT: User-News Matching BERT for News Recommendation. In IJCAI.
- Deeprec: An open-source toolkit for deep learning based recommendation. arXiv (2019).
- RecBole 2.0: towards a more up-to-date recommendation library. In CIKM.
- Deep interest network for click-through rate prediction. In SIGKDD.
- Bars: Towards open benchmarking for recommender systems. In SIGIR.
- Open benchmarking for click-through rate prediction. In CIKM.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.