Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Collaborative Word-based Pre-trained Item Representation for Transferable Recommendation (2311.10501v2)

Published 17 Nov 2023 in cs.IR

Abstract: Item representation learning (IRL) plays an essential role in recommender systems, especially for sequential recommendation. Traditional sequential recommendation models usually utilize ID embeddings to represent items, which are not shared across different domains and lack the transferable ability. Recent studies use pre-trained LLMs (PLM) for item text embeddings (text-based IRL) that are universally applicable across domains. However, the existing text-based IRL is unaware of the important collaborative filtering (CF) information. In this paper, we propose CoWPiRec, an approach of Collaborative Word-based Pre-trained item representation for Recommendation. To effectively incorporate CF information into text-based IRL, we convert the item-level interaction data to a word graph containing word-level collaborations. Subsequently, we design a novel pre-training task to align the word-level semantic- and CF-related item representation. Extensive experimental results on multiple public datasets demonstrate that compared to state-of-the-art transferable sequential recommenders, CoWPiRec achieves significantly better performances in both fine-tuning and zero-shot settings for cross-scenario recommendation and effectively alleviates the cold-start issue. The code is available at: https://github.com/ysh-1998/CoWPiRec.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, “Session-based recommendations with recurrent neural networks,” arXiv preprint arXiv:1511.06939, 2015.
  2. S. Wang, L. Hu, Y. Wang, L. Cao, Q. Z. Sheng, and M. Orgun, “Sequential recommender systems: challenges, progress and prospects,” arXiv preprint arXiv:2001.04830, 2019.
  3. W.-C. Kang and J. McAuley, “Self-attentive sequential recommendation,” in 2018 IEEE international conference on data mining (ICDM).   IEEE, 2018, pp. 197–206.
  4. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  5. H. Ding, Y. Ma, A. Deoras, Y. Wang, and H. Wang, “Zero-shot recommender systems,” arXiv preprint arXiv:2105.08318, 2021.
  6. Y. Hou, S. Mu, W. X. Zhao, Y. Li, B. Ding, and J.-R. Wen, “Towards universal sequence representation learning for recommender systems,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 585–593.
  7. S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme, “Factorizing personalized markov chains for next-basket recommendation,” in Proceedings of the 19th international conference on World wide web, 2010, pp. 811–820.
  8. B. Hidasi and D. Tikk, “General factorization framework for context-aware recommendations,” Data Mining and Knowledge Discovery, vol. 30, no. 2, pp. 342–371, 2016.
  9. J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, and J. Ma, “Neural attentive session-based recommendation,” in Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017, pp. 1419–1428.
  10. S. Jang, H. Lee, H. Cho, and S. Chung, “Cities: Contextual inference of tail-item embeddings for sequential recommendation,” in 2020 IEEE International Conference on Data Mining (ICDM).   IEEE, 2020, pp. 202–211.
  11. J. Tang and K. Wang, “Personalized top-n sequential recommendation via convolutional sequence embedding,” in Proceedings of the eleventh ACM international conference on web search and data mining, 2018, pp. 565–573.
  12. Z. Liu, M. Cheng, Z. Li, Q. Liu, and E. Chen, “One person, one model—learning compound router for sequential recommendation,” in 2022 IEEE International Conference on Data Mining (ICDM), 2022, pp. 289–298.
  13. Z. He, H. Zhao, Z. Lin, Z. Wang, A. Kale, and J. McAuley, “Locker: Locally constrained self-attentive sequential recommendation,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 3088–3092.
  14. F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang, “Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer,” in Proceedings of the 28th ACM international conference on information and knowledge management, 2019, pp. 1441–1450.
  15. Y. Hou, B. Hu, Z. Zhang, and W. X. Zhao, “Core: simple and effective session-based recommendation within consistent representation space,” in Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, 2022, pp. 1796–1801.
  16. J. Chang, C. Gao, Y. Zheng, Y. Hui, Y. Niu, Y. Song, D. Jin, and Y. Li, “Sequential recommendation with graph neural networks,” in Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, 2021, pp. 378–387.
  17. S. Wu, Y. Tang, Y. Zhu, L. Wang, X. Xie, and T. Tan, “Session-based recommendation with graph neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 346–353.
  18. X. Xiao, H. Dai, Q. Dong, S. Niu, Y. Liu, and P. Liu, “Social4rec: Distilling user preference from social graph for video recommendation in tencent,” arXiv preprint arXiv:2302.09971, 2023.
  19. Q. Zhang, J. Li, Q. Jia, C. Wang, J. Zhu, Z. Wang, and X. He, “Unbert: User-news matching bert for news recommendation.” in IJCAI, 2021, pp. 3356–3362.
  20. C. Wu, F. Wu, T. Qi, and Y. Huang, “Empowering news recommendation with pre-trained language models,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021, pp. 1652–1656.
  21. Y. Yu, F. Wu, C. Wu, J. Yi, and Q. Liu, “Tiny-newsrec: Effective and efficient plm-based news recommendation,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 5478–5489.
  22. Y. Hou, Z. He, J. McAuley, and W. X. Zhao, “Learning vector-quantized item representation for transferable sequential recommenders,” arXiv preprint arXiv:2210.12316, 2022.
  23. J. Wang, F. Yuan, M. Cheng, J. M. Jose, C. Yu, B. Kong, Z. Wang, B. Hu, and Z. Li, “Transrec: Learning transferable recommendation from mixture-of-modality feedback,” arXiv preprint arXiv:2206.06190, 2022.
  24. S. Mu, Y. Hou, W. X. Zhao, Y. Li, and B. Ding, “Id-agnostic user behavior pre-training for sequential recommendation,” in Information Retrieval: 28th China Conference, CCIR 2022, Chongqing, China, September 16–18, 2022, Revised Selected Papers.   Springer, 2023, pp. 16–27.
  25. Z. Yuan, F. Yuan, Y. Song, Y. Li, J. Fu, F. Yang, Y. Pan, and Y. Ni, “Where to go next for recommender systems? id-vs. modality-based recommender models revisited,” arXiv preprint arXiv:2303.13835, 2023.
  26. F. Zhu, Y. Wang, C. Chen, J. Zhou, L. Li, and G. Liu, “Cross-domain recommendation: challenges, progress, and prospects,” arXiv preprint arXiv:2103.01696, 2021.
  27. K. Xu, Z. Wang, W. Zheng, Y. Ma, C. Wang, N. Jiang, and C. Cao, “A centralized-distributed transfer model for cross-domain recommendation based on multi-source heterogeneous transfer learning,” in 2022 IEEE International Conference on Data Mining (ICDM).   IEEE, 2022, pp. 1269–1274.
  28. G. Hu, Y. Zhang, and Q. Yang, “Conet: Collaborative cross networks for cross-domain recommendation,” in Proceedings of the 27th ACM international conference on information and knowledge management, 2018, pp. 667–676.
  29. C. Wu, F. Wu, T. Qi, J. Lian, Y. Huang, and X. Xie, “Ptum: Pre-training user model from unlabeled user behaviors via self-supervision,” arXiv preprint arXiv:2010.01494, 2020.
  30. C. Xiao, R. Xie, Y. Yao, Z. Liu, M. Sun, X. Zhang, and L. Lin, “Uprec: User-aware pre-training for recommender systems,” arXiv preprint arXiv:2102.10989, 2021.
  31. F. Yuan, G. Zhang, A. Karatzoglou, J. Jose, B. Kong, and Y. Li, “One person, one model, one world: Learning continual user representation without forgetting,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021, pp. 696–705.
  32. A. P. Singh and G. J. Gordon, “Relational learning via collective matrix factorization,” in Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, 2008, pp. 650–658.
  33. F. Zhu, C. Chen, Y. Wang, G. Liu, and X. Zheng, “Dtcdr: A framework for dual-target cross-domain recommendation,” in Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, pp. 1533–1542.
  34. K. Shin, H. Kwak, S. Y. Kim, M. N. Ramstrom, J. Jeong, J.-W. Ha, and K.-M. Kim, “Scaling law for recommendation models: Towards general-purpose user representations,” arXiv preprint arXiv:2111.11294, 2021.
  35. S. Shi, W. Ma, Z. Wang, M. Zhang, K. Fang, J. Xu, Y. Liu, and S. Ma, “Wg4rec: Modeling textual content with word graph for news recommendation,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 1651–1660.
  36. P. Liu, L. Zhang, and J. A. Gulla, “Pre-train, prompt and recommendation: A comprehensive survey of language modelling paradigm adaptations in recommender systems,” arXiv preprint arXiv:2302.03735, 2023.
  37. W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” Advances in neural information processing systems, vol. 30, 2017.
  38. K. Zhou, H. Wang, W. X. Zhao, Y. Zhu, S. Wang, F. Zhang, Z. Wang, and J.-R. Wen, “S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization,” in Proceedings of the 29th ACM international conference on information & knowledge management, 2020, pp. 1893–1902.
  39. J. Ni, J. Li, and J. McAuley, “Justifying recommendations using distantly-labeled reviews and fine-grained aspects,” in Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019, pp. 188–197.
  40. W. X. Zhao, S. Mu, Y. Hou, Z. Lin, Y. Chen, X. Pan, K. Li, Y. Lu, H. Wang, C. Tian et al., “Recbole: Towards a unified, comprehensive and efficient framework for recommendation algorithms,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 4653–4664.
  41. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al., “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38–45.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now