Bridging Items and Language: A Transition Paradigm for Large Language Model-Based Recommendation (2310.06491v2)

Abstract: Harnessing LLMs for recommendation is rapidly emerging, which relies on two fundamental steps to bridge the recommendation item space and the language space: 1) item indexing utilizes identifiers to represent items in the language space, and 2) generation grounding associates LLMs' generated token sequences to in-corpus items. However, previous methods exhibit inherent limitations in the two steps. Existing ID-based identifiers (e.g., numeric IDs) and description-based identifiers (e.g., titles) either lose semantics or lack adequate distinctiveness. Moreover, prior generation grounding methods might generate invalid identifiers, thus misaligning with in-corpus items. To address these issues, we propose a novel Transition paradigm for LLM-based Recommender (named TransRec) to bridge items and language. Specifically, TransRec presents multi-facet identifiers, which simultaneously incorporate ID, title, and attribute for item indexing to pursue both distinctiveness and semantics. Additionally, we introduce a specialized data structure for TransRec to ensure generating valid identifiers only and utilize substring indexing to encourage LLMs to generate from any position of identifiers. Lastly, TransRec presents an aggregated grounding module to leverage generated multi-facet identifiers to rank in-corpus items efficiently. We instantiate TransRec on two backbone models, BART-large and LLaMA-7B. Extensive results on three real-world datasets under diverse settings validate the superiority of TransRec.

• The paper introduces TransRec, a framework that leverages multi-facet identifiers and constrained generation to improve recommendation accuracy.
• It combines item IDs, titles, and attributes into a natural language format, enabling effective instruction tuning and precise item ranking.
• Empirical evaluations demonstrate that every component in TransRec is critical, with ablation studies confirming its superior performance over traditional models.

Bridging Items and Language: A Transition Paradigm for LLM-Based Recommendation

The paper "Bridging Items and Language: A Transition Paradigm for LLM-Based Recommendation" presents a unique approach to integrating LLMs into recommendation systems by focusing on two critical steps: item indexing and generation grounding. The proposed method, TransRec, enhances recommendation accuracy by incorporating a multi-facet identifier paradigm and advanced generation techniques.

Multi-Facet Item Indexing

The innovative approach in TransRec is the use of multi-facet identifiers, combining item IDs, titles, and attributes to achieve both distinctiveness and semantic richness. This combination allows the recommendation system to effectively leverage the vast knowledge embedded within LLMs.

Figure 1: Illustration of the two pivotal steps for LLM-based recommenders: item indexing and generation grounding.

TransRec processes each item in three facets for conversion into a natural language representation, facilitating instruction tuning. A strategic structure for data reconstruction ensures both comprehensive training and effective alignment with user interaction dynamics.

Generation Grounding Techniques

The generation grounding process in TransRec is meticulously designed to address two major issues: out-of-corpus generation and the reliance on initial token quality. TransRec employs constrained generation through FM-index, enabling both in-corpus identifier generation and position-free generation.

Figure 2: Overview of TransRec. Item indexing assigns each item multi-facet identifiers. For generation grounding, TransRec generates a set of identifiers in each facet and then grounds them to in-corpus items for ranking.

This allows the model to initiate generation from any position within an identifier, enhancing the flexibility and accuracy of recommendations. The aggregated grounding module combines identifiers across different facets to improve the ranking of in-corpus items, effectively utilizing the information gathered during the generation process.

Empirical Evaluation

TransRec's effectiveness is demonstrated through its superior performance on real-world datasets, exceeding traditional recommenders and contemporary LLM-based models. This success underscores the paper's claim that multi-facet identifiers and robust generation grounding significantly enhance LLM-based recommendation systems.

Figure 3: Illustration of reconstructed data based on the multi-facet identifiers. The bold texts in black refer to the user's historical interactions.

Extensive empirical validation, including ablation studies, highlights the importance of each component in TransRec. It was shown that the absence of any facet or constrained generation noticeably decreases performance, reinforcing their integral role in the methodology.

Implications and Future Directions

The research opens new pathways for developing LLM-based recommendation systems by emphasizing the need for distinctiveness and semantics in item indexing and robust solutions for generation grounding. Future work may include the development of automated approaches for selecting multi-facet identifiers, or neural network-based grounding modules to further harness the capabilities of LLMs in various recommendation contexts.

In conclusion, TransRec presents a compelling framework that effectively bridges the divide between natural LLMs and item recommendation systems, setting a new benchmark for future innovations in this domain.