Reindex-Then-Adapt: Improving Large Language Models for Conversational Recommendation (2405.12119v1)

Abstract: LLMs are revolutionizing conversational recommender systems by adeptly indexing item content, understanding complex conversational contexts, and generating relevant item titles. However, controlling the distribution of recommended items remains a challenge. This leads to suboptimal performance due to the failure to capture rapidly changing data distributions, such as item popularity, on targeted conversational recommendation platforms. In conversational recommendation, LLMs recommend items by generating the titles (as multiple tokens) autoregressively, making it difficult to obtain and control the recommendations over all items. Thus, we propose a Reindex-Then-Adapt (RTA) framework, which converts multi-token item titles into single tokens within LLMs, and then adjusts the probability distributions over these single-token item titles accordingly. The RTA framework marries the benefits of both LLMs and traditional recommender systems (RecSys): understanding complex queries as LLMs do; while efficiently controlling the recommended item distributions in conversational recommendations as traditional RecSys do. Our framework demonstrates improved accuracy metrics across three different conversational recommendation datasets and two adaptation settings

• The paper introduces the RTA framework to realign LLM outputs with dynamic recommendation platforms by reindexing multi-token item titles into single tokens and adapting logits.
• It employs trainable aggregators and adapts logits via bias adjustment or RecSys gating, achieving significant efficiency gains and reduced model sizes compared to traditional methods.
• Experiments across multiple CRS datasets show improved recommendation accuracy (HIT@10 and NDCG@10) and up to 100× faster inference than generative retrieval.

Reindex-Then-Adapt: Enhancing LLMs for Conversational Recommendation

Introduction

The paper introduces the Reindex-Then-Adapt (RTA) framework to address the distribution misalignment between LLMs and target conversational recommendation platforms. LLMs, when used for conversational recommendation, exhibit strong capabilities in indexing item content and understanding complex conversational contexts. However, their generative retrieval paradigm leads to suboptimal control over the distribution of recommended items, particularly in dynamic environments where item popularity shifts rapidly. The RTA framework proposes a two-step solution: reindexing multi-token item titles into single tokens within LLMs, followed by adapting the probability distributions over these single-token item titles to better align with target data distributions.

LLMs as Differentiable Search Indexes in CRS

LLMs can be conceptualized as Differentiable Search Index (DSI) models, where item content and conversational context are mapped to item indices via Learn to Index (L2I) and Learn to Retrieve (L2R) tasks, respectively. Empirical analysis demonstrates that LLMs, even without fine-tuning, possess substantial knowledge of popular items, as evidenced by high HIT@5 scores for frequently occurring movies in the ReDIAL dataset.

Figure 1: HIT@5 accuracy of various LLMs on item indexing tasks using Wikipedia movie descriptions, stratified by item frequency in ReDIAL.

Despite this, LLMs' internal item popularity distributions often diverge from those of the target platform, both statically and dynamically. This misalignment is exacerbated by the rapid evolution of item popularity over time, as shown in the Reddit-Movie dataset.

The RTA Framework

The RTA framework consists of two main steps:

1. Reindex Step: Multi-token item titles are aggregated into single-token embeddings using trainable aggregators (e.g., weighted pooling, RNN, transformer-based models). This enables efficient extraction of logit vectors for all items, facilitating direct control over recommendation distributions.
2. Adapt Step: The logit vectors from the reindexed LLM are adjusted via bias terms or combined with traditional RecSys models using a gating mechanism. This step aligns the recommendation probabilities with the target data distribution, allowing for rapid adaptation to changing item popularity.

   Figure 2: The RTA framework: reindexing item titles as single tokens and adapting logits via bias adjustment or RecSys gating.

Implementation Details

Reindexing

The reindex step aggregates multi-token embeddings into single-token representations. The aggregation is learned using a contrastive loss, ensuring semantic preservation. Aggregators such as weighted pooling, GRU-based RNNs, and transformers are evaluated. The weighted pooling method, despite its simplicity, achieves competitive performance, indicating that existing LLM token embeddings are sufficiently expressive for this task.

Figure 3: Comparison of single-token embedding methods and their HIT@5 accuracy after reindexing.

Adaptation

The adapt step employs either bias term adjustment (affine transformation of logits) or RecSys gating (weighted combination of LLM and RecSys logits). The bias term approach is parameter-efficient and effective for small datasets, while RecSys gating leverages collaborative filtering for larger datasets with richer interaction data. Both methods are trained using maximum likelihood estimation on the target platform's data.

Experimental Results

Experiments are conducted on three CRS datasets: INSPIRED, ReDIAL, and Reddit-V1.5. The RTA framework consistently outperforms traditional RecSys models, zero-shot LLMs, dense retrievers, and prior CRS models in recommendation accuracy (HIT@10 and NDCG@10). Notably, the reindex step alone yields substantial efficiency gains, with the aggregator-based methods being approximately $10\times$ smaller than OOV embedding tables and $233\times$ smaller than the Llama2-7b base model. Recommendation inference is accelerated by $100\times$ compared to generative retrieval.

The adapt step further improves accuracy, with bias term adjustment excelling on small datasets and RecSys gating providing superior results on larger datasets. Ablation studies confirm the effectiveness of both multiplicative and additive bias terms, as well as the choice of RecSys model (FISM vs. SASRec) depending on dataset size and complexity.

Qualitative Analysis

Case studies demonstrate that the RTA framework enables LLMs to generate recommendations that are both contextually relevant and aligned with platform-specific popularity distributions. The framework also supports seamless integration with natural language generation for user-facing responses.

Figure 4: Example recommendations from Llama2, Llama-R, and Llama-RTA (+SASRec) on a ReDIAL conversation, with corresponding natural language responses.

Implications and Future Directions

The RTA framework provides a scalable, architecture-agnostic solution for aligning LLM-based recommenders with dynamic target distributions. Its modularity allows for rapid adaptation to distribution shifts without retraining the base LLM, making it suitable for production environments with evolving item catalogs. The approach is compatible with proprietary models (e.g., GPT-3.5-turbo) and can be extended to larger LLMs as compute resources permit.

Theoretically, the framework bridges the gap between generative retrieval and traditional collaborative filtering, leveraging the strengths of both paradigms. Practically, it enables fine-grained control over recommendation outputs, supporting objectives such as fairness, diversity, and platform-specific constraints.

Future work may explore more sophisticated aggregation strategies, dynamic gating mechanisms, and integration with user modeling for personalized recommendations. Additionally, extending the framework to other domains (e.g., e-commerce, music) and evaluating its robustness to cold-start items remain promising directions.

Conclusion

The Reindex-Then-Adapt framework effectively addresses the distribution misalignment challenge in LLM-based conversational recommendation. By reindexing item titles as single tokens and adapting recommendation distributions via bias adjustment or RecSys gating, the framework achieves superior accuracy, efficiency, and controllability. Its design facilitates rapid adaptation to dynamic environments and provides a foundation for further advances in LLM-powered recommender systems.