RecVAE: a New Variational Autoencoder for Top-N Recommendations with Implicit Feedback (1912.11160v1)

Abstract: Recent research has shown the advantages of using autoencoders based on deep neural networks for collaborative filtering. In particular, the recently proposed Mult-VAE model, which used the multinomial likelihood variational autoencoders, has shown excellent results for top-N recommendations. In this work, we propose the Recommender VAE (RecVAE) model that originates from our research on regularization techniques for variational autoencoders. RecVAE introduces several novel ideas to improve Mult-VAE, including a novel composite prior distribution for the latent codes, a new approach to setting the $\beta$ hyperparameter for the $\beta$-VAE framework, and a new approach to training based on alternating updates. In experimental evaluation, we show that RecVAE significantly outperforms previously proposed autoencoder-based models, including Mult-VAE and RaCT, across classical collaborative filtering datasets, and present a detailed ablation study to assess our new developments. Code and models are available at https://github.com/ilya-shenbin/RecVAE.

• The paper introduces RecVAE, a refined VAE that enhances collaborative filtering through a novel encoder design and a composite prior to stabilize training.
• It employs an adaptive β rescaling scheme and alternating update strategy to better handle data distribution and optimize learning.
• Empirical results demonstrate improved Recall@20, Recall@50, and NDCG@100 on datasets like MovieLens-20M and Netflix, indicating strong practical impact.

Overview of RecVAE: A Variational Autoencoder Framework for Collaborative Filtering with Implicit Feedback

The paper in question introduces RecVAE, a sophisticated variant of Variational Autoencoders (VAE) tailored specifically for collaborative filtering tasks with implicit feedback. Built upon the widely acknowledged Mult-VAE model, RecVAE represents a significant refinement aimed at enhancing recommendation accuracy and training stability across several classical datasets.

In parallel to the increasing ubiquity of latent variable models and deep learning techniques in information retrieval and recommender systems, the authors propose a series of methodological advancements over existing VAE-based models. Central to these improvements is the introduction of a novel encoder architecture that borrows from state-of-the-art neural network designs. Furthermore, the RecVAE model integrates a composite prior distribution for the latent representation, dynamic rescaling of the $\beta$ hyperparameter in the $\beta$-VAE framework, and a sophisticated training procedure involving alternating updates between the encoder and decoder.

Key Methodological Contributions

1. Composite Prior:

RecVAE features a composite prior, a blend of a Gaussian prior and the approximate posterior from a former iteration. This innovation draws inspiration from reinforcement learning, particularly techniques used to avoid the phenomenon of "catastrophic forgetting." The formulation significantly enhances training stability, as evidenced by the restrained variability in resultant NDCG scores across epochs.

2. $\beta$ Rescaling:

An adaptive scheme for setting the $\beta$ hyperparameter is another pivotal contribution. Unlike previous works, this approach considers the variability in the amount of implicit feedback data available for each user. Empirically, this modification has yielded superior results, indicating that personalized weighting of the regularization term is crucial in coping with uneven data distribution.

3. Alternating Updates:

To address complexities arising from asymmetries between encoder and decoder networks, the paper introduces an innovative training scheme. By allowing multiple updates of the encoder for a single decoder update, this method capitalizes on the differences in network complexity, thereby optimizing learning trajectories. Furthermore, they cleverly separate denoising from decoder training, ensuring the training data integrity during updates.

Theoretical Implications

The RecVAE model not only provides practitioners with a practical tool that outperforms prior VAE-based algorithms but also opens new theoretical avenues. The idea of employing mixtures in prior distributions for VAEs and personalized adjustments to the ELBO’s regularization term could stimulate additional research around disentangled representations and their application in personalized systems.

Practical Implications

On the practical front, the RecVAE model demonstrates marked improvements in Recall@20, Recall@50, and NDCG@100 across large-scale datasets like MovieLens-20M and the Netflix Prize Dataset. Its robust performance points to direct applicability in real-world recommendation systems, particularly in domains where feedback is typically sparse and implicit.

The compositional and methodological developments presented in RecVAE showcase a successful fusion of theoretical exploration and empirical validation, proffering a model that is practically superior and theoretically insightful.

Looking ahead, the trajectory set forth by RecVAE will likely incite further inquiries into nuanced prior modeling and, potentially, the development of hybrid models that amalgamate state-of-the-art innovations across different domains.