Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 28 tok/s
Gemini 2.5 Pro 40 tok/s Pro
GPT-5 Medium 16 tok/s Pro
GPT-5 High 13 tok/s Pro
GPT-4o 103 tok/s Pro
Kimi K2 197 tok/s Pro
GPT OSS 120B 471 tok/s Pro
Claude Sonnet 4 38 tok/s Pro
2000 character limit reached

Autoencoder-based General Purpose Representation Learning for Customer Embedding (2402.18164v2)

Published 28 Feb 2024 in cs.LG and cs.AI

Abstract: Recent advances in representation learning have successfully leveraged the underlying domain-specific structure of data across various fields. However, representing diverse and complex entities stored in tabular format within a latent space remains challenging. In this paper, we introduce DEEPCAE, a novel method for calculating the regularization term for multi-layer contractive autoencoders (CAEs). Additionally, we formalize a general-purpose entity embedding framework and use it to empirically show that DEEPCAE outperforms all other tested autoencoder variants in both reconstruction performance and downstream prediction performance. Notably, when compared to a stacked CAE across 13 datasets, DEEPCAE achieves a 34% improvement in reconstruction error.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. A deep contractive autoencoder for solving multiclass classification problems. Evolutionary Intelligence, 14:1619–1633, 2021.
  2. Robust variational autoencoder for tabular data with beta divergence. CoRR, abs/2006.08204, 2020. URL https://arxiv.org/abs/2006.08204.
  3. Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  3762–3769, 2014.
  4. Neural networks and principal component analysis: Learning from examples without local minima. Neural Networks, 2(1):53–58, 1989. ISSN 0893-6080. doi: https://doi.org/10.1016/0893-6080(89)90014-2. URL https://www.sciencedirect.com/science/article/pii/0893608089900142.
  5. Understanding disentangling in β𝛽\betaitalic_β-vae, 2018.
  6. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912–9924, 2020.
  7. Customer lifetime value prediction using embeddings. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, pp.  1753–1762, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450348874. doi: 10.1145/3097983.3098123. URL https://doi.org/10.1145/3097983.3098123.
  8. Dynamic customer embeddings for financial service applications, 2021.
  9. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116, 2019.
  10. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  11. Dupont, E. Learning disentangled joint continuous and discrete representations. In Advances in Neural Information Processing Systems, pp. 707–717, 2018.
  12. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728, 2018.
  13. The dimensional fact model: A conceptual model for data warehouses. International Journal of Cooperative Information Systems, 7(02n03):215–247, 1998.
  14. On embeddings for numerical features in tabular deep learning. Advances in Neural Information Processing Systems, 35:24991–25004, 2022.
  15. On embeddings for numerical features in tabular deep learning, 2023.
  16. Deep residual learning for image recognition, 2015.
  17. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  9729–9738, 2020.
  18. Early visual concept learning with unsupervised deep learning, 2016.
  19. Tabtransformer: Tabular data modeling using contextual embeddings. arXiv preprint arXiv:2012.06678, 2020.
  20. Auto-encoding variational bayes, 2013.
  21. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
  22. dsprites: Disentanglement testing sprites dataset, 2017.
  23. Umap: Uniform manifold approximation and projection for dimension reduction, 2020.
  24. Disentangling generative factors in natural language with discrete variational autoencoders, 2021.
  25. Efficient estimation of word representations in vector space, 2013.
  26. Munson, M. A. A study on the importance of and time spent on different modeling steps. SIGKDD Explor. Newsl., 13(2):65–71, may 2012. ISSN 1931-0145. doi: 10.1145/2207243.2207253. URL https://doi.org/10.1145/2207243.2207253.
  27. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
  28. OpenAI. Gpt-4 technical report, 2023.
  29. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024–8035. Curran Associates, Inc., 2019. URL http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.
  30. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp.  1532–1543, 2014.
  31. Press, G. Data preparation: Most time-consuming, least enjoyable data science task, survey says. Forbes, Mar 2016. URL https://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/.
  32. Efficient learning of sparse representations with an energy-based model. In Schölkopf, B., Platt, J., and Hoffman, T. (eds.), Advances in Neural Information Processing Systems, volume 19. MIT Press, 2006. URL https://proceedings.neurips.cc/paper_files/paper/2006/file/87f4d79e36d68c3031ccf6c55e9bbd39-Paper.pdf.
  33. Higher order contractive auto-encoder. In Gunopulos, D., Hofmann, T., Malerba, D., and Vazirgiannis, M. (eds.), Machine Learning and Knowledge Discovery in Databases, pp. 645–660, Berlin, Heidelberg, 2011a. Springer Berlin Heidelberg.
  34. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedings of the 28th international conference on international conference on machine learning, pp.  833–840, 2011b.
  35. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3):1181–1191, 2020. ISSN 0169-2070. doi: https://doi.org/10.1016/j.ijforecast.2019.07.001. URL https://www.sciencedirect.com/science/article/pii/S0169207019301888.
  36. Syne Tune: A library for large scale hyperparameter tuning and reproducible research. In International Conference on Automated Machine Learning, AutoML 2022, 2022. URL https://proceedings.mlr.press/v188/salinas22a.html.
  37. Dptvae: Data-driven prior-based tabular variational autoencoder for credit data synthesizing. Expert Systems with Applications, 241:122071, 2024. ISSN 0957-4174. doi: https://doi.org/10.1016/j.eswa.2023.122071. URL https://www.sciencedirect.com/science/article/pii/S0957417423025733.
  38. Subtab: Subsetting features of tabular data for self-supervised representation learning. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp.  18853–18865. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/9c8661befae6dbcd08304dbf4dcaf0db-Paper.pdf.
  39. Causal disentangled variational auto-encoder for preference understanding in recommendation. arXiv preprint arXiv:2304.07922, 2023.
  40. Cloud intrusion detection method based on stacked contractive auto-encoder and support vector machine. IEEE transactions on cloud computing, 10(3):1634–1646, 2020.
  41. Pilots’ fatigue status recognition using deep contractive autoencoder network. IEEE Transactions on Instrumentation and Measurement, 68(10):3907–3919, 2019. doi: 10.1109/TIM.2018.2885608.
  42. Interpretation for variational autoencoder used to generate financial synthetic tabular data. Algorithms, 16(2), 2023. ISSN 1999-4893. doi: 10.3390/a16020121. URL https://www.mdpi.com/1999-4893/16/2/121.
  43. Image denoising and inpainting with deep neural networks. In Pereira, F., Burges, C., Bottou, L., and Weinberger, K. (eds.), Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. URL https://proceedings.neurips.cc/paper_files/paper/2012/file/6cdd60ea0045eb7a6ec44c54d29ed402-Paper.pdf.
  44. Causalvae: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  9593–9602, June 2021.
  45. Overview of artificial neural networks. Artificial neural networks: methods and applications, pp. 14–22, 2009.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-Up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now