Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Optimal Transport for Structure Learning Under Missing Data (2402.15255v2)

Published 23 Feb 2024 in cs.LG and cs.AI

Abstract: Causal discovery in the presence of missing data introduces a chicken-and-egg dilemma. While the goal is to recover the true causal structure, robust imputation requires considering the dependencies or, preferably, causal relations among variables. Merely filling in missing values with existing imputation methods and subsequently applying structure learning on the complete data is empirically shown to be sub-optimal. To address this problem, we propose a score-based algorithm for learning causal structures from missing data based on optimal transport. This optimal transport viewpoint diverges from existing score-based approaches that are dominantly based on expectation maximization. We formulate structure learning as a density fitting problem, where the goal is to find the causal model that induces a distribution of minimum Wasserstein distance with the observed data distribution. Our framework is shown to recover the true causal graphs more effectively than competing methods in most simulations and real-data settings. Empirical evidence also shows the superior scalability of our approach, along with the flexibility to incorporate any off-the-shelf causal discovery methods for complete data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (86)
  1. Learning bayesian networks with incomplete data by augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
  2. Learning causal graphs via monotone triangular transport maps. arXiv preprint arXiv:2305.18210, 2023.
  3. Amos, B. et al. Tutorial on amortized optimization. Foundations and Trends® in Machine Learning, 16(5):592–732, 2023.
  4. Wasserstein generative adversarial networks. In International conference on machine learning, pp.  214–223. PMLR, 2017.
  5. Causal reasoning in the presence of latent confounders via neural admg learning. In The Eleventh International Conference on Learning Representations, 2022.
  6. On minimum kantorovich distance estimators. Statistics & probability letters, 76(12):1298–1302, 2006.
  7. Dagma: Learning dags via m-matrices and a log-determinant acyclicity characterization. Advances in Neural Information Processing Systems, 35:8226–8239, 2022.
  8. On parameter estimation with the wasserstein distance. Information and Inference: A Journal of the IMA, 8(4):657–676, 2019.
  9. A unified wasserstein distributional robustness framework for adversarial training. In International Conference on Learning Representations, 2021.
  10. Learning probability measures with respect to optimal transport metrics. Advances in Neural Information Processing Systems, 25, 2012.
  11. Differentiable dag sampling. arXiv preprint arXiv:2203.08509, 2022.
  12. Chickering, D. M. Learning bayesian networks is np-complete. Learning from data: Artificial intelligence and statistics V, pp.  121–130, 1996.
  13. Chickering, D. M. Optimal structure identification with greedy search. Journal of machine learning research, 3(Nov):507–554, 2002.
  14. Large-sample learning of bayesian networks is np-hard. Journal of Machine Learning Research, 5:1287–1330, 2004.
  15. Polyhedral aspects of score equivalence in bayesian network structure learning. Mathematical Programming, 164:285–324, 2017.
  16. Multiple imputation via generative adversarial network for high-dimensional blockwise missing value problems. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), pp.  791–798. IEEE, 2021.
  17. Maximum Likelihood from Incomplete Data Via the EM Algorithm . Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1–22, 1977. doi: 10.1111/j.2517-6161.1977.tb01600.x.
  18. Optimizing notears objectives via topological swaps. In International Conference on Machine Learning, pp.  7563–7595. PMLR, 2023.
  19. Fragmgan: generative adversarial nets for fragmentary data imputation and prediction. Statistical Theory and Related Fields, pp.  1–14, 2023.
  20. Friedman, N. et al. Learning belief networks in the presence of missing values and hidden variables. In Icml, volume 97, pp.  125–133. Berkeley, CA, 1997.
  21. Structure learning under missing data. In International conference on probabilistic graphical models, pp.  121–132. PMLR, 2018.
  22. Missdag: Causal discovery in the presence of missing data with continuous additive noise models. Advances in Neural Information Processing Systems, 35:5024–5038, 2022.
  23. Handling missing data via max-entropy regularized graph autoencoder. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.  7651–7659, 2023.
  24. Deep end-to-end causal inference. arXiv preprint arXiv:2202.02195, 2022.
  25. Review of causal discovery methods based on graphical models. Frontiers in genetics, 10:524, 2019.
  26. Dream4: Combining genetic and dynamic information to identify biological networks and dynamical models. PloS one, 5(10):e13397, 2010.
  27. A bayesian approach to causal discovery. Innovations in Machine Learning: Theory and Applications, pp.  1–28, 2006.
  28. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21, 2008.
  29. Hyperimpute: Generalized iterative imputation with automatic model selection. In International Conference on Machine Learning, pp.  9916–9937. PMLR, 2022.
  30. Kantorovich, L. V. On a problem of monge. Journal of Mathematical Sciences, 133(4):1383–1383, 2006.
  31. Miracle: Causally-aware imputation via learning missing data mechanisms. Advances in Neural Information Processing Systems, 34:23806–23817, 2021.
  32. Gradient-based neural dag learning. In International Conference on Learning Representations, 2019.
  33. Statistical analysis with missing data, volume 793. John Wiley & Sons, 2019.
  34. On the stationary distribution of iterative imputations. Biometrika, 101(1):155–173, 2014.
  35. Dibs: Differentiable bayesian structure learning. Advances in Neural Information Processing Systems, 34:24111–24123, 2021.
  36. Amortized inference for causal structure learning. Advances in Neural Information Processing Systems, 35:13104–13118, 2022.
  37. Revealing strengths and weaknesses of methods for gene network inference. Proceedings of the national academy of sciences, 107(14):6286–6291, 2010.
  38. Miwae: Deep generative modelling and imputation of incomplete data sets. In International conference on machine learning, pp.  4413–4423. PMLR, 2019.
  39. Graphical models for processing missing data. Journal of the American Statistical Association, 116(534):1023–1037, 2021.
  40. Simultaneous missing value imputation and structure learning with groups. Advances in Neural Information Processing Systems, 35:20011–20024, 2022.
  41. Missing data imputation using optimal transport. In International Conference on Machine Learning, pp.  7130–7140. PMLR, 2020.
  42. Handling incomplete heterogeneous data using vaes. Pattern Recognition, 107:107501, 2020.
  43. On the role of sparsity and dag constraints for learning linear dags. Advances in Neural Information Processing Systems, 33:17943–17954, 2020.
  44. Masked gradient-based causal structure learning. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pp.  424–432. SIAM, 2022.
  45. Cycle class consistency with distributional optimal transport and knowledge distillation for unsupervised domain adaptation. In Uncertainty in Artificial Intelligence, pp.  1519–1529. PMLR, 2022.
  46. Finding optimal gene networks using biological constraints. Genome Informatics, 14:124–133, 2003.
  47. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics, pp.  1595–1605. PMLR, 2020.
  48. Pearl, J. Causality. Cambridge university press, 2009.
  49. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830, 2011.
  50. Missing data imputation and acquisition with deep hierarchical models and hamiltonian monte carlo. Advances in Neural Information Processing Systems, 35:35839–35851, 2022.
  51. Causal discovery with continuous additive noise models. 2014.
  52. Computational optimal transport. Center for Research in Economics and Statistics Working Papers, (2017-86), 2017.
  53. Mcflow: Monte carlo flow models for data imputation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  14205–14214, 2020.
  54. Improving the accuracy of medical diagnosis with causal machine learning. Nature communications, 11(1):3923, 2020.
  55. Nonparametric sparsity and regularization. 2013.
  56. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308(5721):523–529, 2005.
  57. Santambrogio, F. Optimal transport for applied mathematicians. Birkäuser, NY, 55(58-63):94, 2015.
  58. Singh, M. Learning bayesian networks from incomplete data. AAAI/IAAI, 1001:534–539, 1997.
  59. An algorithm for fast recovery of sparse causal graphs. Social science computer review, 9(1):62–72, 1991.
  60. Causation, prediction, and search. MIT press, 2000.
  61. Fast causal inference with non-random missingness by test-wise deletion. International journal of data science and analytics, 6:47–62, 2018.
  62. Ordering-based search: A simple and effective algorithm for learning bayesian networks. arXiv preprint arXiv:1207.1429, 2012.
  63. Wasserstein auto-encoders. arXiv preprint arXiv:1711.01558, 2017.
  64. Causal discovery in the presence of missing data. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.  1762–1770. PMLR, 2019.
  65. Optimal transport for causal discovery. In International Conference on Learning Representations, 2021.
  66. mice: Multivariate imputation by chained equations in r. Journal of statistical software, 45:1–67, 2011.
  67. Multivariate imputation by chained equations, 2000.
  68. Fully conditional specification in multivariate imputation. Journal of statistical computation and simulation, 76(12):1049–1064, 2006.
  69. Villani, C. et al. Optimal transport: old and new, volume 338. Springer, 2009.
  70. Learning directed graphical models with optimal transport. arXiv preprint arXiv:2305.15927, 2023.
  71. Vector quantized wasserstein auto-encoder. arXiv preprint arXiv:2302.05917, 2023.
  72. Ordering-based causal discovery with reinforcement learning. arXiv preprint arXiv:2105.06631, 2021.
  73. Causal inference for recommender systems. In Proceedings of the 14th ACM Conference on Recommender Systems, pp.  426–431, 2020.
  74. Gain: Missing data imputation using generative adversarial nets. In International conference on machine learning, pp.  5689–5698. PMLR, 2018.
  75. Gamin: Generative adversarial multiple imputation network for highly missing data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  8456–8464, 2020.
  76. Handling missing data with graph representation learning. Advances in Neural Information Processing Systems, 33:19075–19087, 2020.
  77. Dag-gnn: Dag structure learning with graph neural networks. In International Conference on Machine Learning, pp.  7154–7163. PMLR, 2019.
  78. Learning optimal bayesian networks using a* search. In Twenty-second international joint conference on artificial intelligence, 2011.
  79. Integrated systems approach identifies genetic nodes and networks in late-onset alzheimer’s disease. Cell, 153(3):707–720, 2013.
  80. Deepemd: Differentiable earth mover’s distance for few-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5632–5648, 2022.
  81. Neural topic model via optimal transport. In International Conference on Learning Representations, 2020.
  82. Transformed distribution matching for missing value imputation. In International Conference on Machine Learning, pp.  42159–42186. PMLR, 2023.
  83. Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems, 31, 2018.
  84. Learning sparse nonparametric dags. In International Conference on Artificial Intelligence and Statistics, pp.  3414–3425. PMLR, 2020.
  85. Convergence properties of a sequential regression multiple imputation algorithm. Journal of the American Statistical Association, 110(511):1112–1124, 2015.
  86. Causal discovery with reinforcement learning. arXiv preprint arXiv:1906.04477, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Vy Vo (12 papers)
  2. He Zhao (117 papers)
  3. Trung Le (94 papers)
  4. Edwin V. Bonilla (33 papers)
  5. Dinh Phung (147 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now