Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Predictive Coding beyond Correlations (2306.15479v2)

Published 27 Jun 2023 in cs.LG

Abstract: Recently, there has been extensive research on the capabilities of biologically plausible algorithms. In this work, we show how one of such algorithms, called predictive coding, is able to perform causal inference tasks. First, we show how a simple change in the inference process of predictive coding enables to compute interventions without the need to mutilate or redefine a causal graph. Then, we explore applications in cases where the graph is unknown, and has to be inferred from observational data. Empirically, we show how such findings can be used to improve the performance of predictive coding in image classification tasks, and conclude that such models are able to perform simple end-to-end causal inference tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. DAGMA: Learning DAGs via M-matrices and a log-determinant acyclicity characterization. In Advances in Neural Information Processing Systems, 2022.
  2. Occam’s razor. Information Processing Letters, 24(6):377–380, 1987.
  3. Robust graph representation learning via predictive coding. arXiv:2212.04656, 2022.
  4. Interventional and counterfactual inference with diffusion models. arXiv:2302.00860, 2023.
  5. A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning, 2020.
  6. D. M. Chickering. Learning Bayesian networks is NP-complete. Learning from Data: Artificial Intelligence and Statistics V, pages 121–130, 1996.
  7. D. M. Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3(Nov):507–554, 2002.
  8. A. Clark. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3):181–204, 2013.
  9. E. De Brouwer. Deep counterfactual estimation with categorical background variables. arXiv:2210.05811, 2022.
  10. K. Friston. Learning and inference in the brain. Neural Networks, 16(9):1325–1352, 2003.
  11. K. Friston. A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1456), 2005.
  12. K. Friston. Hierarchical models in the brain. PLoS Computational Biology, 2008.
  13. K. Friston. The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2):127–138, 2010.
  14. K. Friston. The history of the future of the Bayesian brain. NeuroImage, 62(2):1230–1233, 2012.
  15. Variational free energy and the Laplace approximation. Neuroimage, 2007.
  16. Deep end-to-end causal inference. arXiv:2202.02195, 2022a.
  17. Deep end-to-end causal inference. arXiv:2202.02195, 2022b.
  18. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012.
  19. Deep predictive coding network with local recurrent processing for object recognition. Advances in Neural Information Processing Systems, 31, 2018.
  20. Causal structure learning. Annual Review of Statistics and Its Application, 5:371–391, 2018.
  21. M. Kalisch and P. Bühlman. Estimating high-dimensional directed acyclic graphs with the PC-algorithm. Journal of Machine Learning Research, 8(3), 2007.
  22. Algorithmic recourse under imperfect causal knowledge: A probabilistic approach. Advances in Neural Information Processing Systems, 33:265–277, 2020.
  23. Learning the difference that makes a difference with counterfactually-augmented data. arXiv:1909.12434, 2019.
  24. Causal autoregressive flows. In International Conference on Artificial Intelligence and Statistics, pages 3520–3528. PMLR, 2021.
  25. D. C. Knill and A. Pouget. The Bayesian brain: The role of uncertainty in neural coding and computation. TRENDS in Neurosciences, 27(12):712–719, 2004.
  26. Counterfactual fairness. Advances in Neural Information Processing Systems, 30, 2017.
  27. Discovering causal signals in images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6979–6987, 2017.
  28. Predictive coding approximates backprop along arbitrary computation graphs. arXiv:2006.04182, 2020.
  29. Predictive coding: A theoretical and experimental review, 2021.
  30. Simultaneous missing value imputation and structure learning with groups. Advances in Neural Information Processing Systems, 35:20011–20024, 2022.
  31. A. Ororbia and D. Kifer. The neural coding framework for learning generative models. Nature Communications, 13(1):2064, 2022.
  32. Lifelong neural predictive coding: Learning cumulatively online without forgetting. Advances in Neural Information Processing Systems, 35:5867–5881, 2022.
  33. A. G. Ororbia and A. Mali. Biologically motivated algorithms for propagating local target representations. In Proc. AAAI, volume 33, pages 4651–4658, 2019.
  34. Deep structural causal models for tractable counterfactual inference. Advances in Neural Information Processing Systems, 33:857–869, 2020a.
  35. Deep structural causal models for tractable counterfactual inference. Advances in Neural Information Processing Systems, 33:857–869, 2020b.
  36. J. Pearl. Bayesian networks: A model self-activated memory for evidential reasoning. In Proceedings of the 7th Conference of the Cognitive Science Society, University of California, Irvine, CA, USA, pages 15–17, 1985.
  37. J. Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669–688, 1995.
  38. J. Pearl. Causality. Cambridge University Press, 2009.
  39. Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press, 2017.
  40. Predictive coding beyond Gaussian distributions. In Advances in Neural Information Processing Systems, volume 35, 2022.
  41. Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79–87, 1999.
  42. Inferring causation from time series in earth system sciences. Nature Communications, 10(1):2553, 2019.
  43. S. Saha and U. Garain. On noise abduction for answering counterfactual queries: A practical outlook. Transactions on Machine Learning Research, 2022. ISSN 2835-8856.
  44. Associative memories via predictive coding. In Advances in Neural Information Processing Systems, volume 34, 2021.
  45. Learning on arbitrary graph topologies via predictive coding. arXiv:2201.13180, 2022a.
  46. Reverse differentiation via predictive coding. In Proc. AAAI, 2022b.
  47. Incremental predictive coding: A parallel and fully automatic learning algorithm. arXiv:2212.00720, 2022c.
  48. P. Sanchez and S. A. Tsaftaris. Diffusion causal models for counterfactual estimation. arXiv:2202.10166, 2022a.
  49. P. Sanchez and S. A. Tsaftaris. Diffusion causal models for counterfactual estimation. In Conference on Causal Learning and Reasoning, pages 647–668. PMLR, 2022b.
  50. Vaca: Designing variational graph autoencoders for causal queries. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8159–8168, 2022.
  51. A. K. Seth. The cybernetic Bayesian brain. In Open Mind. Open MIND. Frankfurt am Main: MIND Group, 2014.
  52. Estimating individual treatment effect: Generalization bounds and algorithms. In International Conference on Machine Learning, pages 3076–3085. PMLR, 2017.
  53. A. Sharma and E. Kiciman. DoWhy: An end-to-end library for causal inference. arXiv:2011.04216, 2020.
  54. A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
  55. Can the brain do backpropagation? — Exact implementation of backpropagation in predictive coding networks. In Advances in Neural Information Processing Systems, volume 33, 2020.
  56. Inferring neural activity before plasticity: A foundation for learning beyond backpropagation. bioRxiv, pages 2022–05, 2022.
  57. Causation, Prediction, and Search. MIT Press, 2000.
  58. Recurrent predictive coding models for associative memory employing covariance learning. PLOS Computational Biology, 19(4):e1010719, 2023.
  59. J. C. Whittington and R. Bogacz. An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity. Neural Computation, 29(5), 2017.
  60. J. Yoo and F. Wood. BayesPCN: A continually learnable predictive coding associative memory. Advances in Neural Information Processing Systems, 35:29903–29914, 2022.
  61. DAG-GNN: DAG structure learning with graph neural networks. In International Conference on Machine Learning, pages 7154–7163. PMLR, 2019.
  62. DAGs with no tears: Continuous optimization for structure learning. Advances in Neural Information Processing Systems, 31, 2018.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now