Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Approximate Implication for Probabilistic Graphical Models (2310.13942v1)

Published 21 Oct 2023 in cs.AI, cs.IT, and math.IT

Abstract: The graphical structure of Probabilistic Graphical Models (PGMs) represents the conditional independence (CI) relations that hold in the modeled distribution. Every separator in the graph represents a conditional independence relation in the distribution, making them the vehicle through which new conditional independencies are inferred and verified. The notion of separation in graphs depends on whether the graph is directed (i.e., a Bayesian Network), or undirected (i.e., a Markov Network). The premise of all current systems-of-inference for deriving CIs in PGMs, is that the set of CIs used for the construction of the PGM hold exactly. In practice, algorithms for extracting the structure of PGMs from data discover approximate CIs that do not hold exactly in the distribution. In this paper, we ask how the error in this set propagates to the inferred CIs read off the graphical structure. More precisely, what guarantee can we provide on the inferred CI when the set of CIs that entailed it hold only approximately? It has recently been shown that in the general case, no such guarantee can be provided. In this work, we prove new negative and positive results concerning this problem. We prove that separators in undirected PGMs do not necessarily represent approximate CIs. That is, no guarantee can be provided for CIs inferred from the structure of undirected graphs. We prove that such a guarantee exists for the set of CIs inferred in directed graphical models, making the $d$-separation algorithm a sound and complete system for inferring approximate CIs. We also establish improved approximation guarantees for independence relations derived from marginal and saturated CIs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Decomposition and functional dependencies in relations. ACM Trans. Database Syst., 5(4):404–430, 1980. doi:10.1145/320610.320620.
  2. A complete axiomatization for functional and multivalued dependencies in database relations. In Proceedings of the 1977 ACM SIGMOD International Conference on Management of Data, Toronto, Canada, August 3-5, 1977., pages 47–61, 1977. doi:10.1145/509404.509414.
  3. Improving bayesian network structure learning with mutual information-based node ordering in the k2 algorithm. IEEE Transactions on Knowledge and Data Engineering, 20(5):628–640, 2008. doi:10.1109/TKDE.2007.190732.
  4. Learning bayesian networks from data: An information-theory based approach. Artificial Intelligence, 137(1):43 – 90, 2002. URL: http://www.sciencedirect.com/science/article/pii/S0004370202001911, doi:https://doi.org/10.1016/S0004-3702(02)00191-1.
  5. A. P. Dawid. Conditional independence in statistical theory. Journal of the Royal Statistical Society. Series B (Methodological), 41(1):1–31, 1979. URL: http://www.jstor.org/stable/2984718.
  6. Luis M. de Campos. A scoring function for learning bayesian networks based on mutual information and conditional independence tests. Journal of Machine Learning Research, 7(77):2149–2187, 2006. URL: http://jmlr.org/papers/v7/decampos06a.html.
  7. Axioms and algorithms for inferences involving probabilistic independence. Inf. Comput., 91(1):128–141, 1991. doi:10.1016/0890-5401(91)90077-F.
  8. Axioms and algorithms for inferences involving probabilistic independence. Information and Computation, 91(1):128 – 141, 1991. URL: http://www.sciencedirect.com/science/article/pii/089054019190077F, doi:https://doi.org/10.1016/0890-5401(91)90077-F.
  9. On the logic of causal models. In UAI ’88: Proceedings of the Fourth Annual Conference on Uncertainty in Artificial Intelligence, Minneapolis, MN, USA, July 10-12, 1988, pages 3–14, 1988. URL: https://dslpitt.org/uai/displayArticleDetails.jsp?mmnu=1&smnu=2&article_id=1831&proceeding_id=1004.
  10. Logical and algorithmic properties of conditional independence and graphical models. The Annals of Statistics, 21(4):2001–2021, 1993. URL: http://www.jstor.org/stable/2242326.
  11. d-separation: From theorems to algorithms. In Max Henrion, Ross D. Shachter, Laveen N. Kanal, and John F. Lemmer, editors, UAI ’89: Proceedings of the Fifth Annual Conference on Uncertainty in Artificial Intelligence, Windsor, Ontario, Canada, August 18-20, 1989, pages 139–148. North-Holland, 1989. URL: https://dslpitt.org/uai/displayArticleDetails.jsp?mmnu=1&smnu=2&article_id=1872&proceeding_id=1005.
  12. Identifying independence in bayesian networks. Networks, 20(5):507–534, 1990. doi:10.1002/net.3230200504.
  13. On the completeness of the semigraphoid axioms for deriving arbitrary from saturated conditional independence statements. Inf. Process. Lett., 114(11):628–633, 2014. doi:10.1016/j.ipl.2014.05.010.
  14. C. Herrmann. On the undecidability of implications between embedded multivalued database dependencies. Inf. Comput., 122(2):221–235, November 1995. doi:10.1006/inco.1995.1148.
  15. Batya Kenig. Approximate implication with d-separation. In Cassio P. de Campos, Marloes H. Maathuis, and Erik Quaeghebeur, editors, Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, UAI 2021, Virtual Event, 27-30 July 2021, volume 161 of Proceedings of Machine Learning Research, pages 301–311. AUAI Press, 2021. URL: https://proceedings.mlr.press/v161/kenig21a.html.
  16. Mining approximate acyclic schemes from relations. In David Maier, Rachel Pottinger, AnHai Doan, Wang-Chiew Tan, Abdussalam Alawini, and Hung Q. Ngo, editors, Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14-19, 2020, pages 297–312. ACM, 2020. doi:10.1145/3318464.3380573.
  17. Integrity constraints revisited: From exact to approximate implication. In Carsten Lutz and Jean Christoph Jung, editors, 23rd International Conference on Database Theory, ICDT 2020, March 30-April 2, 2020, Copenhagen, Denmark, volume 155 of LIPIcs, pages 18:1–18:20. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. doi:10.4230/LIPIcs.ICDT.2020.18.
  18. Integrity constraints revisited: From exact to approximate implication. Log. Methods Comput. Sci., 18(1), 2022. doi:10.46298/lmcs-18(1:5)2022.
  19. Probabilistic Graphical Models - Principles and Techniques. MIT Press, 2009. URL: http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=11886.
  20. Independence in database relations. In Leonid Libkin, Ulrich Kohlenbach, and Ruy de Queiroz, editors, Logic, Language, Information, and Computation, pages 179–193, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg.
  21. Tony T. Lee. An information-theoretic analysis of relational databases - part I: data dependencies and information metric. IEEE Trans. Software Eng., 13(10):1049–1061, 1987. doi:10.1109/TSE.1987.232847.
  22. David Maier. Theory of Relational Databases. Computer Science Pr, 1983.
  23. Judea Pearl. Probabilistic reasoning in intelligent systems - networks of plausible inference. Morgan Kaufmann series in representation and reasoning. Morgan Kaufmann, 1989.
  24. Conditional independence and its representations. Kybernetika, 25(7):33–44, 1989. URL: http://www.kybernetika.cz/content/1989/7/33.
  25. Graphoids: Graph-based logic for reasoning about relevance relations or when would x tell you more about y if you already know z? In ECAI, pages 357–363, 1986.
  26. The implication problem for measure-based constraints. Information Systems, 33(2):221 – 239, 2008. Performance Evaluation of Data Management Systems. URL: http://www.sciencedirect.com/science/article/pii/S0306437907000531, doi:https://doi.org/10.1016/j.is.2007.07.005.
  27. Milan Studený. Conditional independence relations have no finite complete characterization. In 11th Prague Conf. Information Theory, Statistical Decision Foundation and Random Processes, pages 377–396. Norwell, MA, 1990.
  28. Milan Studený. Conditional independence and markov properties for basic graphs. In Marloes Maathuis, Mathias Drton, Steffen Lauritzen, and Martin Wainwright, editors, Handbook of Graphical Models, pages 3–38. CRC Press, November 2018.
  29. Causal networks: semantics and expressiveness. In Ross D. Shachter, Tod S. Levitt, Laveen N. Kanal, and John F. Lemmer, editors, UAI ’88: Proceedings of the Fourth Annual Conference on Uncertainty in Artificial Intelligence, Minneapolis, MN, USA, July 10-12, 1988, pages 69–78. North-Holland, 1988.
  30. Causal networks: Semantics and expressiveness. In Ross D. SHACHTER, Tod S. LEVITT, Laveen N. KANAL, and John F. LEMMER, editors, Uncertainty in Artificial Intelligence, volume 9 of Machine Intelligence and Pattern Recognition, pages 69–76. North-Holland, 1990. URL: https://www.sciencedirect.com/science/article/pii/B9780444886507500111, doi:https://doi.org/10.1016/B978-0-444-88650-7.50011-1.
  31. Raymond W. Yeung. A new outlook of shannon’s information measures. IEEE Trans. Information Theory, 37(3):466–474, 1991. doi:10.1109/18.79902.
  32. Raymond W. Yeung. Information Theory and Network Coding. Springer Publishing Company, Incorporated, 1 edition, 2008.
  33. Part mutual information for quantifying direct associations in networks. Proceedings of the National Academy of Sciences, 113(18):5130–5135, 2016. URL: https://www.pnas.org/content/113/18/5130, arXiv:https://www.pnas.org/content/113/18/5130.full.pdf, doi:10.1073/pnas.1522586113.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now