Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions (2105.03248v2)

Abstract: We develop simple methods for constructing parameter priors for model choice among Directed Acyclic Graphical (DAG) models. In particular, we introduce several assumptions that permit the construction of parameter priors for a large number of DAG models from a small set of assessments. We then present a method for directly computing the marginal likelihood of every DAG model given a random sample with no missing observations. We apply this methodology to Gaussian DAG models which consist of a recursive set of linear regression models. We show that the only parameter prior for complete Gaussian DAG models that satisfies our assumptions is the normal-Wishart distribution. Our analysis is based on the following new characterization of the Wishart distribution: let $W$ be an $n \times n$, $n \ge 3$, positive-definite symmetric matrix of random variables and $f(W)$ be a pdf of $W$. Then, f$(W)$ is a Wishart distribution if and only if $W_{11} - W_{12} W_{22}^{-1} W'{12}$ is independent of ${W{12},W_{22}}$ for every block partitioning $W_{11},W_{12}, W'{12}, W{22}$ of $W$. Similar characterizations of the normal and normal-Wishart distributions are provided as well.

• The paper introduces a systematic method for assigning parameter priors to Gaussian DAG models, reducing computational challenges through modular likelihood and prior structures.
• It characterizes key probability distributions, including normal, Wishart, and normal-Wishart, under global parameter independence to enhance Bayesian model selection.
• The approach enables direct calculation of marginal likelihoods from complete data, streamlining the evaluation process of candidate DAG models.

Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions

This paper provides a mathematical framework for constructing parameter priors in the context of Directed Acyclic Graphical (DAG) models, with specific applicability to Gaussian DAG models. The authors, Geiger and Heckerman, develop a robust methodology, introducing several foundational assumptions to derive parameter priors from a limited number of assessments. This is particularly valuable given the exponential growth of possible DAG models with the increase in variables.

A pivotal aspect of the paper is a method for directly calculating the marginal likelihood of each DAG model, assuming complete observation data. This facilitates Bayesian model selection, where computational intensity burgeons alongside the structure of potential models. The authors confine their examination primarily to Gaussian DAG models, identifying that under complete assumptions, the normal-Wishart distribution emerges as the exclusive parameter prior for these models.

Key contributions of this paper include:

1. Methodology for Parameter Priors: The authors present a systematic approach to assign parameter priors to DAG models, leveraging complete model equivalence and modularity of likelihoods and priors. This approach is presumed to simplify the computational challenges in processing vast candidate DAG models through limited direct assessments.
2. Characterization of Distributions: The paper introduces or confirms characterizations of several distributions: Wishart, normal, and normal-Wishart, based on global parameter independence. The authors demonstrate that for Gaussian DAG models, these characterizations necessitate the use of a normal-Wishart distribution for its priors.
3. Computation of Marginal Likelihoods: The authors develop and optimize formulas for calculating marginal likelihoods for DAG models utilizing complete data. This involves an intricate transformation between different parametric spaces and the standardized forms of the normal and Wishart distributions.
4. Theoretical Implications: Through detailed theorems, the paper defines conditions under which certain probability distributions remain invariant or equivalently modeled across various transformations of the underlying graphical structure.

The findings hold significant implications both practically and theoretically. For practitioners in the field of statistics, artificial intelligence, and machine learning focusing on probabilistic graphical models, these advancements propose a streamlined methodology for prior specification, offering potentially enhanced model selection and learning processes. Theoretically, the characterizations of distribution forms advance understanding of DAG model behaviors and their corresponding statistical properties under Bayesian frameworks.

Speculatively, the results presented in this paper set a path for future investigations into other probability distributions that might satisfy the outlined assumptions, extending beyond Gaussian models. Furthermore, given the constraints identified with global parameter independence, subsequent research might delve into hierarchical or flexible prior structures to maintain computational efficiency without overly restrictive assumptions.

This paper's contribution lies in unifying the prior formation process across different types of graphical models, thereby providing a comprehensive methodology applicable in diverse modeling environments. Future developments in this area might further explore decomposition and approximation methods to broaden the applicability of these findings under less idealized assumptions.