Gaussian Belief Propagation: Theory and Aplication

The canonical problem of solving a system of linear equations arises in numerous contexts in information theory, communication theory, and related fields. In this contribution, we develop a solution based upon Gaussian belief propagation (GaBP) that does not involve direct matrix inversion. The iterative nature of our approach allows for a distributed message-passing implementation of the solution algorithm. In the first part of this thesis, we address the properties of the GaBP solver. We characterize the rate of convergence, enhance its message-passing efficiency by introducing a broadcast version, discuss its relation to classical solution methods including numerical examples. We present a new method for forcing the GaBP algorithm to converge to the correct solution for arbitrary column dependent matrices. In the second part we give five applications to illustrate the applicability of the GaBP algorithm to very large computer networks: Peer-to-Peer rating, linear detection, distributed computation of support vector regression, efficient computation of Kalman filter and distributed linear programming. Using extensive simulations on up to 1,024 CPUs in parallel using IBM Bluegene supercomputer we demonstrate the attractiveness and applicability of the GaBP algorithm, using real network topologies with up to millions of nodes and hundreds of millions of communication links. We further relate to several other algorithms and explore their connection to the GaBP algorithm.