Decentralize and Randomize: Faster Algorithm for Wasserstein Barycenters (1806.03915v3)

Abstract: We study the decentralized distributed computation of discrete approximations for the regularized Wasserstein barycenter of a finite set of continuous probability measures distributedly stored over a network. We assume there is a network of agents/machines/computers, and each agent holds a private continuous probability measure and seeks to compute the barycenter of all the measures in the network by getting samples from its local measure and exchanging information with its neighbors. Motivated by this problem, we develop, and analyze, a novel accelerated primal-dual stochastic gradient method for general stochastic convex optimization problems with linear equality constraints. Then, we apply this method to the decentralized distributed optimization setting to obtain a new algorithm for the distributed semi-discrete regularized Wasserstein barycenter problem. Moreover, we show explicit non-asymptotic complexity for the proposed algorithm.

• The paper presents an accelerated primal-dual stochastic gradient method that efficiently computes decentralized Wasserstein barycenters.
• It leverages a regularized approach to solve stochastic convex optimization with linear constraints in distributed network settings.
• Experimental validations on Gaussian and von Mises distributions demonstrate the method's scalability and practical applicability.

Decentralize and Randomize: Faster Algorithm for Wasserstein Barycenters

The paper introduces a decentralized distributed algorithm for computing regularized Wasserstein barycenters over networks. This research addresses the efficient computation of barycenters from a finite set of continuous probability measures distributed across agents in a network, leveraging decentralized communication to obtain the barycenter. The key contribution is the development of an accelerated primal-dual stochastic gradient method that solves stochastic convex optimization problems with linear equality constraints.

Key Contributions

1. Accelerated Primal-Dual Stochastic Gradient Method (APDSGM): The authors propose a novel algorithm specifically tailored for stochastic optimization problems with linear constraints. This method is particularly beneficial in distributed network settings where data is distributed, and communication overhead needs to be minimized.
2. Decentralized Regularized Wasserstein Barycenter Computation: The paper applies the APDSGM to compute a discrete approximation of Wasserstein barycenters in networks with unknown arbitrary topology. This approach is vital for large-scale distributed systems where each agent only has local data access and limited communication capabilities.
3. Iterative Complexity Analysis: The authors provide explicit, non-asymptotic iteration complexity for their proposed algorithm. This theoretical foundation helps researchers understand the efficiency and potential scalability of the method in practical applications.
4. Experimental Validations: Demonstrations using univariate Gaussian and von Mises distributions, as well as applications in image aggregation, showcase the algorithm's effectiveness in real-world scenarios. The experiments highlight the advantage of the proposed method in graph topologies like Erdős-Rényi and cycle graphs.

Implications and Future Directions

The implications of this research are profound for fields requiring distributed computation of barycenters. Practically, this method is applicable in areas such as image processing, economics, and machine learning, where data may be distributed over many sensors or nodes. Theoretically, the algorithm expands on existing works in convex optimization, offering a viable solution to compute barycenters efficiently even with decentralized data sources.

Future work could explore extensions to dynamic or time-varying networks, which present challenges due to their continuously evolving nature. Additionally, integrating the algorithm with communication-efficient strategies could further enhance its applicability in bandwidth-constrained environments. While the current work provides promising results, adapting the algorithm to various graph types beyond static and connected graphs remains an unresolved topic worth investigating.

In summation, the paper presents a significant advancement in the field of decentralized optimization, specifically for computing Wasserstein barycenters. Its contributions lay a foundation for future research to build upon, potentially influencing a broad range of distributed computing applications.