2000 character limit reached
Zero-Sum Games between Large-Population Teams: Reachability-based Analysis under Mean-Field Sharing (2303.12243v3)
Published 22 Mar 2023 in eess.SY and cs.SY
Abstract: This work studies the behaviors of two large-population teams competing in a discrete environment. The team-level interactions are modeled as a zero-sum game while the agent dynamics within each team is formulated as a collaborative mean-field team problem. Drawing inspiration from the mean-field literature, we first approximate the large-population team game with its infinite-population limit. Subsequently, we construct a fictitious centralized system and transform the infinite-population game to an equivalent zero-sum game between two coordinators. We study the optimal coordination strategies for each team via a novel reachability analysis and later translate them back to decentralized strategies that the original agents deploy. We prove that the strategies are $\epsilon$-optimal for the original finite-population team game, and we further show that the suboptimality diminishes when team size approaches infinity. The theoretical guarantees are verified by numerical examples.
- Decentralized optimal control of Markov chains with a common past information set. IEEE Transactions on Automatic Control, 32(11):1028–1031.
- Team optimal control of coupled subsystems with mean-field sharing. In 53rd IEEE Conference on Decision and Control, pages 1669–1674.
- Team-optimal solution of finite number of mean-field coupled LQG subsystems. In 54th IEEE Conference on Decision and Control, pages 5308–5313, Osaka, Japan, Dec. 15–18, 2015.
- The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research, 27(4):819–840.
- Neuro-dynamic Programming. Athena Scientific.
- Chung, K. L. (2001). A Course in Probability Theory. Academic press.
- Approximately solving mean field games via entropy-regularized deep reinforcement learning. In International Conference on Artificial Intelligence and Statistics, pages 1909–1917. PMLR.
- The Existence of Value in Differential Games, volume 126. American Mathematical Soc.
- Robust Nonlinear Control Design: State-space and Lyapunov Techniques. Springer Science & Business Media.
- The Handbook of Organizational Economics. Princeton University Press Princeton, NJ.
- On the adversarial convex body chasing problem. In 2023 American Control Conference, pages 435–440.
- Shaping large population agent behaviors through entropy-regularized mean-field games. In 2022 American Control Conference (ACC), pages 4429–4435. IEEE.
- Ho, Y.-C. (1980). Team decision theory and information structures. Proceedings of the IEEE, 68(6):644–654.
- Zero-sum games involving teams against teams: existence of equilibria, and comparison and regularity in information. Systems & Control Letters, 172:105454.
- Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized ϵitalic-ϵ\epsilonitalic_ϵ-Nash equilibria. IEEE Transactions on Automatic Control, 52(9):1560–1571.
- Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the nash certainty equivalence principle. Communications in Information & Systems, 6(3):221–252.
- Common information belief based dynamic programs for stochastic zero-sum games with competing teams. arXiv preprint arXiv:2102.05838.
- Kuroiwa, D. (1996). Convexity for set-valued maps. Applied Mathematics Letters, 9(2):97–101.
- Learning in zero-sum team Markov games using factored value functions. Advances in Neural Information Processing Systems, 15.
- Mean field games. Japanese Journal of Mathematics, 2(1):229–260.
- Learning mean field games: A survey. arXiv preprint arXiv:2205.12944.
- Lifelong multi-agent path finding in large-scale warehouses. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11272–11281.
- Mahajan, A. (2011). Optimal decentralized control of coupled subsystems with control sharing. In 50th IEEE Conference on Decision and Control and European Control Conference, pages 5726–5731, Orlando, FL, Dec. 12–15, 2011.
- Information structures in optimal decentralized control. In 51st IEEE Conference on Decision and Control, pages 1291–1306, Maui, HW, Dec. 10–13, 2012. IEEE.
- Sufficient statistics for linear control strategies in decentralized systems with partial history sharing. IEEE Transactions on Automatic Control, 60(8):2046–2056.
- Marschak, J. (1955). Elements for a theory of teams. Management Science, 1(2):127–137.
- Decentralized stochastic control with partial history sharing: A common information approach. IEEE Transactions on Automatic Control, 58(7):1644–1658.
- Owen, G. (2013). Game Theory. Emerald Group Publishing.
- Intractable problems in control theory. In 24th IEEE Conference on Decision and Control, pages 1099–1103, Fort Lauderdale, FL, Dec. 11–13, 1985.
- Radner, R. (1962). Team decision problems. The Annals of Mathematical Statistics, 33(3):857–881.
- Nash equilibria for exchangeable team against team games and their mean field limit. In 2023 American Control Conference, pages 1104–1109. IEEE.
- Search and rescue under the forest canopy using multiple UAVs. The International Journal of Robotics Research, 39(10-11):1201–1221.
- Autonomous teammates for squad tactics. In International Conference on Unmanned Aircraft Systems, pages 1667–1672, Athens, Greece, Sept. 1–4, 2020.
- Witsenhausen, H. S. (1971). Separation of estimation and control for discrete-time systems. Proceedings of the IEEE, 59(11):1557–1566.
- Yüksel, S. (2009). Stochastic nestedness and the belief sharing information pattern. IEEE Transactions on Automatic Control, 54(12):2773–2786.
- Dynamic potential games with constraints: Fundamentals and applications in communications. IEEE Transactions on Signal Processing, 64(14):3806–3821.