Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Robust Constrained Consensus and Inequality-constrained Distributed Optimization with Guaranteed Differential Privacy and Accurate Convergence (2404.02327v1)

Published 2 Apr 2024 in math.OC, cs.SY, and eess.SY

Abstract: We address differential privacy for fully distributed optimization subject to a shared inequality constraint. By co-designing the distributed optimization mechanism and the differential-privacy noise injection mechanism, we propose the first distributed constrained optimization algorithm that can ensure both provable convergence to a global optimal solution and rigorous $\epsilon$-differential privacy, even when the number of iterations tends to infinity. Our approach does not require the Lagrangian function to be strictly convex/concave, and allows the global objective function to be non-separable. As a byproduct of the co-design, we also propose a new constrained consensus algorithm that can achieve rigorous $\epsilon$-differential privacy while maintaining accurate convergence, which, to our knowledge, has not been achieved before. Numerical simulation results on a demand response control problem in smart grid confirm the effectiveness of the proposed approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. T.-H. Chang, A. Nedić, and A. Scaglione, “Distributed constrained optimization by consensus-based primal-dual perturbation method,” IEEE Transactions on Automatic Control, vol. 59, no. 6, pp. 1524–1538, 2014.
  2. I. Notarnicola and G. Notarstefano, “Constraint-coupled distributed optimization: A relaxation and duality approach,” IEEE Transactions on Control of Network Systems, vol. 7, no. 1, pp. 483–492, 2019.
  3. A. Patrascu and I. Necoara, “On the convergence of inexact projection primal first-order methods for convex minimization,” IEEE Transactions on Automatic Control, vol. 63, no. 10, pp. 3317–3329, 2018.
  4. D. E. Hershberger and H. Kargupta, “Distributed multivariate regression using wavelet-based collective data mining,” Journal of Parallel and Distributed Computing, vol. 61, no. 3, pp. 372–400, 2001.
  5. G. Notarstefano, I. Notarnicola, A. Camisa et al., “Distributed optimization for smart cyber-physical networks,” Foundations and Trends in Systems and Control, vol. 7, no. 3, pp. 253–383, 2019.
  6. B. Yang and M. Johansson, “Distributed optimization and games: A tutorial overview,” Networked Control Systems, pp. 109–148, 2010.
  7. Y. Su, Z. Wang, M. Cao, M. Jia, and F. Liu, “Convergence analysis of dual decomposition algorithm in distributed optimization: Asynchrony and inexactness,” IEEE Transactions on Automatic Control, 2022.
  8. M. Zhu and S. Martínez, “On distributed convex optimization under inequality and equality constraints,” IEEE Transactions on Automatic Control, vol. 57, no. 1, pp. 151–164, 2011.
  9. D. Yuan, S. Xu, and H. Zhao, “Distributed primal–dual subgradient method for multiagent optimization via consensus algorithms,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 41, no. 6, pp. 1715–1724, 2011.
  10. C. Zhang, M. Ahmad, and Y. Wang, “ADMM based privacy-preserving decentralized optimization,” IEEE Transactions on Information Forensics and Security, vol. 14, no. 3, pp. 565–580, 2019.
  11. Z. Huang, S. Mitra, and N. Vaidya, “Differentially private distributed optimization,” in Proceedings of the 2015 International Conference on Distributed Computing and Networking, 2015, pp. 1–10.
  12. D. A. Burbano-L, J. George, R. A. Freeman, and K. M. Lynch, “Inferring private information in wireless sensor networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2019, pp. 4310–4314.
  13. Y. Wang and T. Başar, “Quantization enabled privacy protection in decentralized stochastic optimization,” IEEE Transactions on Automatic Control, 2022.
  14. Y. Wang and A. Nedić, “Tailoring gradient methods for differentially-private distributed optimization,” IEEE Transactions on Automatic Control, 2023.
  15. Y. Wang and H. V. Poor, “Decentralized stochastic optimization with inherent privacy protection,” IEEE Transactions on Automatic Control, 2022.
  16. A. Falsone, K. Margellos, S. Garatti, and M. Prandini, “Dual decomposition for multi-agent distributed optimization with coupling constraints,” Automatica, vol. 84, pp. 149–158, 2017.
  17. K. Tjell and R. Wisniewski, “Privacy preservation in distributed optimization via dual decomposition and admm,” in 2019 IEEE 58th Conference on Decision and Control (CDC).   IEEE, 2019, pp. 7203–7208.
  18. D. Han, K. Liu, H. Sandberg, S. Chai, and Y. Xia, “Privacy-preserving dual averaging with arbitrary initial conditions for distributed optimization,” IEEE Transactions on Automatic Control, vol. 67, no. 6, pp. 3172–3179, 2021.
  19. C. Zhang and Y. Wang, “Enabling privacy-preservation in decentralized optimization,” IEEE Transactions on Control of Network Systems, vol. 6, no. 2, pp. 679–689, 2018.
  20. N. M. Freris and P. Patrinos, “Distributed computing over encrypted data,” in 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).   IEEE, 2016, pp. 1116–1122.
  21. Y. Lu and M. Zhu, “Privacy preserving distributed optimization using homomorphic encryption,” Automatica, vol. 96, pp. 314–325, 2018.
  22. Y. Lou, L. Yu, S. Wang, and P. Yi, “Privacy preservation in distributed subgradient optimization algorithms,” IEEE Transactions on Cybernetics, vol. 48, no. 7, pp. 2154–2165, 2017.
  23. S. Gade and N. H. Vaidya, “Private optimization on networks,” in American Control Conference.   IEEE, 2018, pp. 1402–1409.
  24. H. Gao, Y. Wang, and A. Nedić, “Dynamics based privacy preservation in decentralized optimization,” Automatica, vol. 151, p. 110878, 2023.
  25. Y. Wang and A. Nedić, “Decentralized gradient methods with time-varying uncoordinated stepsizes: Convergence analysis and privacy design,” IEEE Transactions on Automatic Control, 2022.
  26. C. Dwork, A. Roth et al., “The algorithmic foundations of differential privacy.” Foundations and Trends in Theoretical Computer Science, vol. 9, no. 3-4, pp. 211–407, 2014.
  27. S. Han, U. Topcu, and G. J. Pappas, “Differentially private distributed constrained optimization,” IEEE Transactions on Automatic Control, vol. 62, no. 1, pp. 50–64, 2016.
  28. M. T. Hale and M. Egerstedt, “Cloud-enabled differentially private multiagent optimization with constraints,” IEEE Transactions on Control of Network Systems, vol. 5, no. 4, pp. 1693–1706, 2017.
  29. Y. Wang, Z. Huang, S. Mitra, and G. E. Dullerud, “Differential privacy in linear distributed control systems: Entropy minimizing mechanisms and performance tradeoffs,” IEEE Transactions on Control of Network Systems, vol. 4, no. 1, pp. 118–130, 2017.
  30. X. Zhang, M. M. Khalili, and M. Liu, “Recycled ADMM: Improving the privacy and accuracy of distributed algorithms,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 1723–1734, 2019.
  31. J. He, L. Cai, and X. Guan, “Differential private noise adding mechanism and its application on consensus algorithm,” IEEE Transactions on Signal Processing, vol. 68, pp. 4069–4082, 2020.
  32. J. Cortés, G. E. Dullerud, S. Han, J. Le Ny, S. Mitra, and G. J. Pappas, “Differential privacy in control and network systems,” in IEEE 55th Conference on Decision and Control (CDC), 2016, pp. 4252–4272.
  33. Y. Xiong, J. Xu, K. You, J. Liu, and L. Wu, “Privacy preserving distributed online optimization over unbalanced digraphs via subgradient rescaling,” IEEE Transactions on Control of Network Systems, 2020.
  34. T. Ding, S. Zhu, J. He, C. Chen, and X.-P. Guan, “Differentially private distributed optimization via state and direction perturbation in multi-agent systems,” IEEE Transactions on Automatic Control, 2021.
  35. E. Nozari, P. Tallapragada, and J. Cortés, “Differentially private distributed convex optimization via functional perturbation,” IEEE Transactions on Control of Network Systems, vol. 5, no. 1, pp. 395–408, 2016.
  36. Z. Huang, S. Mitra, and G. Dullerud, “Differentially private iterative synchronous consensus,” in Proceedings of the 2012 ACM workshop on Privacy in the electronic society.   ACM, 2012, pp. 81–90.
  37. E. Nozari, P. Tallapragada, and J. Cortés., “Differentially private average consensus: obstructions, trade-offs, and optimal algorithm design,” Automatica, vol. 81, no. 7, pp. 221–231, 2017.
  38. Y. Wang, “A robust dynamic average consensus algorithm that ensures both differential privacy and accurate convergence,” in 62nd IEEE Conference on Decision and Control (CDC).   IEEE, 2023, pp. 1130–1137.
  39. M. Ye, G. Hu, L. Xie, and S. Xu, “Differentially private distributed Nash equilibrium seeking for aggregative games,” IEEE Transactions on Automatic Control, vol. 67, no. 5, pp. 2451–2458, 2021.
  40. J. Wang, J.-F. Zhang, and X. He, “Differentially private distributed algorithms for stochastic aggregative games,” Automatica, vol. 142, p. 110440, 2022.
  41. A. Camisa, F. Farina, I. Notarnicola, and G. Notarstefano, “Distributed constraint-coupled optimization over random time-varying graphs via primal decomposition and block subgradient approaches,” in 2019 IEEE 58th Conference on Decision and Control (CDC).   IEEE, 2019, pp. 6374–6379.
  42. I. Necoara and V. Nedelcu, “Rate analysis of inexact dual first-order methods application to dual decomposition,” IEEE Transactions on Automatic Control, vol. 59, no. 5, pp. 1232–1243, 2013.
  43. T.-H. Chang, A. Nedić, and A. Scaglione, “Distributed sparse regression by consensus-based primal-dual perturbation optimization,” in 2013 IEEE Global Conference on Signal and Information Processing.   IEEE, 2013, pp. 289–292.
  44. A. Nedić and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Transactions on Automatic Control, vol. 54, no. 1, pp. 48–61, 2009.
  45. S. Sundhar Ram, A. Nedić, and V. V. Veeravalli, “A new class of distributed optimization algorithms: Application to regression of distributed data,” Optimization Methods and Software, vol. 27, no. 1, pp. 71–88, 2012.
  46. H. Uzawa, “Iterative methods for concave programming,” Studies in Linear and Nonlinear Programming, vol. 6, pp. 154–165, 1958.
  47. M. Kallio and A. Ruszczynski, “Perturbation methods for saddle point computation,” IIASA WP-94-038, 1994.
  48. M. Kallio and C. H. Rosa, “Large-scale convex optimization via saddle point computation,” Operations Research, vol. 47, no. 1, pp. 93–101, 1999.
  49. B. Polyak, “Introduction to optimization,” Optimization Software Inc., Publications Division, New York, vol. 1, 1987.
  50. C. Dwork, M. Naor, T. Pitassi, and G. N. Rothblum, “Differential privacy under continual observation,” in Proceedings of the forty-second ACM Symposium on Theory of Computing, 2010, pp. 715–724.
  51. X. Chen, L. Huang, L. He, S. Dey, and L. Shi, “A differentially private method for distributed optimization in directed networks via state decomposition,” IEEE Transactions on Control of Network Systems, 2023.
  52. L. Huang, J. Wu, D. Shi, S. Dey, and L. Shi, “Differential privacy in distributed optimization with gradient tracking,” IEEE Transactions on Automatic Control, 2024.
  53. A. Nedić, A. Ozdaglar, and P. A. Parrilo, “Constrained consensus and optimization in multi-agent networks,” IEEE Transactions on Automatic Control, vol. 55, no. 4, pp. 922–938, 2010.
  54. K. L. Chung, “On a stochastic approximation method,” The Annals of Mathematical Statistics, pp. 463–483, 1954.
  55. G. Belgioioso, A. Nedić, and S. Grammatico, “Distributed generalized Nash equilibrium seeking in aggregative games on time-varying networks,” IEEE Transactions on Automatic Control, vol. 66, no. 5, pp. 2061–2075, 2021.
  56. A. Nedić and A. Ozdaglar, “Approximate primal solutions and rate analysis for dual subgradient methods,” SIAM Journal on Optimization, vol. 19, no. 4, pp. 1757–1780, 2009.
  57. N. Cai and J. Mitra, “A decentralized control architecture for a microgrid with power electronic interfaces,” in North American Power Symposium 2010.   IEEE, 2010, pp. 1–8.
  58. L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov, “Exploiting unintended feature leakage in collaborative learning,” in 2019 IEEE Symposium on Security and Privacy (SP).   IEEE, 2019, pp. 691–706.
  59. L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” in Advances in Neural Information Processing Systems, 2019, pp. 14 774–14 784.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now