A cutting plane algorithm for globally solving low dimensional k-means clustering problems (2402.13595v1)
Abstract: Clustering is one of the most fundamental tools in data science and machine learning, and k-means clustering is one of the most common such methods. There is a variety of approximate algorithms for the k-means problem, but computing the globally optimal solution is in general NP-hard. In this paper we consider the k-means problem for instances with low dimensional data and formulate it as a structured concave assignment problem. This allows us to exploit the low dimensional structure and solve the problem to global optimality within reasonable time for large data sets with several clusters. The method builds on iteratively solving a small concave problem and a large linear programming problem. This gives a sequence of feasible solutions along with bounds which we show converges to zero optimality gap. The paper combines methods from global optimization theory to accelerate the procedure, and we provide numerical results on their performance.
- James MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281–297. Oakland, CA, USA, 1967.
- Stuart Lloyd. Least squares quantization in PCM. IEEE transactions on information theory, 28(2):129–137, 1982.
- K-means++ the advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pages 1027–1035, 2007.
- A local search approximation algorithm for k-means clustering. In Proceedings of the eighteenth annual symposium on Computational geometry, pages 10–18, 2002.
- David Sculley. Web-scale k-means clustering. In Proceedings of the 19th international conference on World wide web, pages 1177–1178, 2010.
- Charles Elkan. Using the triangle inequality to accelerate k-means. In Proceedings of the 20th international conference on Machine Learning (ICML-03), pages 147–153, 2003.
- Adaptive sampling for k-means clustering. In International Workshop on Approximation Algorithms for Combinatorial Optimization, pages 15–28. Springer, 2009.
- Jiří Matoušek. On approximate geometric k-clustering. Discrete & Computational Geometry, 24(1):61–84, 2000.
- A better k-means++ algorithm via local search. In International Conference on Machine Learning, pages 3662–3671. PMLR, 2019.
- Better guarantees for k-means and euclidean k-median by primal-dual algorithms. SIAM Journal on Computing, 49(4):FOCS17–97, 2019.
- Improved and simplified inapproximability for k-means. Information Processing Letters, 120:40–43, 2017.
- Robert L Thorndike. Who belongs in the family? Psychometrika, 18(4):267–276, 1953.
- Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53–65, 1987.
- Convex optimization. Cambridge university press, 2004.
- Globally solving the Gromov-Wasserstein problem for point clouds in low dimensional euclidean spaces. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
- Toby Walsh. General symmetry breaking constraints. In International Conference on Principles and Practice of Constraint Programming, pages 650–664. Springer, 2006.
- Polytopes associated with symmetry handling. Mathematical Programming, 175(1-2):197–240, 2019.
- Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2023.
- Pseudo basic steps: bound improvement guarantees from lagrangian decomposition in convex disjunctive programming. EURO Journal on Computational Optimization, 6:55–83, 2018.
- Between steps: Intermediate relaxations between big-m and convex hull formulations. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pages 299–314. Springer, 2021.
- Yann LeCun. The MNIST database of handwritten digits. http://yann. lecun. com/exdb/mnist/, 1998.
- Doubly stochastic normalization of the Gaussian kernel is robust to heteroskedastic noise. SIAM journal on mathematics of data science, 3(1):388–413, 2021.
- T. Kaijser. Computing the Kantorovich distance for images. Journal of Mathematical Imaging and Vision, 9(2):173–191, 1998.