ExKMC: Expanding Explainable $k$-Means Clustering (2006.02399v2)

Abstract: Despite the popularity of explainable AI, there is limited work on effective methods for unsupervised learning. We study algorithms for $k$-means clustering, focusing on a trade-off between explainability and accuracy. Following prior work, we use a small decision tree to partition a dataset into $k$ clusters. This enables us to explain each cluster assignment by a short sequence of single-feature thresholds. While larger trees produce more accurate clusterings, they also require more complex explanations. To allow flexibility, we develop a new explainable $k$-means clustering algorithm, ExKMC, that takes an additional parameter $k' \geq k$ and outputs a decision tree with $k'$ leaves. We use a new surrogate cost to efficiently expand the tree and to label the leaves with one of $k$ clusters. We prove that as $k'$ increases, the surrogate cost is non-increasing, and hence, we trade explainability for accuracy. Empirically, we validate that ExKMC produces a low cost clustering, outperforming both standard decision tree methods and other algorithms for explainable clustering. Implementation of ExKMC available at https://github.com/navefr/ExKMC.

• The paper introduces a surrogate cost function that efficiently expands decision trees to minimize clustering loss.
• The paper proves competitive performance with an O(k²) approximation and validates the method across diverse datasets.
• The paper pioneers merging decision trees with k-means to balance scalability and transparency in unsupervised learning.

Expanding Explainable $k$-Means Clustering: A Review of ExKMC

The paper "ExKMC: Expanding Explainable $k$-Means Clustering" introduces an innovative algorithm, ExKMC, which addresses the challenge of achieving a trade-off between explainability and accuracy in $k$-means clustering. This topic is essential in the context of explainable AI, especially for unsupervised learning methodologies, which have historically lacked robust explainability models compared to their supervised counterparts.

Overview

The authors extend previous approaches by introducing a decision tree-based mechanism to partition datasets into $k$ clusters. This method allows for explaining cluster assignments using simple feature thresholds, thus enhancing interpretability. Unlike traditional $k$-means methods, where cluster assignments can be opaque, the proposed approach bridges this gap by employing decision trees with a user-defined number of leaves, $k’ \geq k$, allowing for flexible control over the complexity of explanations.

Key Contributions

1. Surrogate Cost Function: A novel surrogate cost function is central to ExKMC. This function, based on fixed reference centers derived from a standard $k$-means algorithm, enables efficient expansion of decision trees by minimizing a proxy for the $k$-means cost. This ensures that trees can be expanded without dynamically adjusting cluster centers, significantly reducing computational overhead.
2. Performance Guarantees: The authors prove that the surrogate cost is non-increasing as the tree grows—each additional leaf refines the clustering. The theoretical analysis shows that ExKMC can achieve competitive approximation ratios, specifically an $O(k^2)$ approximation compared to the optimal $k$-means cost when initialized with IMM (Iterative Mistake Minimization) trees.
3. Empirical Validation: Extensive experiments across a variety of datasets, including both synthetic and real-world data, demonstrate the effectiveness of ExKMC. The method frequently produces lower-cost clusterings compared to existing algorithms like CUBT and CLTree, and it closely approaches the cost of non-explainable $k$-means implementations, particularly as the number of leaves increases.
4. Scalability: The ExKMC algorithm, with optimizations like dynamic programming for efficient threshold selection, can process large datasets swiftly, making it a practical alternative in real-world applications.

Implications and Future Directions

The introduction of ExKMC presents significant implications for the development of explainable clustering algorithms. By combining decision tree methods with clustering, ExKMC retains high interpretability without greatly sacrificing clustering accuracy. For practitioners, this offers a means to leverage $k$-means clustering in environments where model transparency is critical.

The work also opens avenues for further research, including:

• Theoretical Frameworks: Developing robust theoretical models under various assumptions (e.g., Gaussian mixtures) to predict convergence behaviors or improve upon the current approximation guarantees.
• Fair and Bias-aware Clustering: Extending ExKMC to consider fairness constraints, ensuring that the clustering process is not only transparent but also equitable across different demographic groups.
• Optimizations and Extensions: Investigating techniques to enhance computational efficiency through parallelization or adapt the model to handle streaming data or dynamic datasets.

In conclusion, ExKMC represents an important step towards reconciling the objectives of accuracy and explainability in unsupervised learning, suggesting promising directions for both theoretical exploration and practical application in the field of machine learning.