Sparse Coding and Dictionary Learning for Symmetric Positive Definite Matrices: A Kernel Approach (1304.4344v1)

Abstract: Recent advances suggest that a wide range of computer vision problems can be addressed more appropriately by considering non-Euclidean geometry. This paper tackles the problem of sparse coding and dictionary learning in the space of symmetric positive definite matrices, which form a Riemannian manifold. With the aid of the recently introduced Stein kernel (related to a symmetric version of Bregman matrix divergence), we propose to perform sparse coding by embedding Riemannian manifolds into reproducing kernel Hilbert spaces. This leads to a convex and kernel version of the Lasso problem, which can be solved efficiently. We furthermore propose an algorithm for learning a Riemannian dictionary (used for sparse coding), closely tied to the Stein kernel. Experiments on several classification tasks (face recognition, texture classification, person re-identification) show that the proposed sparse coding approach achieves notable improvements in discrimination accuracy, in comparison to state-of-the-art methods such as tensor sparse coding, Riemannian locality preserving projection, and symmetry-driven accumulation of local features.

• The paper introduces a novel kernel-based approach using the Stein kernel to perform sparse coding and dictionary learning on symmetric positive definite (SPD) matrices.
• The proposed method, Riemannian Sparse Representation (RSR), formulates the problem as a convex Lasso-like optimization and achieves superior accuracy on visual classification tasks like face recognition and texture analysis.
• The research demonstrates the practical efficacy of integrating Riemannian geometry with kernel methods and extends the approach to dictionary learning on SPD manifolds.

Sparse Coding and Dictionary Learning for Symmetric Positive Definite Matrices: A Kernel Approach

This paper addresses the complex task of sparse coding and dictionary learning for symmetric positive definite (SPD) matrices, a problem of significant importance in the areas of computer vision and machine learning. The authors, Harandi et al., introduce a novel methodology leveraging a kernel-based approach to address these tasks in the context of Riemannian geometry, utilizing the Stein kernel. The proposed method leads to a convex formulation akin to the Lasso problem, which is efficiently solvable and exhibits superior performance across several visual classification tasks.

The core contribution of the paper is the embedding of Riemannian manifolds into reproducing kernel Hilbert spaces (RKHS) using the Stein kernel. This strategy enables a kernelised version of sparse coding, addressing the inherent challenges posed by the non-linear nature of computations on Riemannian manifolds. Notably, this approach facilitates a tighter integration with the Riemannian structure, allowing the proposed method, referred to as Riemannian Sparse Representation (RSR), to achieve discrimination accuracies superior to existing techniques such as tensor sparse coding and other Riemannian geometry-based methods.

Numerical Results and Key Insights

The empirical evaluation is thorough, encompassing synthetic data and real-world datasets used in face recognition, texture classification, and person re-identification tasks. The numerical results unequivocally demonstrate the efficacy of the RSR method:

• Synthetic Data: The proposed method attains a significant improvement in recognition accuracy over competitors such as tensor sparse coding (TSC) and log-Euclidean sparse representation (logE-SR). This performance advantage is achieved with considerably lower computational cost compared to TSC.
• Face Recognition: Using the FERET dataset, the method achieves impressive gains in accuracy, particularly when distinguishing between non-frontal face images, underscoring its robustness to variations in pose and illumination.
• Texture Classification and Person Re-identification: Consistently higher accuracy is reported for RSR across several complicated scenarios, again validating its superiority over state-of-the-art techniques.

Dictionary Learning in Riemannian Manifolds

The paper also extends its contributions to the domain of dictionary learning on SPD manifolds. The authors propose an iterative learning algorithm, using the Stein kernel, that efficiently updates the dictionary in RKHS. When evaluated against Riemannian k-means, the proposed algorithm exhibits lower representation error and better supports subsequent classification tasks.

Practical and Theoretical Implications

Practically, this research provides a robust framework for addressing tasks that naturally reside on SPD manifolds, such as those found in computer vision and medical imaging. Theoretically, it advances our understanding of incorporating Riemannian geometry into machine learning algorithms via kernel methods. The work suggests that the use of the Stein kernel is not only effective for sparse coding but could also be extended to other machine learning paradigms within tensor spaces.

Future Directions

Looking ahead, the integration of the Stein kernel into more complex models, such as large-margin classifiers on Riemannian manifolds, presents an intriguing avenue. This could potentially extend the reach and efficacy of RSR techniques to broader applications, thereby enriching both the theoretical constructs and practical implementations of machine learning on Riemannian spaces.

In conclusion, this paper makes substantial contributions to the field by marrying the expressive power of sparse representation with the nuanced understanding of Riemannian geometry, all through an efficient kernel-based approach. The results and insights presented provide a solid foundation for further exploration and development in this rich area of research.