Improved Rectangular Matrix Multiplication using Powers of the Coppersmith-Winograd Tensor (1708.05622v2)

Abstract: In the past few years, successive improvements of the asymptotic complexity of square matrix multiplication have been obtained by developing novel methods to analyze the powers of the Coppersmith-Winograd tensor, a basic construction introduced thirty years ago. In this paper we show how to generalize this approach to make progress on the complexity of rectangular matrix multiplication as well, by developing a framework to analyze powers of tensors in an asymmetric way. By applying this methodology to the fourth power of the Coppersmith-Winograd tensor, we succeed in improving the complexity of rectangular matrix multiplication. Let $\alpha$ denote the maximum value such that the product of an $n\times n^\alpha$ matrix by an $n^\alpha\times n$ matrix can be computed with $O(n^{2+\epsilon})$ arithmetic operations for any $\epsilon>0$. By analyzing the fourth power of the Coppersmith-Winograd tensor using our methods, we obtain the new lower bound $\alpha>0.31389$, which improves the previous lower bound $\alpha>0.30298$ obtained five years ago by Le Gall (FOCS'12) from the analysis of the second power of the Coppersmith-Winograd tensor. More generally, we give faster algorithms computing the product of an $n\times n^k$ matrix by an $n^k\times n$ matrix for any value $k\neq 1$. (In the case $k=1$, we recover the bounds recently obtained for square matrix multiplication). These improvements immediately lead to improvements in the complexity of a multitude of fundamental problems for which the bottleneck is rectangular matrix multiplication, such as computing the all-pair shortest paths in directed graphs with bounded weights.

• The paper introduces a novel approach that extends square matrix methods to rectangular matrices using powers of the Coppersmith-Winograd tensor.
• It improves the dual exponent lower bound from 0.30298 to 0.31389 and provides faster algorithms for multiplying n×nᵏ matrices for k ≠ 1.
• The improved bounds offer practical benefits for computational tasks such as all-pairs shortest paths and sparse matrix multiplication.

Improved Rectangular Matrix Multiplication Using Powers of the Coppersmith-Winograd Tensor

In the field of computational mathematics, optimizing matrix multiplication is a critical problem due to its wide application across scientific computing, data analysis, and machine learning. The paper "Improved Rectangular Matrix Multiplication Using Powers of the Coppersmith-Winograd Tensor" by Fran{\c c}ois Le Gall and Florent Urrutia addresses recent advances in the asymptotic complexity of matrix multiplication using the Coppersmith-Winograd tensor. While prior work focused primarily on square matrices, this paper extends those methods to rectangular matrix multiplication, revealing novel approaches and improved bounds.

Progress on Rectangular Matrix Multiplication Complexity

Matrix multiplication complexity is traditionally tied to the exponent, $\omega$, representing the minimal value such that two $n \times n$ matrices can be multiplied using $O(n^{\omega+\epsilon})$ operations for any $\epsilon>0$. Recent improvements have involved analyzing higher powers of the Coppersmith-Winograd tensor, originally introduced by Coppersmith and Winograd in 1990. This tensor has been foundational, providing an upper bound $\omega < 2.376$ for square matrix multiplication.

This paper advances these ideas by developing a framework that successfully generalizes the analysis to rectangular matrices. Le Gall and Urrutia improve the lower bound on the dual exponent, $\alpha$, from 0.30298 to 0.31389. This progress is made possible by analyzing the fourth power of the Coppersmith-Winograd tensor asymmetrically, a method not fully explored in previous works. Additionally, the paper furnishes faster algorithms for multiplying matrices of size $n \times n^k$ by $n^k \times n$ for any $k$ other than 1.

Implications of Improved Bounds

The results of this research carry significant implications for computational problems where rectangular matrix multiplication is a bottleneck. For instance, this includes the computation of all-pairs shortest paths in directed graphs and speed-ups for sparse matrix multiplication. Therefore, the improved bounds enable more efficient algorithms for these problems, expanding the applicability and efficiency of computational methods in various domains.

Future Directions

While the paper sets a new standard for analyzing rectangular matrix multiplication, it acknowledges hurdles in further refining these bounds using higher tensor powers, such as 64 or 128. The authors suggest the potential exploration of convex optimization methods as applied in the square matrix multiplication context, which may yield additional improvements. The pursuit of closing gaps further towards the conjecture $\omega = 2$ remains a compelling challenge.

To conclude, this paper marks a noteworthy contribution to the computational mathematics community by linking advanced theoretical techniques to tangible improvements in algorithmic complexity. The methodology adapted for rectangular matrices opens avenues for further research and implementation, pushing the boundaries of efficient computation. For researchers focused on matrix algorithms and complexity theory, this work provides a robust foundation for exploring new horizons and applications.