Gauss-Seidel Method with Oblique Direction

In this paper, a Gauss-Seidel method with oblique direction (GSO) is proposed for finding the least-squares solution to a system of linear equations, where the coefficient matrix may be full rank or rank deficient and the system is overdetermined or underdetermined. Through this method, the number of iteration steps and running time can be reduced to a greater extent to find the least-squares solution, especially when the columns of matrix A are close to linear correlation. It is theoretically proved that GSO method converges to the least-squares solution. At the same time, a randomized version--randomized Gauss-Seidel method with oblique direction (RGSO) is established, and its convergence is proved. Theoretical proof and numerical results show that the GSO method and the RGSO method are more efficient than the coordinate descent (CD) method and the randomized coordinate descent (RCD) method.