STGFormer: Spatio-Temporal GraphFormer for 3D Human Pose Estimation in Video (2407.10099v2)

Abstract: The current methods of video-based 3D human pose estimation have achieved significant progress.However, they still face pressing challenges, such as the underutilization of spatiotemporal bodystructure features in transformers and the inadequate granularity of spatiotemporal interaction modeling in graph convolutional networks, which leads to pervasive depth ambiguity in monocular 3D human pose estimation. To address these limitations, this paper presents the Spatio-Temporal GraphFormer framework (STGFormer) for 3D human pose estimation in videos. First, we introduce a Spatio-Temporal criss-cross Graph (STG) attention mechanism, designed to more effectively leverage the inherent graph priors of the human body within continuous sequence distributions while capturing spatiotemporal long-range dependencies. Next, we present a dual-path Modulated Hop-wise Regular GCN (MHR-GCN) to independently process temporal and spatial dimensions in parallel, preserving features rich in temporal dynamics and the original or high-dimensional representations of spatial structures. Furthermore, the module leverages modulation to optimize parameter efficiency and incorporates spatiotemporal hop-wise skip connections to capture higher-order information. Finally, we demonstrate that our method achieves state-of-the-art performance on the Human3.6M and MPIINF-3DHP datasets.