HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative (2207.13921v3)

Abstract: AI-based protein structure prediction pipelines, such as AlphaFold2, have achieved near-experimental accuracy. These advanced pipelines mainly rely on Multiple Sequence Alignments (MSAs) as inputs to learn the co-evolution information from the homologous sequences. Nonetheless, searching MSAs from protein databases is time-consuming, usually taking dozens of minutes. Consequently, we attempt to explore the limits of fast protein structure prediction by using only primary sequences of proteins. HelixFold-Single is proposed to combine a large-scale protein LLM with the superior geometric learning capability of AlphaFold2. Our proposed method, HelixFold-Single, first pre-trains a large-scale protein LLM (PLM) with thousands of millions of primary sequences utilizing the self-supervised learning paradigm, which will be used as an alternative to MSAs for learning the co-evolution information. Then, by combining the pre-trained PLM and the essential components of AlphaFold2, we obtain an end-to-end differentiable model to predict the 3D coordinates of atoms from only the primary sequence. HelixFold-Single is validated in datasets CASP14 and CAMEO, achieving competitive accuracy with the MSA-based methods on the targets with large homologous families. Furthermore, HelixFold-Single consumes much less time than the mainstream pipelines for protein structure prediction, demonstrating its potential in tasks requiring many predictions. The code of HelixFold-Single is available at https://github.com/PaddlePaddle/PaddleHelix/tree/dev/apps/protein_folding/helixfold-single, and we also provide stable web services on https://paddlehelix.baidu.com/app/drug/protein-single/forecast.

• The paper demonstrates a novel PLM-based method that bypasses MSAs by integrating transformer models with geometric techniques.
• The paper achieves competitive accuracy on CASP14 and CAMEO datasets while significantly reducing computation time compared to traditional methods.
• The paper suggests that scaling PLM architectures could further enhance predictions, minimizing reliance on extensive homologous sequence data.

HelixFold-Single: MSA-Free Protein Structure Prediction Using Protein LLMs

HelixFold-Single introduces a novel approach to protein structure prediction that bypasses the traditional reliance on Multiple Sequence Alignments (MSAs). By leveraging the capabilities of large-scale protein LLMs (PLMs) as an alternative, HelixFold-Single seeks to enhance both the efficiency and accuracy of protein structure prediction.

Methodology

The HelixFold-Single framework integrates a PLM with geometric modeling techniques inspired by AlphaFold2. The process begins with pre-training a PLM on a vast dataset of protein sequences using self-supervised learning tasks like masked LLMing (MLM). This PLM encodes co-evolutionary information traditionally captured by MSAs.

The model architecture consists of:

1. PLM Base: A transformer-based model capturing sequence relationships, including long-range dependencies, vital for structural predictions.
2. Geometric Modeling: Encompassing components from AlphaFold2, the model processes sequence and pair representations to predict atomic coordinates in 3D space.
3. Adaptor Module: Facilitates the integration of PLM outputs into the geometric modeling framework.

Training involves a two-stage optimization process. Initially, the PLM is trained with a masked language task on a diversified protein dataset. Subsequently, the model is fine-tuned using a dataset of experimentally determined and computationally generated structures, focusing on learning end-to-end differentiable structure predictions.

Results

HelixFold-Single's efficacy was evaluated against established methods, including AlphaFold2, on datasets such as CASP14 and CAMEO. Remarkably, HelixFold-Single performs comparably to MSA-dependent models, achieving high accuracy, particularly for proteins with abundant homologous sequences. This performance demonstrates that the PLM effectively embeds co-evolutionary information critical for accurate structure prediction.

Key findings include:

• Superior performance on targets with large homologous families, showing potential to match MSA-based models.
• Significant reduction in computation time, making it well-suited for high-throughput applications.
• Case studies illustrating enhanced prediction accuracy for specific protein structures where AlphaFold2 struggles.

Implications and Future Directions

The success of HelixFold-Single underlines the viability of PLMs as robust alternatives to traditional MSA-based methods. By significantly reducing time overhead associated with MSA searches, HelixFold-Single has broad applicability in various protein-related tasks.

Looking forward, further scaling of PLM architectures may enhance predictive capabilities, particularly for proteins with limited homologous sequences. Additionally, the integration of more diverse and extensive datasets could address current limitations in modeling orphan proteins. This approach implies a shift towards more efficient computational strategies, potentially revolutionizing protein engineering and drug design efforts.

HelixFold-Single showcases how marrying advanced natural language processing techniques with protein studies can streamline intricate biological computations, paving the way for further innovation in the field of structural bioinformatics.