Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multi-Task Learning with Loop Specific Attention for CDR Structure Prediction (2306.13045v1)

Published 22 Jun 2023 in cs.LG

Abstract: The Complementarity Determining Region (CDR) structure prediction of loops in antibody engineering has gained a lot of attraction by researchers. When designing antibodies, a main challenge is to predict the CDR structure of the H3 loop. Compared with the other CDR loops, that is the H1 and H2 loops, the CDR structure of the H3 loop is more challenging due to its varying length and flexible structure. In this paper, we propose a Multi-task learning model with Loop Specific Attention, namely MLSA. In particular, to the best of our knowledge we are the first to jointly learn the three CDR loops, via a novel multi-task learning strategy. In addition, to account for the structural and functional similarities and differences of the three CDR loops, we propose a loop specific attention mechanism to control the influence of each CDR loop on the training of MLSA. Our experimental evaluation on widely used benchmark data shows that the proposed MLSA method significantly reduces the prediction error of the CDR structure of the H3 loop, by at least 19%, when compared with other baseline strategies. Finally, for reproduction purposes we make the implementation of MLSA publicly available at https://anonymous.4open.science/r/MLSA-2442/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. M. S. Maddur, S. Lacroix-Desmazes, J. D. Dimitrov, M. D. Kazatchkine, J. Bayry, and S. V. Kaveri, “Natural antibodies: from first-line defense against pathogens to perpetual immune homeostasis,” Clinical Reviews in Allergy & Immunology, vol. 58, pp. 213–228, 2020.
  2. J. Maynard and G. Georgiou, “Antibody engineering,” Annual review of biomedical engineering, vol. 2, no. 1, pp. 339–376, 2000.
  3. D. Kuroda, H. Shirai, M. P. Jacobson, and H. Nakamura, “Computer-aided antibody design,” Protein engineering, design & selection, vol. 25, no. 10, pp. 507–522, 2012.
  4. L. Chatenoud, “Treatment of autoimmune disease: Biological and molecular therapies,” in The Autoimmune Diseases, pp. 1221–1245, Elsevier, 2014.
  5. S. J. Kim, Y. Park, and H. J. Hong, “Antibody engineering for the development of therapeutic antibodies.,” Molecules & Cells (Springer Science & Business Media BV), vol. 20, no. 1, 2005.
  6. T. Fu and J. Sun, “Antibody complementarity determining regions (cdrs) design using constrained energy model,” in Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 389–399, 2022.
  7. R. L. Stanfield and I. A. Wilson, “Antibody structure,” Antibodies for Infectious Diseases, pp. 49–62, 2015.
  8. M. L. Chiu, D. R. Goulet, A. Teplyakov, and G. L. Gilliland, “Antibody structure and function: the basis for engineering therapeutics,” Antibodies, vol. 8, no. 4, p. 55, 2019.
  9. W.-L. Ling, W.-H. Lua, and S. K.-E. Gan, “Sagacity in antibody humanization for therapeutics, diagnostics and research purposes: considerations of antibody elements and their roles,” Antibody Therapeutics, vol. 3, no. 2, pp. 71–79, 2020.
  10. S. Saini and Y. Kumar, “Bispecific antibodies: A promising entrant in cancer immunotherapy,” in Translational Biotechnology, pp. 233–266, Elsevier, 2021.
  11. C. Chothia, A. M. Lesk, A. Tramontano, M. Levitt, S. J. Smith-Gill, G. Air, S. Sheriff, E. A. Padlan, D. Davies, W. R. Tulip, et al., “Conformations of immunoglobulin hypervariable regions,” Nature, vol. 342, no. 6252, pp. 877–883, 1989.
  12. B. North, A. Lehmann, and R. L. Dunbrack Jr, “A new clustering of antibody cdr loop conformations,” Journal of molecular biology, vol. 406, no. 2, pp. 228–256, 2011.
  13. H. Shirai, A. Kidera, and H. Nakamura, “Structural classification of cdr-h3 in antibodies,” FEBS letters, vol. 399, no. 1-2, pp. 1–8, 1996.
  14. J. Dunbar, A. Fuchs, J. Shi, and C. M. Deane, “Abangle: characterising the vh–vl orientation in antibodies,” Protein Engineering, Design & Selection, vol. 26, no. 10, pp. 611–620, 2013.
  15. J. C. Almagro, A. Teplyakov, J. Luo, R. W. Sweet, S. Kodangattil, F. Hernandez-Guzman, and G. L. Gilliland, “Second antibody modeling assessment (ama-ii),” 2014.
  16. S. Luo, Y. Su, X. Peng, S. Wang, J. Peng, and J. Ma, “Antigen-specific antibody design and optimization with diffusion-based generative models,” bioRxiv, pp. 2022–07, 2022.
  17. R. W. Shuai, J. A. Ruffolo, and J. J. Gray, “Generative language modeling for antibody design,” bioRxiv, pp. 2021–12, 2021.
  18. J. Li, R. Abel, K. Zhu, Y. Cao, S. Zhao, and R. A. Friesner, “The vsgb 2.0 model: a next generation energy model for high resolution protein structure modeling,” Proteins: Structure, Function, and Bioinformatics, vol. 79, no. 10, pp. 2794–2812, 2011.
  19. J. Mintseris, B. Pierce, K. Wiehe, R. Anderson, R. Chen, and Z. Weng, “Integrating statistical pair potentials into protein complex prediction,” Proteins: Structure, Function, and Bioinformatics, vol. 69, no. 3, pp. 511–520, 2007.
  20. X. Kong, W. Huang, and Y. Liu, “Conditional antibody design as 3d equivariant graph translation,” in Proceedings of the International Conference on Learning Representations, 2022.
  21. Z. Shui and G. Karypis, “Heterogeneous molecular graph neural networks for predicting molecule properties,” in Proceedings of the IEEE International Conference on Data Mining, pp. 492–500, 2020.
  22. L. A. Clark, S. Ganesan, S. Papp, and H. W. van Vlijmen, “Trends in antibody sequence changes during the somatic hypermutation process,” The Journal of Immunology, vol. 177, no. 1, pp. 333–340, 2006.
  23. S. Ruder, “An overview of multi-task learning in deep neural networks,” DBLP, 2017.
  24. R. Collobert and J. Weston, “A unified architecture for natural language processing: Deep neural networks with multitask learning,” in Proceedings of the International Conference on Machine Learning, pp. 160–167, 2008.
  25. P. Liu, X. Qiu, and X. Huang, “Deep multi-task learning with shared memory,” in Proceedings of the Empirical Methods in Natural Language Processing, 2016.
  26. L. Deng, G. Hinton, and B. Kingsbury, “New types of deep neural network learning for speech recognition and related applications: An overview,” in Proceedings of the IEEE international conference on acoustics, speech and signal processing, pp. 8599–8603, 2013.
  27. S. Kim, T. Hori, and S. Watanabe, “Joint ctc-attention based end-to-end speech recognition using multi-task learning,” in Proceedings of the IEEE international conference on acoustics, speech and signal processing, pp. 4835–4839, 2017.
  28. Y. Gao, J. Ma, M. Zhao, W. Liu, and A. L. Yuille, “Nddr-cnn: Layerwise feature fusing in multi-task cnns by neural discriminative dimensionality reduction,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3205–3214, 2019.
  29. R. Girshick, “Fast r-cnn,” in Proceedings of the IEEE international conference on computer vision, pp. 1440–1448, 2015.
  30. S. Lin, C. Shi, and J. Chen, “Generalizeddta: combining pre-training and multi-task learning to predict drug-target binding affinity for unknown drug discovery,” BMC bioinformatics, vol. 23, no. 1, pp. 1–17, 2022.
  31. S. Liu, M. Qu, Z. Zhang, H. Cai, and J. Tang, “Structured multi-task learning for molecular property prediction,” in Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 8906–8920, PMLR, 2022.
  32. W. Jin, J. Wohlwend, R. Barzilay, and T. Jaakkola, “Iterative refinement graph neural network for antibody sequence-structure co-design,” in Proceedings of the International Conference on Learning Representations, 2021.
  33. M. Bule, N. Jalalimanesh, Z. Bayrami, M. Baeeri, and M. Abdollahi, “The rise of deep learning and transformations in bioactivity prediction power of molecular modeling tools,” Chemical Biology & Drug Design, vol. 98, no. 5, pp. 954–967, 2021.
  34. J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, et al., “Highly accurate protein structure prediction with alphafold,” Nature, vol. 596, no. 7873, pp. 583–589, 2021.
  35. G. Arvanitidis, L. K. Hansen, and S. Hauberg, “Latent space oddity: on the curvature of deep generative models,” in Proceedings of the International Conference on Learning Representations, 2017.
  36. B. Lai, M. McPartlon, and J. Xu, “End-to-end deep structure generative model for protein design,” bioRxiv, pp. 2022–07, 2022.
  37. M. Liu, K. Yan, B. Oztekin, and S. Ji, “Graphebm: Molecular graph generation with energy-based models,” in Proceedings of the Energy-Based Models Workshop ICLR, 2021.
  38. M. Welling and Y. W. Teh, “Bayesian learning via stochastic gradient langevin dynamics,” in Proceedings of the international conference on machine learning, pp. 681–688, 2011.
  39. C. H. Norn, G. Lapidoth, and S. J. Fleishman, “High-accuracy modeling of antibody structures by a search for minimum-energy recombination of backbone fragments,” Proteins: Structure, Function, and Bioinformatics, vol. 85, no. 1, pp. 30–38, 2017.
  40. J. A. Ruffolo, J. Sulam, and J. J. Gray, “Antibody structure prediction using interpretable deep learning,” Patterns, vol. 3, no. 2, p. 100406, 2022.
  41. J. Ingraham, V. Garg, R. Barzilay, and T. Jaakkola, “Generative models for graph-based protein design,” Advances in neural information processing systems, vol. 32, 2019.
  42. J. A. Ruffolo, L.-S. Chu, S. P. Mahajan, and J. J. Gray, “Fast, accurate antibody structure prediction from deep learning on massive set of natural antibodies,” Nature communications, vol. 14, no. 1, p. 2389, 2023.
  43. B. Abanades, G. Georges, A. Bujotzek, and C. M. Deane, “Ablooper: fast accurate antibody cdr loop structure prediction with accuracy estimation,” Bioinformatics, vol. 38, no. 7, pp. 1877–1880, 2022.
  44. S. Kearnes, K. McCloskey, M. Berndl, V. Pande, and P. Riley, “Molecular graph convolutions: moving beyond fingerprints,” Journal of computer-aided molecular design, vol. 30, pp. 595–608, 2016.
  45. M. Elbadawi, S. Gaisford, and A. W. Basit, “Advanced machine-learning techniques in drug discovery,” Drug Discovery Today, vol. 26, no. 3, pp. 769–777, 2021.
  46. H. Yuan, I. Paskov, H. Paskov, A. J. González, and C. S. Leslie, “Multitask learning improves prediction of cancer drug sensitivity,” Scientific reports, vol. 6, no. 1, p. 31619, 2016.
  47. S. Bickel, J. Bogojeska, T. Lengauer, and T. Scheffer, “Multi-task learning for hiv therapy screening,” in Proceedings of the International Conference on Machine Learning, pp. 56–63, 2008.
  48. L. Rosenbaum, A. Dörr, M. R. Bauer, F. M. Boeckler, and A. Zell, “Inferring multi-target qsar models with taxonomy-based multi-task learning,” Journal of cheminformatics, vol. 5, no. 1, pp. 1–20, 2013.
  49. S. Liu, E. Johns, and A. J. Davison, “End-to-end multi-task learning with attention,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1871–1880, 2019.
  50. “Huber loss.” https://en.wikipedia.org/wiki/Huber_loss. Accessed in 3/2023.
  51. T. Xia and W.-S. Ku, “Geometric graph representation learning on protein structure prediction,” in Proceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1873–1883, 2021.
  52. J. R. Jeliazkov, R. Frick, J. Zhou, and J. J. Gray, “Robustification of rosettaantibody and rosetta snugdock,” PloS one, vol. 16, no. 3, p. e0234282, 2021.
  53. N. A. Marze, S. Lyskov, and J. J. Gray, “Improved prediction of antibody vl–vh orientation,” Protein Engineering, Design and Selection, vol. 29, no. 10, pp. 409–418, 2016.
  54. B. D. Weitzner and J. J. Gray, “Accurate structure prediction of cdr h3 loops enabled by a novel structure-based c-terminal constraint,” The Journal of Immunology, vol. 198, no. 1, pp. 505–515, 2017.
  55. J. Adolf-Bryfogle, Q. Xu, B. North, A. Lehmann, and R. L. Dunbrack Jr, “Pyigclassify: a database of antibody cdr structural classifications,” Nucleic acids research, vol. 43, no. D1, pp. D432–D438, 2015.
  56. W. Kabsch, “A solution for the best rotation to relate two sets of vectors,” Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography, vol. 32, no. 5, pp. 922–923, 1976.
  57. B. D. Weitzner, J. R. Jeliazkov, S. Lyskov, N. Marze, D. Kuroda, R. Frick, J. Adolf-Bryfogle, N. Biswas, R. L. Dunbrack Jr, and J. J. Gray, “Modeling and docking of antibody structures with rosetta,” Nature protocols, vol. 12, no. 2, pp. 401–416, 2017.
  58. J. Leem, J. Dunbar, G. Georges, J. Shi, and C. M. Deane, “Abodybuilder: Automated antibody structure prediction with data–driven accuracy estimation,” in MAbs, vol. 8, pp. 1259–1268, Taylor & Francis, 2016.
  59. R.-M. Lu, Y.-C. Hwang, I.-J. Liu, C.-C. Lee, H.-Z. Tsai, H.-J. Li, and H.-C. Wu, “Development of therapeutic antibodies for the treatment of diseases,” Journal of biomedical science, vol. 27, no. 1, pp. 1–30, 2020.
  60. P. J. Carter and G. A. Lazar, “Next generation antibody drugs: pursuit of the’high-hanging fruit’,” Nature Reviews Drug Discovery, vol. 17, no. 3, pp. 197–223, 2018.
  61. H. Kaplon, S. Crescioli, A. Chenoweth, J. Visweswaraiah, and J. M. Reichert, “Antibodies to watch in 2023,” in Proceedings of Mabs, vol. 15, p. 2153410, Taylor & Francis, 2023.
  62. E. K. Wagner and J. A. Maynard, “Engineering therapeutic antibodies to combat infectious diseases,” Current opinion in chemical engineering, vol. 19, pp. 131–141, 2018.
  63. C. E. Z. Chan, A. Chan, B. Hanson, E. Ooi, et al., “The use of antibodies in the treatment of infectious diseases,” Singapore Med J, vol. 50, no. 7, pp. 663–672, 2009.
  64. Z. Chen, R. K. Kankala, Z. Yang, W. Li, S. Xie, H. Li, A.-Z. Chen, and L. Zou, “Antibody-based drug delivery systems for cancer therapy: Mechanisms, challenges, and prospects,” Theranostics, vol. 12, no. 8, p. 3719, 2022.
  65. P. Sapra and B. Shor, “Monoclonal antibody-based therapies in cancer: advances and challenges,” Pharmacology & therapeutics, vol. 138, no. 3, pp. 452–469, 2013.
  66. T. Zhou, J. Zhu, X. Wu, S. Moquin, B. Zhang, P. Acharya, I. S. Georgiev, H. R. Altae-Tran, G.-Y. Chuang, M. G. Joyce, et al., “Multidonor analysis reveals structural elements, genetic determinants, and maturation pathway for hiv-1 neutralization by vrc01-class antibodies,” Immunity, vol. 39, no. 2, pp. 245–258, 2013.
  67. P. A. Ott, Z. Hu, D. B. Keskin, S. A. Shukla, J. Sun, D. J. Bozym, W. Zhang, A. Luoma, A. Giobbie-Hurder, L. Peter, et al., “An immunogenic personal neoantigen vaccine for patients with melanoma,” Nature, vol. 547, no. 7662, pp. 217–221, 2017.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now