Empirical Evidence for the Fragment level Understanding on Drug Molecular Structure of LLMs (2401.07657v1)

Abstract: AI for drug discovery has been a research hotspot in recent years, and SMILES-based LLMs has been increasingly applied in drug molecular design. However, no work has explored whether and how LLMs understand the chemical spatial structure from 1D sequences. In this work, we pre-train a transformer model on chemical language and fine-tune it toward drug design objectives, and investigate the correspondence between high-frequency SMILES substrings and molecular fragments. The results indicate that LLMs can understand chemical structures from the perspective of molecular fragments, and the structural knowledge learned through fine-tuning is reflected in the high-frequency SMILES substrings generated by the model.