Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
60 tokens/sec
GPT-4o
12 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PACIA: Parameter-Efficient Adapter for Few-Shot Molecular Property Prediction (2310.00614v2)

Published 1 Oct 2023 in cs.LG and cs.AI

Abstract: Molecular property prediction (MPP) plays a crucial role in biomedical applications, but it often encounters challenges due to a scarcity of labeled data. Existing works commonly adopt gradient-based strategy to update a large amount of parameters for task-level adaptation. However, the increase of adaptive parameters can lead to overfitting and poor performance. Observing that graph neural network (GNN) performs well as both encoder and predictor, we propose PACIA, a parameter-efficient GNN adapter for few-shot MPP. We design a unified adapter to generate a few adaptive parameters to modulate the message passing process of GNN. We then adopt a hierarchical adaptation mechanism to adapt the encoder at task-level and the predictor at query-level by the unified GNN adapter. Extensive results show that PACIA obtains the state-of-the-art performance in few-shot MPP problems, and our proposed hierarchical adaptation mechanism is rational and effective.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Low data drug discovery with one-shot learning. ACS Central Science, 3(4):283–293, 2017.
  2. Marc Brockschmidt. GNN-FiLM: Graph neural networks with feature-wise linear modulation. In International Conference on Machine Learning, pages 1144–1152, 2020.
  3. Meta-learning adaptive deep kernel gaussian processes for molecular property prediction. In International Conference on Learning Representations, 2022.
  4. Principal neighbourhood aggregation for graph nets. In Advances in Neural Information Processing Systems, pages 13260–13271, 2020.
  5. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pages 1126–1135, 2017.
  6. Neural message passing for quantum chemistry. In International Conference on Machine Learning, pages 1263–1272, 2017.
  7. Recasting gradient-based meta-learning as hierarchical Bayes. In International Conference on Learning Representations, 2018.
  8. Few-shot graph learning for molecular property prediction. In The Web Conference, pages 2559–2567, 2021.
  9. Hypernetworks. In International Conference on Learning Representations, 2017.
  10. GraphMAE: Self-supervised masked graph autoencoders. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 594–604, 2022.
  11. Strategies for pre-training graph neural networks. In International Conference on Learning Representations, 2019.
  12. Edge-labeling graph neural network for few-shot learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11–20, 2019.
  13. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, 2015.
  14. The SIDER database of drugs and side effects. Nucleic Acids Research, 44(D1):D1075–D1079, 2016.
  15. Adaptive graph convolutional neural networks. In AAAI Conference on Artificial Intelligence, pages 3546–3553, 2018.
  16. Task-adaptive neural process for user cold-start recommendation. In The Web Conference, pages 1306–1316, 2021.
  17. Deep neural nets as a method for quantitative structure–activity relationships. Journal of Chemical Information and Modeling, 55(2):263–274, 2015.
  18. Molecule attention transformer. arXiv preprint arXiv:2002.08264, 2020.
  19. Chembl: towards direct deposition of bioassay data. Nucleic Acids Research, 47(D1):D930–D940, 2019.
  20. Molecule property prediction and classification with graph hypernetworks. arXiv preprint arXiv:2002.00240, 2020.
  21. National Center for Advancing Translational Sciences. Tox21 challenge. http://tripod.nih.gov/tox21/challenge/, 2017. Accessed: 2016-11-06.
  22. FiLM: Visual reasoning with a general conditioning layer. In AAAI Conference on Artificial Intelligence, pages 3942–3951, 2018.
  23. HyperMAML: Few-shot adaptation of deep models with hypernetworks. arXiv preprint arXiv:2205.15745, 2022.
  24. Meta-learning with implicit gradients. In Advances in Neural Information Processing Systems, pages 113–124, 2019.
  25. Fast and flexible multi-task classification using conditional neural adaptive processes. In Advances in Neural Information Processing Systems, pages 7957–7968, 2019.
  26. ToxCast chemical landscape: Paving the road to 21st century toxicology. Chemical Research in Toxicology, 29(8):1225–1251, 2016.
  27. Extended-connectivity fingerprints. Journal of Chemical Information and Modeling, 50(5):742–754, 2010.
  28. Maximum unbiased validation (MUV) data sets for virtual screening based on PubChem bioactivity data. Journal of Chemical Information and Modeling, 49(2):169–184, 2009.
  29. Context-enriched molecule representations improve few-shot drug discovery. arXiv preprint arXiv:2305.09481, 2023.
  30. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4080–4090, 2017.
  31. FS-Mol: A few-shot learning dataset of molecules. In Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
  32. ZINC 15–ligand discovery for everyone. Journal of Chemical Information and Modeling, 55(11):2324–2337, 2015.
  33. Deep learning as an opportunity in virtual screening. In NIPS Deep Learning Workshop, volume 27, pages 1–9, 2014.
  34. Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(11), 2008.
  35. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, pages 3630–3638, 2016.
  36. Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys, 53(3):1–34, 2020.
  37. Property-aware relation networks for few-shot molecular property prediction. In Advances in Neural Information Processing Systems, pages 17441–17454, 2021.
  38. An analysis of the attrition of drug candidates from four major pharmaceutical companies. Nature Reviews Drug discovery, 14(7):475–486, 2015.
  39. MoleculeNet: A benchmark for molecular machine learning. Chemical Science, 9(2):513–530, 2018.
  40. ColdNAS: Search to modulate for user cold-start recommendation. In The Web Conference, pages 1021–1031, 2023.
  41. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of Medicinal Chemistry, 63(16):8749–8760, 2019.
  42. How powerful are graph neural networks? In International Conference on Learning Representations, 2019.
  43. Self-supervised graph-level representation learning with local and global structure. In International Conference on Machine Learning, pages 11548–11558, 2021.
  44. Analyzing learned molecular representations for property prediction. Journal of Chemical Information and Modeling, 59(8):3370–3388, 2019.
  45. Meta-learning without memorization. In International Conference on Learning Representations, 2020.
  46. Graph sampling-based meta-learning for molecular property prediction. arXiv preprint arXiv:2306.16780, 2023.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets