一种用于优化分子属性预测的双层模型轻量化方法

分子属性预测在新药研发和材料设计等诸多科学领域中具有重要作用.由于分子天然可以表示为图结构，许多基于图的模型被广泛应用于该任务.随着分子空间的迅速扩展，基于图的方法正面临巨大的计算挑战，模型轻量化对于提升预测速度和效率至关重要.然而，现有的解决方案仍然较为有限，难以在保持预测性能的同时显著提高推理效率.提出一种新颖的双层模型轻量化方法LW⁃MPP，首先引入一种新的知识蒸馏框架，将大规模基于图的模型转换为更小更高效的基于SMILES的模型；其次，应用一种训练后剪枝技术，结合掩码搜索和重排序方法，进一步优化模型的推理效率.在大规模PCQM4M⁃LSC数据集上的基准测试结果表明，与传统基于图的模型相比，提出的方法实现了3.82~17倍的推理加速，同时保持了接近最优的性能，并优于大多数基于SMILES⁃Transformer的模型.当应用于MoleculeNet中小规模数据集的特定下游任务时，提出的模型在大多数情况下均实现了最佳的预测准确率.

Molecular property prediction is a fundamental task in various scientific domains，including drug discovery and material design. Given that molecular structures are naturally represented as graphs，numerous graph⁃based models have been developed to tackle this problem. However，as the molecular space continues to expand，these approaches face significant computational challenges，necessitating the development of lightweight models to enable faster and more efficient predictions. Despite this pressing need，effective solutions remain scarce. In this paper，we propose a novel two⁃level model lightweighting approach，named LW⁃MPP (Lightweighting Method for Efficient Molecular Property Prediction). First，we introduce a new knowledge distillation framework that converts large graph⁃based models into smaller SMILES (Simplified Molecular Input Line Entry System)⁃based models. Second，we apply a post⁃training pruning technique，which leverages masked search and reordering methods to further optimize model inference. Benchmark results on the large⁃scale PCQM4M⁃LSC (Predicting Quantum Mechanical Properties of Molecular Data⁃Large Scale Challenge) dataset demonstrate that our approach achieves a 3.82~17 times speedup in inference compared to traditional graph⁃based models，while maintaining near⁃optimal performance. Furthermore，our model outperforms most SMILES⁃Transformer⁃based models. When applied to specific downstream tasks with small⁃scale datasets from MoleculeNet，our model consistently achieves the best predictive accuracy in most cases.

PDF(918 KB)

南京大学学报(自然科学版) ›› 2025, Vol. 61 ›› Issue (6) : 953-962. DOI: 10.13232/j.cnki.jnju.2025.06.006

{{article.zuoZhe_CN}}

作者信息 +

A two⁃level model lightweighting method for efficient molecular property prediction

{{article.zuoZhe_EN}}

Author information +

文章历史 +

收稿日期	出版日期
2025-09-09	2025-11-30
发布日期
2025-12-17

本文亮点

HeighLight

摘要

Abstract

关键词

Key words

本文二维码

引用本文

EndNote

Ris (Procite)

Bibtex

导出引用

{{article.zuoZheCn_L}}.{{article.title_cn}}[J]. {{journal.qiKanMingCheng_CN}}, 2025, 61(6): 953-962 https://doi.org/10.13232/j.cnki.jnju.2025.06.006

{{article.zuoZheEn_L}}.{{article.title_en}}[J]. {{journal.qiKanMingCheng_EN}}, 2025, 61(6): 953-962 https://doi.org/10.13232/j.cnki.jnju.2025.06.006

中图分类号：

参考文献

列表(原文顺序|文献年度倒序|文中引用次数倒序) 可视化分析

参考文献

基金

编委：

主编：

责任编辑：

编辑:

版权

PDF(918 KB)

Accesses

Citation

Detail

段落导航

[1]	Hamilton W L. Graph representation learning. The 1^st Edition. Cham：Springer，2020.
[2]	Li Z， Jiang M J， Wang S，et al.Deep learning methods for molecular representation and property prediction. Drug Discovery Today，2022，27(12)：103373.
[3]	Lecun Y， Bottou L， Bengio Y，et al. Gradient?based learning applied to document recognition. Proceedings of the IEEE，1998，86(11)：2278-2324.
[4]	Scarselli F， Gori M， Tsoi A C，et al. The graph neural network model. IEEE Transactions on Neural Networks，2009，20(1)：61-80.
[5]	Gilmer J， Schoenholz S S， Riley P F，et al. Neural message passing for quantum chemistry∥Proceedings of the 34th International Conference on Machine Learning. Volume 70. Sydney，Australia：JMLR.org，2017：1263-1272.
[6]	Schütt K T， Kindermans P J， Sauceda H E，et al. SchNet：A continuous?filter convolutional neural network for modeling quantum interactions∥Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach，CA，USA：Curran Associates Inc.，2017：992-1002.
[7]	Gasteiger J， Gro? J， Günnemann S. Directional message passing for molecular graphs. https://arxiv.org/abs/2003.03123，2022-04-05.
[8]	Liu Y， Wang L M， Liu M，et al. Spherical message passing for 3D graph networks. https://arxiv.org/abs/2102.05013,2022-11-24.
[9]	Vaswani A， Shazeer N， Parmar N，et al. Attention is all you need∥Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach，CA，USA：Curran Associates Inc.，2017：6000-6010.
[10]	Dosovitskiy A， Beyer L， Kolesnikov A，et al. An image is worth 16×16 words：Transformers for image recognition at scale. https://arxiv.org/abs/2010.11929，2021-06-03.
[11]	Jumper J， Evans R， Pritzel A，et al. Highly accurate protein structure prediction with AlphaFold. Nature，2021，596(7873)：583-589.
[12]	Ying C X， Cai T L， Luo S J，et al. Do transformers really perform badly for graph representation?∥Advances in Neural Information Processing Systems. Vol. 34. New York，NY，USA：Curran Associates，2021：28877-28888.
[13]	Zhang Z X， Liu Q， Hu Q Y，et al. Hierarchical graph transformer with adaptive node sampling. Advances in Neural Information Processing Systems，2022，35：21171-21183.
[14]	Li H， Zhao D， Zeng J Y. KPGT：Knowledge?guided pre?training of graph transformer for molecular property prediction∥Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York，NY，USA：Association for Computing Machinery，2022：857-867.
[15]	Jiang Y H， Jin S T， Jin X R，et al. Pharmacophoric?constrained heterogeneous graph transformer model for molecular property prediction. Communications Chemistry，2023，6(1)：60.
[16]	Zhu W H， Li Z Y， Cai L S，et al. Stepping back to SMILES transformers for fast molecular represen?tation inference. https://arxiv.org/abs/2112.13305，2021-12-26.
[17]	Hinton G， Vinyals O， Dean J. Distilling the knowledge in a neural network. https://arxiv.org/abs/1503.02531,2015-03-09.
[18]	Kwon W， Kim S， Mahoney M W，et al. A fast post?training pruning framework for transformers. Advances in Neural Information Processing Systems，2022，35：24101-24116.
[19]	Hu W H， Fey M， Ren H Y，et al. OGB?LSC：A large?scale challenge for machine learning on graphs. https://arxiv.org/abs/2103.09430，2021-10-20.
[20]	Wu Z Q， Ramsundar B， Feinberg E N，et al. MoleculeNet：A benchmark for molecular machine learning. Chemical Science，2018，9(2)：513-530.
[21]	Bento A P， Hersey A， Félix E，et al. An open source chemical structure curation pipeline using RDKit. Journal of Cheminformatics，2020，12(1)：51.
[22]	Ly A， Marsman M， Verhagen J，et al. A tutorial on fisher information. Journal of Mathematical Psychology，2017，80：40-55.
[23]	Romero A， Ballas N， Kahou S E，et al. FitNets：Hints for thin deep nets. https://arxiv.org/abs/1412.6550，2015-03-27.
[24]	Nakata M， Shimazaki T. PubChemQC project：A large?scale first?principles electronic structure database for data?driven chemistry. Journal of Chemical Information and Modeling，2017，57(6)：1300-1308.
[25]	Wilson S， McWeeny R， Bernath P F. Handbook of molecular physics and quantum chemistry. The 1^st Edition. Hoboken：Wiley，2003.
[26]	Kipf T N， Welling M.Semi?supervised classification with graph convolutional networks. https://arxiv.org/abs/1609.02907，2017-02-22.
[27]	Xu K， Hu W H， Leskovec J，et al. How powerful are graph neural networks? https://arxiv.org/abs/1810.00826，2019-02-22.
[28]	Morgan H L. The generation of a unique machine description for chemical structures：A technique developed at chemical abstracts service. Journal of Chemical Information and Modeling，1965，5(2)：107-113.
[29]	Xiong Z P， Wang D Y， Liu X H，et al. Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism. Journal of Medicinal Chemistry，2020，63(16)：8749-8760.
[30]	Ross J， Belgodere B， Chenthamarakshan V，et al. Do large scale molecular language representations capture important structural information? https://doi.org/10.48550/arXiv.2106.09553，2022-12-14.

选择文件类型/文献管理软件名称

选择包含的内容