{"type":"ScholarlyArticle","title":"Transformer-based Objective-reinforced Generative Adversarial Network to Generate Desired Molecules","authors":[{"name":"Chen Li","slug":"chen-li","given":"Chen","family":"Li","affiliation":"Department of Bioscience and Bioinformatics , Faculty of Computer Science and Systems Engineering , Kyushu Institute of Technology , Iizuka , Japan"},{"name":"Chikashige Yamanaka","slug":"chikashige-yamanaka","given":"Chikashige","family":"Yamanaka","affiliation":"Department of Bioscience and Bioinformatics , Faculty of Computer Science and Systems Engineering , Kyushu Institute of Technology , Iizuka , Japan"},{"name":"Kazuma Kaitoh","slug":"kazuma-kaitoh","given":"Kazuma","family":"Kaitoh","affiliation":"Department of Bioscience and Bioinformatics , Faculty of Computer Science and Systems Engineering , Kyushu Institute of Technology , Iizuka , Japan"},{"name":"Yoshihiro Yamanishi","slug":"yoshihiro-yamanishi","given":"Yoshihiro","family":"Yamanishi","affiliation":"Department of Bioscience and Bioinformatics , Faculty of Computer Science and Systems Engineering , Kyushu Institute of Technology , Iizuka , Japan"}],"abstract":"Deep generative models of sequence-structure data have attracted widespread attention in drug discovery. However, such models cannot fully extract the semantic features of molecules from sequential representations. Moreover, mode collapse reduces the diversity of the generated molecules. This paper proposes a transformer-based objective-reinforced generative adversarial network (TransORGAN) to generate molecules. TransORGAN leverages a transformer architecture as a generator and uses a stochastic policy gradient for reinforcement learning to generate plausible molecules with rich semantic features. The discriminator grants rewards that guide the policy update of the generator, while an objective-reinforced penalty encourages the generation of diverse molecules. Experiments were performed using the ZINC chemical dataset, and the results demonstrated the usefulness of TransORGAN in terms of uniqueness, novelty, and diversity of the generated molecules.","doi":"10.24963/ijcai.2022/539","url":"https://ijcai.org/proceedings/2022/539","pdf":"https://ijcai.org/proceedings/2022/0539.pdf","markdown":"https://ijcai.org/proceedings/2022/539.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/539","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":539,"pages":{"first":3884,"last":3890},"keywords":["Multidisciplinary Topics and Applications: Bioinformatics","Multidisciplinary Topics and Applications: Health and Medicine","Multidisciplinary Topics and Applications: Life Science"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986781","length_sec":86,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984924","length_sec":329,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Arús-Pous et al., 2019] Josep Arús-Pous, Thomas Blaschke, Silas Ulander, Jean-Louis Reymond, Hong- ming Chen, and Ola Engkvist. Exploring the gdb-13 chemical space using deep generative models. Journal of cheminformatics, 11(1):1-14, 2019.","title":"Exploring the gdb-13 chemical space using deep generative models","authors":"Arús-Pous","year":"2019","venue":"Journal of cheminformatics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bickerton et al., 2012] G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90-98, 2012.","title":"Quantifying the chemical beauty of drugs","authors":"Richard Bickerton, V Gaia, Jérémy Paolini, Besnard, Andrew L Sorel Muresan, Hopkins","year":"2012","venue":"Nature chemistry","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bjerrum, 2017] Esben Jannik Bjerrum. Smiles enumera- tion as data augmentation for neural network modeling of molecules. arXiv preprint arXiv:1703.07076, 2017.","title":"Smiles enumeration as data augmentation for neural network modeling of molecules","authors":"Esben Jannik, Bjerrum","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cereto-Massagué et al., 2015] Adrià Cereto-Massagué, María José Ojeda, Cristina Valls, Miquel Mulero, San- tiago Garcia-Vallvé, and Gerard Pujadas. Molecular fingerprint similarity search in virtual screening. Methods, 71:58-63, 2015.","title":"Molecular fingerprint similarity search in virtual screening","authors":"Cereto-Massagué","year":"2015","venue":"Methods","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dai et al., 2018] Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoen- coder for molecule generation. In Proceedings of the in- ternational conference on learning representations, 2018.","title":"Syntax-directed variational autoencoder for molecule generation","authors":"Dai","year":"2018","venue":"Proceedings of the international conference on learning representations","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Danel et al., 2020] Tomasz Danel, Przemysław Spurek, Jacek Tabor, Marek Śmieja, Łukasz Struski, Agnieszka Słowik, and Łukasz Maziarka. Spatial graph convolutional networks. In International Conference on Neural Informa- tion Processing, pages 668-675. Springer, 2020.","title":"Spatial graph convolutional networks","authors":"Tomasz Danel, Przemysław Spurek, Jacek Tabor, Marek Śmieja, Łukasz Struski, Agnieszka Słowik, Łukasz Maziarka","year":"2020","venue":"International Conference on Neural Information Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[De Cao and Kipf, 2018] Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs. arXiv preprint arXiv:1805.11973, 2018.","title":"Molgan: An implicit generative model for small molecular graphs","authors":"Nicola De, Cao, Thomas Kipf","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ertl and Schuffenhauer, 2009] Peter Ertl and Ansgar Schuf- fenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics, 1(1):1-11, 2009.","title":"Peter Ertl and Ansgar Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions","authors":"Schuffenhauer Ertl","year":"2009","venue":"Journal of cheminformatics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao and Coley, 2020] Wenhao Gao and Connor W Coley. The synthesizability of molecules proposed by generative models. Journal of chemical information and modeling, 60(12):5714-5723, 2020.","title":"The synthesizability of molecules proposed by generative models","authors":"Coley ; ] Wenhao Gao, Connor W Gao, Coley","year":"2020","venue":"Journal of chemical information and modeling","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. Advances in neural information processing systems, 27, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guimaraes et al., 2017] Gabriel Lima Guimaraes, Ben- jamin Sanchez-Lengeling, Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik. Objective- reinforced generative adversarial networks (organ) for sequence generation models. arXiv preprint arXiv:1705.10843, 2017.","title":"Objectivereinforced generative adversarial networks (organ) for sequence generation models","authors":"Gabriel Lima, Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Pedro, Luis Cunha Farias, Alán Aspuru-Guzik","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2021] Jiazhen He, Huifang You, Emil Sandström, Eva Nittinger, Esben Jannik Bjerrum, Christian Tyrchan, Werngard Czechtizky, and Ola Engkvist. Molecular opti- mization by capturing chemist's intuition using deep neu- ral networks. Journal of cheminformatics, 13(1):1-17, 2021.","title":"Molecular optimization by capturing chemist's intuition using deep neural networks","authors":"He","year":"2021","venue":"Journal of cheminformatics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jin et al., 2018] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molec- ular graph generation. In International conference on ma- chine learning, pages 2323-2332. PMLR, 2018.","title":"Junction tree variational autoencoder for molecular graph generation","authors":"Jin","year":"2018","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kusner et al., 2017] Matt J Kusner, Brooks Paige, and José Miguel Hernández-Lobato. Grammar variational au- toencoder. In International Conference on Machine Learn- ing, pages 1945-1954. PMLR, 2017.","title":"Grammar variational autoencoder","authors":"Matt J Kusner, Brooks Paige, José Miguel Hernández-Lobato","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Olivecrona et al., 2017] Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):1-14, 2017.","title":"Molecular de-novo design through deep reinforcement learning","authors":"Olivecrona","year":"2017","venue":"Journal of cheminformatics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ramakrishnan et al., 2014] Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilien- feld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7, 2014.","title":"Quantum chemistry structures and properties of 134 kilo molecules","authors":"Raghunathan Ramakrishnan, O Pavlo, Matthias Dral, O Rupp, Von Anatole, Lilienfeld","year":"2014","venue":"Scientific data","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ruddigkeit et al., 2012] Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17. Journal of chemical information and modeling, 52(11):2864-2875, 2012.","title":"Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17","authors":"Ruddigkeit","year":"2012","venue":"Journal of chemical information and modeling","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi et al., 2019] Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow-based autoregressive model for molecular graph gen- eration. In International Conference on Learning Repre- sentations, 2019.","title":"Graphaf: a flow-based autoregressive model for molecular graph generation","authors":"Shi","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sutton et al., 2000] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approxi- mation. In Advances in neural information processing sys- tems, pages 1057-1063, 2000.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"David A Richard S Sutton, Mcallester, P Satinder, Yishay Singh, Mansour","year":"2000","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tanimoto, 1968] Taffee T Tanimoto. An elementary mathe- matical theory of classification and prediction, ibm report (november, 1958). Automatic Information Organization and Retrieval, 1968.","title":"An elementary mathematical theory of classification and prediction","authors":"T Taffee, Tanimoto","year":"1958","venue":"Automatic Information Organization and Retrieval","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Weininger, 1988] David Weininger. Smiles, a chemical lan- guage and information system. 1. introduction to method- ology and encoding rules. Journal of chemical information and computer sciences, 28(1):31-36, 1988.","title":"Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules","authors":"David Weininger","year":"1988","venue":"Journal of chemical information and computer sciences","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2017] Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: Sequence generative adversarial nets with policy gradient. In Proceedings of the AAAI confer- ence on artificial intelligence, volume 31, 2017.","title":"Seqgan: Sequence generative adversarial nets with policy gradient","authors":"Lantao Yu, Weinan Zhang, Jun Wang, Yong Yu","year":"2017","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}