{"type":"ScholarlyArticle","title":"Towards Geometric Normalization Techniques in SE(3) Equivariant Graph Neural Networks for Physical Dynamics Simulations","authors":[{"name":"Ziqiao Meng","slug":"ziqiao-meng","given":"Ziqiao","family":"Meng","affiliation":"The Chinese University of Hong Kong"},{"name":"Liang Zeng","slug":"liang-zeng","given":"Liang","family":"Zeng","affiliation":"Tsinghua University"},{"name":"Zixing Song","slug":"zixing-song","given":"Zixing","family":"Song","affiliation":"The Chinese University of Hong Kong"},{"name":"Tingyang Xu","slug":"tingyang-xu","given":"Tingyang","family":"Xu","affiliation":"Tencent AI Lab"},{"name":"Peilin Zhao","slug":"peilin-zhao","given":"Peilin","family":"Zhao","affiliation":"Tencent AI Lab"},{"name":"Irwin King","slug":"irwin-king","given":"Irwin","family":"King","affiliation":"The Chinese University of Hong Kong"}],"abstract":"SE(3) equivariance is a fundamental property that is highly desirable to maintain in physical dynamics modeling. This property ensures neural outputs to remain robust when the inputs are translated or rotated. Recently, there have been several proposals for SE(3) equivariant graph neural networks (GNNs) that have shown promising results in simulating particle dynamics. However, existing works have neglected an important issue that current SE(3) equivariant GNNs cannot scale to large particle systems. Although some simple normalization techniques are already in use to stabilize the training dynamics of equivariant graph networks, they actually break the SE(3) equivariance of the architectures. In this work, we first show the numerical instability of training equivariant GNNs on large particle systems and then analyze some existing normalization strategies adopted in modern works. We propose a new normalization layer called GeoNorm, which can satisfy the SE(3) equivariance and simultaneously stabilize the training process. We conduct comprehensive experiments on N-body system simulation tasks with larger particle system sizes. The experimental results demonstrate that GeoNorm successfully preserves the SE(3) equivariance compared to baseline techniques and stabilizes the training dynamics of SE(3) equivariant GNNs on large systems.","doi":"10.24963/ijcai.2024/661","url":"https://ijcai.org/proceedings/2024/661","pdf":"https://ijcai.org/proceedings/2024/0661.pdf","markdown":"https://ijcai.org/proceedings/2024/661.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/661","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":661,"pages":{"first":5981,"last":5989},"keywords":["Machine Learning: ML: Geometric learning","Machine Learning: ML: Sequence and graph learning","Multidisciplinary Topics and Applications: MTA: Physical sciences","Machine Learning: ML: Deep learning architectures"],"videos":[],"references":[{"position":1,"raw":"[Anderson et al., 2019] Brandon Anderson, Truong Son Hy, and Risi Kondor. Cormorant: Covariant molecular neural networks. In NeurIPS, volume 32.","title":"Cormorant: Covariant molecular neural networks","authors":"Brandon Anderson, Truong, Son Hy, Risi Kondor","year":null,"venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Curran Associates, Inc., 2019.","title":null,"authors":"Curran Associates, Inc","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ba et al., 2016] Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. CoRR, abs/1607.06450, 2016.","title":"Layer normalization","authors":"Jimmy Lei, Jamie Ba, Geoffrey E Ryan Kiros, Hinton","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Batatia et al., 2022] Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, and Gabor Csanyi. Mace: Higher order equivariant message passing neural networks for fast and accurate force fields. In NeurIPS, volume 35, pages 11423-11436.","title":"Mace: Higher order equivariant message passing neural networks for fast and accurate force fields","authors":"Ilyes Batatia, David P Kovacs, Gregor Simm, Christoph Ortner, Gabor Csanyi","year":null,"venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Curran Associates, Inc., 2022.","title":null,"authors":"Curran Associates, Inc","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Batzner et al., 2022] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P. Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E. Smidt, and Boris Kozinsky. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 13(1), 2022.","title":"E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials","authors":"Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, Boris Kozinsky","year":"2022","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Brandstetter et al., 2022] Johannes Brandstetter, Rob Hes- selink, Elise van der Pol, Erik J Bekkers, and Max Welling. Geometric and physical quantities improve e(3) equivari- ant message passing. In ICLR, 2022.","title":"Geometric and physical quantities improve e(3) equivariant message passing","authors":"Johannes Brandstetter, Rob Hesselink, Elise Van Der Pol, Erik J Bekkers, Max Welling","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Brock et al., 2021] Andrew Brock, Soham De, and Samuel L Smith. Characterizing signal propagation to close the performance gap in unnormalized resnets. In ICLR, 2021.","title":"Characterizing signal propagation to close the performance gap in unnormalized resnets","authors":"Andrew Brock, Soham De, Samuel L Smith","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cai et al., 2021] Tianle Cai, Shengjie Luo, Keyulu Xu, Di He, Tie-Yan Liu, and Liwei Wang. Graphnorm: A principled approach to accelerating graph neural network training. In ICML, volume 139 of Proceedings of Machine Learning Research, pages 1204-1215. PMLR, 18-24 Jul 2021.","title":"Graphnorm: A principled approach to accelerating graph neural network training","authors":"Cai","year":"2021","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Daneshmand et al., 2020] Hadi Daneshmand, Jonas Kohler, Francis Bach, Thomas Hofmann, and Aurelien Lucchi. Batch normalization provably avoids ranks collapse for randomly initialised deep networks. In NeurIPS, vol- ume 33, pages 18387-18398.","title":"Batch normalization provably avoids ranks collapse for randomly initialised deep networks","authors":"Jonas Hadi Daneshmand, Francis Kohler, Thomas Bach, Aurelien Hofmann, Lucchi","year":null,"venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Deng et al., 2021] Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J. Guibas. Vector neurons: A general framework for so(3)- equivariant networks. In ICCV, pages 12180-12189. IEEE, 2021.","title":"Vector neurons: A general framework for so(3)equivariant networks","authors":"Deng","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Du et al., 2022] Weitao Du, He Zhang, Yuanqi Du, Qi Meng, Wei Chen, Nanning Zheng, Bin Shao, and Tie- Yan Liu. SE(3) equivariant graph neural networks with complete local frames. In ICML, volume 162 of Proceed- ings of Machine Learning Research, pages 5583-5608. PMLR, 17-23 Jul 2022.","title":"SE(3) equivariant graph neural networks with complete local frames","authors":"Du","year":"2022","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Dwivedi et al., 2023] Vijay Prakash Dwivedi, Chaitanya K. Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural net- works. J. Mach. Learn. Res., 24:43:1-43:48, 2023.","title":"Benchmarking graph neural networks","authors":"Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson","year":"2023","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Finzi et al., 2020] Marc Finzi, Samuel Stanton, Pavel Iz- mailov, and Andrew Gordon Wilson. Generalizing con- volutional neural networks for equivariance to lie groups on arbitrary continuous data. In ICML, volume 119 of Proceedings of Machine Learning Research, pages 3165- 3176. PMLR, 2020.","title":"Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data","authors":"Finzi","year":"2020","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Frank et al., 2022] Thorben Frank, Oliver Thorsten Unke, and Klaus Robert Muller. So3krates: Equivariant atten- tion for interactions on arbitrary length-scales in molecular systems. In NeurIPS, 2022.","title":"So3krates: Equivariant attention for interactions on arbitrary length-scales in molecular systems","authors":"Thorben Frank, Oliver Thorsten Unke, Klaus Robert Muller","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Fuchs et al., 2020] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling. Se(3)-transformers: 3d roto- translation equivariant attention networks. In NeurIPS, volume 33, pages 1970-1981.","title":"Se(3)-transformers: 3d rototranslation equivariant attention networks","authors":"Fabian Fuchs, Daniel Worrall, Volker Fischer, Max Welling","year":null,"venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ganea et al., 2022] Octavian-Eugen Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi S. Jaakkola, and Andreas Krause. Independent SE(3)-equivariant models for end-to-end rigid protein docking. In ICLR, 2022.","title":"Independent SE(3)-equivariant models for end-to-end rigid protein docking","authors":"Ganea","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Gasteiger et al., 2020] Johannes Gasteiger, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs. In ICLR, 2020.","title":"Directional message passing for molecular graphs","authors":"Gasteiger","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, volume 30.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Hoogeboom et al., 2022] Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivari- ant diffusion for molecule generation in 3d. In ICML 2022, volume 162 of Proceedings of Machine Learning Research, pages 8867-8887. PMLR, 2022.","title":"Equivariant diffusion for molecule generation in 3d","authors":"Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, Max Welling","year":"2022","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Huang et al., 2022] Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang. Equivari- ant graph mechanics networks with constraints. In ICLR, 2022.","title":"Equivariant graph mechanics networks with constraints","authors":"Huang","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Hutchinson et al., 2021] Michael J Hutchinson, Charline Le Lan, Sheheryar Zaidi, Emilien Dupont, Yee Whye Teh, and Hyunjik Kim. Lietransformer: Equivariant self- attention for lie groups. In ICML, volume 139 of Proceed- ings of Machine Learning Research, pages 4533-4543. PMLR, 18-24 Jul 2021.","title":"Lietransformer: Equivariant selfattention for lie groups","authors":"Charline Michael J Hutchinson, Sheheryar Le Lan, Emilien Zaidi, Yee Whye Dupont, Hyunjik Teh, Kim","year":"2021","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In ICML, volume 37 of Proceedings of Machine Learning Research, pages 448-456, Lille, France, 07-09 Jul 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"PMLR.","title":null,"authors":"Pmlr","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Jiao et al., 2023] Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal structure prediction by joint equivariant diffusion. In NeurIPS 2023, 2023.","title":"Crystal structure prediction by joint equivariant diffusion","authors":"Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, Yang Liu","year":"2023","venue":"NeurIPS 2023","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Jing et al., 2021] Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael John Lamarre Townshend, and Ron Dror. Learning from protein structure with geometric vector per- ceptrons. In ICLR, 2021.","title":"Learning from protein structure with geometric vector perceptrons","authors":"Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael, John Lamarre Townshend, Ron Dror","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Kipf et al., 2018] Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. In ICML, pages 2688- 2697. PMLR, 2018.","title":"Neural relational inference for interacting systems","authors":"Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, Richard Zemel","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Klicpera et al., 2021] Johannes Klicpera, Florian Becker, and Stephan Günnemann. Gemnet: Universal directional graph neural networks for molecules. In NeurIPS, 2021.","title":"Gemnet: Universal directional graph neural networks for molecules","authors":"Klicpera","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Köhler et al., 2020] Jonas Köhler, Leon Klein, and Frank Noe. Equivariant flows: Exact likelihood generative learn- ing for symmetric densities. In ICML, pages 5361-5370, 2020.","title":"Equivariant flows: Exact likelihood generative learning for symmetric densities","authors":"Köhler","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Li et al., 2022] Guohao Li, Chenxin Xiong, Guocheng Qian, Ali Thabet, and Bernard Ghanem. DeeperGCN: All you need to train deeper GCNs, 2022.","title":"DeeperGCN: All you need to train deeper GCNs","authors":"Li","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Liao and Smidt, 2023] Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. In ICLR, 2023.","title":"Equiformer: Equivariant graph attention transformer for 3d atomistic graphs","authors":"Smidt ; ] Yi-Lun Liao, Tess Liao, Smidt","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Liu et al., 2022] Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spher- ical message passing for 3d molecular graphs. In ICLR, 2022.","title":"Spherical message passing for 3d molecular graphs","authors":"Yi Liu, Limei Wang, Meng Liu, Yuchao Lin, Xuan Zhang, Bora Oztekin, Shuiwang Ji","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Meng et al., 2023] Ziqiao Meng, Peilin Zhao, Yang Yu, and Irwin King. A unified view of deep learning for reaction and retrosynthesis prediction: Current status and future challenges. In IJCAI 2023, pages 6723-6731. ijcai.org, 2023.","title":"A unified view of deep learning for reaction and retrosynthesis prediction: Current status and future challenges","authors":"Meng","year":"2023","venue":"IJCAI 2023","doi":null,"url":null,"internal_url":"/proceedings/2023/753"},{"position":39,"raw":"[Miyato et al., 2018] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral nor- malization for generative adversarial networks. In ICLR, 2018.","title":"Spectral normalization for generative adversarial networks","authors":"Miyato","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Salimans and Kingma, 2016] Tim Salimans and Durk P Kingma. Weight normalization: A simple reparameteri- zation to accelerate training of deep neural networks. In NIPS, volume 29.","title":"Weight normalization: A simple reparameterization to accelerate training of deep neural networks","authors":"Kingma Salimans, Tim Salimans, P Durk, Kingma","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"Curran Associates, Inc., 2016.","title":null,"authors":"Curran Associates, Inc","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Satorras et al., 2021a] Victor Garcia Satorras, Emiel Hoogeboom, Fabian Bernd Fuchs, Ingmar Posner, and Max Welling. E(n) equivariant normalizing flows. In NeurIPS, 2021.","title":"equivariant normalizing flows","authors":"Satorras","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Satorras et al., 2021b] Víctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural networks. In ICML, volume 139 of Proceedings of Machine Learning Research, pages 9323-9332. PMLR, 18-24 Jul 2021.","title":"E(n) equivariant graph neural networks","authors":"Satorras","year":"2021","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Schütt et al., 2017] Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexan- dre Tkatchenko, and Klaus-Robert Müller. Schnet: A continuous-filter convolutional neural network for model- ing quantum interactions. In NIPS, volume 30.","title":"Schnet: A continuous-filter convolutional neural network for modeling quantum interactions","authors":"Schütt","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Schütt et al., 2021] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In ICML, volume 139 of Proceedings of Machine Learn- ing Research, pages 9377-9388. PMLR, 18-24 Jul 2021.","title":"Equivariant message passing for the prediction of tensorial properties and molecular spectra","authors":"Schütt","year":"2021","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Shen et al., 2020] Wen Shen, Binbin Zhang, Shikun Huang, Zhihua Wei, and Quanshi Zhang. 3d-rotation-equivariant quaternion neural networks. In ECCV, volume 12365 of Lecture Notes in Computer Science, pages 531-547. Springer, 2020.","title":"3d-rotation-equivariant quaternion neural networks","authors":"Wen Shen, Binbin Zhang, Shikun Huang, Zhihua Wei, Quanshi Zhang","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Singh and Krishnan, 2020] Saurabh Singh and Shankar Kr- ishnan. Filter response normalization layer: Eliminating batch dependence in the training of deep neural networks. In CVPR, pages 11234-11243. Computer Vision Founda- tion / IEEE, 2020.","title":"Filter response normalization layer: Eliminating batch dependence in the training of deep neural networks","authors":"Saurabh Singh, Shankar Krishnan","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Song et al., 2024] Zixing Song, Ziqiao Meng, and Irwin King. A diffusion-based pre-training framework for crys- tal property prediction. In AAAI 2024, pages 8993-9001. AAAI Press, 2024.","title":"Zixing Song, Ziqiao Meng, and Irwin King. A diffusion-based pre-training framework for crystal property prediction","authors":"Song","year":"2024","venue":"AAAI 2024","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Thölke and Fabritiis, 2022] Philipp Thölke and Gianni De Fabritiis. Equivariant transformers for neural network based molecular potentials. In ICLR, 2022.","title":"Equivariant transformers for neural network based molecular potentials","authors":"Philipp Thölke, Gianni De, Fabritiis","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Thomas et al., 2018] Nathaniel Thomas, Tess E. Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds. CoRR, abs/1802.08219, 2018.","title":"Tensor field networks: Rotationand translation-equivariant neural networks for 3d point clouds","authors":"Nathaniel Thomas, Tess E Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, Patrick Riley","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Ulyanov et al., 2016] Dmitry Ulyanov, Andrea Vedaldi, and Victor S. Lempitsky. Instance normalization: The miss- ing ingredient for fast stylization. CoRR, abs/1607.08022, 2016.","title":"Lempitsky. Instance normalization: The missing ingredient for fast stylization","authors":"Dmitry Ulyanov, Andrea Vedaldi, S Victor","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Wu and He, 2018] Yuxin Wu and Kaiming He. Group nor- malization. In ECCV, volume 11217 of Lecture Notes in Computer Science, pages 3-19. Springer, 2018.","title":"Group normalization","authors":"He Wu, Yuxin Wu, Kaiming He","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Yang et al., 2020] Chaoqi Yang, Ruijie Wang, Shuochao Yao, Shengzhong Liu, and Tarek F. Abdelzaher. Revisiting \"over-smoothing\" in deep gcns. CoRR, abs/2003.13663, 2020.","title":"Revisiting \"over-smoothing\" in deep gcns","authors":"Yang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Zhao and Akoglu, 2020] Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. In ICLR, 2020.","title":"Pairnorm: Tackling oversmoothing in gnns","authors":"Akoglu Zhao, Lingxiao Zhao, Leman Akoglu","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Zhou et al., 2020] Kaixiong Zhou, Xiao Huang, Yuening Li, Daochen Zha, Rui Chen, and Xia Hu. Towards deeper graph neural networks with differentiable group normal- ization. In NeurIPS, volume 33, pages 4917-4928.","title":"Towards deeper graph neural networks with differentiable group normalization","authors":"Kaixiong Zhou, Xiao Huang, Yuening Li, Daochen Zha, Rui Chen, Xia Hu","year":null,"venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"Curran Associates, Inc., 2020.","title":null,"authors":"Curran Associates, Inc","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Zitnick et al., 2022] C. Lawrence Zitnick, Abhishek Das, Adeesh Kolluru, Janice Lan, Muhammed Shuaibi, Anuroop Sriram, Zachary Ward Ulissi, and Brandon M Wood. Spherical channels for modeling atomic interac- tions. In NeurIPS, 2022.","title":"Spherical channels for modeling atomic interactions","authors":"C Lawrence Zitnick, Abhishek Das, Adeesh Kolluru, Janice Lan, Muhammed Shuaibi, Anuroop Sriram, Zachary Ward Ulissi, Brandon M Wood","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}