{"type":"ScholarlyArticle","title":"Geometry-Guided Conditional Adaptation for Surrogate Models of Large-Scale 3D PDEs on Arbitrary Geometries","authors":[{"name":"Jingyang Deng","slug":"jingyang-deng","given":"Jingyang","family":"Deng","affiliation":"School of Mathematical Sciences and LMAM , Peking University"},{"name":"Xingjian Li","slug":"xingjian-li","given":"Xingjian","family":"Li","affiliation":"Computational Biology Department , Carnegie Mellon University 3 Baidu , Inc."},{"name":"Haoyi Xiong","slug":"haoyi-xiong","given":"Haoyi","family":"Xiong","affiliation":""},{"name":"Xiaoguang Hu","slug":"xiaoguang-hu","given":"Xiaoguang","family":"Hu","affiliation":""},{"name":"Jinwen Ma","slug":"jinwen-ma","given":"Jinwen","family":"Ma","affiliation":"School of Mathematical Sciences and LMAM , Peking University"}],"abstract":"Deep learning surrogate models aim to accelerate the solving of partial differential equations (PDEs) and have achieved certain promising results. Although several main-stream models through neural operator learning have been applied to delve into PDEs on varying geometries, they were designed to map the complex geometry to a latent uniform grid, which is still challenging to learn by the networks with general architectures. In this work, we rethink the critical factors of PDE solutions and propose a novel model-agnostic framework, called 3D Geometry-Guided Conditional adaptation (3D-GeoCA), for solving PDEs on arbitrary 3D geometries. Starting with a 3D point cloud geometry encoder, 3D-GeoCA can extract the essential and robust representations of any kind of geometric shapes, which conditionally guides the adaptation of hidden features in the surrogate model. We conduct experiments on two public computational fluid dynamics datasets, the Shape-Net Car and Ahmed-Body dataset, using several surrogate models as the backbones with various point cloud geometry encoders to simulate corresponding large-scale Reynolds Average Navier-Stokes equations. Equipped with 3D-GeoCA, these backbone models can reduce the L-2 error by a large margin. Moreover, this 3D-GeoCA is model-agnostic so that it can be applied to any surrogate model.","doi":"10.24963/ijcai.2024/640","url":"https://ijcai.org/proceedings/2024/640","pdf":"https://ijcai.org/proceedings/2024/0640.pdf","markdown":"https://ijcai.org/proceedings/2024/640.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/640","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":640,"pages":{"first":5790,"last":5798},"keywords":["Machine Learning: ML: Supervised Learning","Multidisciplinary Topics and Applications: MTA: Physical sciences","Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning","Computer Vision: CV: 3D computer vision"],"videos":[],"references":[{"position":1,"raw":"[Ahmed et al., 2019] S. R. Ahmed, G. Ramm, and G. Faitin. Some salient features of the time -averaged ground vehicle wake. Sae Paper, 840300, 2019.","title":"Some salient features of the time -averaged ground vehicle wake","authors":"S R Ahmed, G Ramm, G Faitin","year":"2019","venue":"Sae Paper","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anandkumar et al., 2020] Anima Anandkumar, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Nikola Kovachki, Zongyi Li, Burigede Liu, and Andrew Stuart. Neural op- erator: Graph kernel network for partial differential equa- tions. In ICLR 2020 Workshop on Integration of Deep Neu- ral Models and Differential Equations, 2020.","title":"Neural operator: Graph kernel network for partial differential equations","authors":"Anima Anandkumar, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Nikola Kovachki, Zongyi Li, Burigede Liu, Andrew Stuart","year":"2020","venue":"ICLR 2020 Workshop on Integration of Deep Neural Models and Differential Equations","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belbute-Peres et al., 2020] Filipe De Avila Belbute-Peres, Thomas Economon, and Zico Kolter. Combining differ- entiable pde solvers and graph neural networks for fluid flow prediction. In international conference on machine learning, pages 2402-2411. PMLR, 2020.","title":"Combining differentiable pde solvers and graph neural networks for fluid flow prediction","authors":"Filipe De, Avila Belbute-Peres, Thomas Economon, Zico Kolter","year":"2020","venue":"international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bonnet et al., 2022a] Florent Bonnet, Jocelyn Mazari, Paola Cinnella, and Patrick Gallinari. Airfrans: High fidelity computational fluid dynamics dataset for ap- proximating reynolds-averaged navier-stokes solutions. Advances in Neural Information Processing Systems, 35:23463-23478, 2022.","title":"Airfrans: High fidelity computational fluid dynamics dataset for approximating reynolds-averaged navier-stokes solutions","authors":"Bonnet","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bonnet et al., 2022b] Florent Bonnet, Jocelyn Ahmed Mazari, Thibaut Munzer, Pierre Yser, and Patrick Gal- linari. An extensible benchmarking graph-mesh dataset for studying steady-state incompressible navier-stokes equations. In ICLR 2022 Workshop on Geometrical and Topological Representation Learning, 2022.","title":"An extensible benchmarking graph-mesh dataset for studying steady-state incompressible navier-stokes equations","authors":"Bonnet","year":"2022","venue":"ICLR 2022 Workshop on Geometrical and Topological Representation Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chang et al., 2015] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015.","title":"Shapenet: An information-rich 3d model repository","authors":"Thomas Angel X Chang, Leonidas Funkhouser, Pat Guibas, Qixing Hanrahan, Zimo Huang, Silvio Li, Manolis Savarese, Shuran Savva, Hao Song, Su","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing sys- tems, 30, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hughes, 2012] Thomas JR Hughes. The finite element method: linear static and dynamic finite element analysis. Courier Corporation, 2012.","title":"The finite element method: linear static and dynamic finite element analysis","authors":"J R Thomas, Hughes","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2020a] Zongyi Li, Nikola Kovachki, Kamyar Az- izzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhattacharya, and Anima Anandkumar. Multipole graph neural operator for parametric partial differential equa- tions. Advances in Neural Information Processing Sys- tems, 33:6755-6766, 2020.","title":"Multipole graph neural operator for parametric partial differential equations","authors":"Li","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2020b] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar, et al. Fourier neural operator for parametric partial differential equations. In Interna- tional Conference on Learning Representations, 2020.","title":"Fourier neural operator for parametric partial differential equations","authors":"Li","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2022] Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. arXiv preprint arXiv:2207.05209, 2022.","title":"Fourier neural operator with learned deformations for pdes on general geometries","authors":"Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, Anima Anandkumar","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2023a] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023.","title":"Blip-2: Bootstrapping language-image pretraining with frozen image encoders and large language models","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2023b] Zongyi Li, Nikola Borislavov Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Prakash Otta, Mohammad Amin Nabian, Maximilian Stadler, Chris- tian Hundt, Kamyar Azizzadenesheli, et al. Geometry- informed neural operator for large-scale 3d pdes. In Thirty-seventh Conference on Neural Information Pro- cessing Systems, 2023.","title":"Geometryinformed neural operator for large-scale 3d pdes","authors":"Li","year":"2023","venue":"Thirty-seventh Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mu et al., 2022] Norman Mu, Alexander Kirillov, David Wagner, and Saining Xie. Slip: Self-supervision meets language-image pre-training. In European Conference on Computer Vision, pages 529-544. Springer, 2022.","title":"Slip: Self-supervision meets language-image pre-training","authors":"Norman Mu, Alexander Kirillov, David Wagner, Saining Xie","year":"2022","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pfaff et al., 2020] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia. Learning mesh-based simulation with graph networks. arXiv preprint arXiv:2010.03409, 2020.","title":"Learning mesh-based simulation with graph networks","authors":"Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, Peter W Battaglia","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Qi et al., 2017] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neu- ral information processing systems, 30, 2017.","title":"Pointnet++: Deep hierarchical feature learning on point sets in a metric space","authors":"Charles Ruizhongtai, Qi, Li Yi, Hao Su, Leonidas J Guibas","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Qian et al., 2022] Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Information Processing Systems, 35:23192-23204, 2022.","title":"Pointnext: Revisiting pointnet++ with improved training and scaling strategies","authors":"Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, Bernard Ghanem","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Raissi et al., 2019] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and in- verse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686- 707, 2019.","title":"Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations","authors":"Maziar Raissi, Paris Perdikaris, George E Karniadakis","year":"2019","venue":"Journal of Computational physics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234- 241. Springer, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Smith and Topin, 2019] Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelli- gence and machine learning for multi-domain operations applications, volume 11006, pages 369-386. SPIE, 2019.","title":"Super-convergence: Very fast training of neural networks using large learning rates","authors":"N Leslie, Nicholay Smith, Topin","year":"2019","venue":"Artificial intelligence and machine learning for multi-domain operations applications","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Strikwerda, 2004] John C Strikwerda. Finite difference schemes and partial differential equations. SIAM, 2004.","title":"Finite difference schemes and partial differential equations","authors":"C John, Strikwerda","year":"2004","venue":"SIAM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tran et al., 2022] Alasdair Tran, Alexander Mathews, Lex- ing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In The Eleventh International Conference on Learning Representations, 2022.","title":"Factorized fourier neural operators","authors":"Alasdair Tran, Alexander Mathews, Lexing Xie, Soon Cheng, Ong","year":"2022","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Umetani and Bickel, 2018] Nobuyuki Umetani and Bernd Bickel. Learning three-dimensional flow for interactive aerodynamic design. ACM Transactions on Graphics (TOG), 37(4):1-10, 2018.","title":"Learning three-dimensional flow for interactive aerodynamic design","authors":"Bickel Umetani, Nobuyuki Umetani, Bernd Bickel","year":"2018","venue":"ACM Transactions on Graphics (TOG)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In Interna- tional Conference on Learning Representations, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xue et al., 2023a] Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1179-1189, 2023.","title":"Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding","authors":"Xue","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xue et al., 2023b] Le Xue, Ning Yu, Shu Zhang, Junnan Li, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip-2: To- wards scalable multimodal pre-training for 3d understand- ing. arXiv preprint arXiv:2305.08275, 2023.","title":"Ulip-2: Towards scalable multimodal pre-training for 3d understanding","authors":"Xue","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2022] Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre- training 3d point cloud transformers with masked point modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19313- 19322, 2022.","title":"Point-bert: Pretraining 3d point cloud transformers with masked point modeling","authors":"Yu","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}