{"type":"ScholarlyArticle","title":"A Universal PINNs Method for Solving Partial Differential Equations with a Point Source","authors":[{"name":"Xiang Huang","slug":"xiang-huang","given":"Xiang","family":"Huang","affiliation":"University of Science and Technology of China"},{"name":"Hongsheng Liu","slug":"hongsheng-liu","given":"Hongsheng","family":"Liu","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Beiji Shi","slug":"beiji-shi","given":"Beiji","family":"Shi","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Zidong Wang","slug":"zidong-wang","given":"Zidong","family":"Wang","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Kang Yang","slug":"kang-yang","given":"Kang","family":"Yang","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Yang Li","slug":"yang-li","given":"Yang","family":"Li","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Min Wang","slug":"min-wang","given":"Min","family":"Wang","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Haotian Chu","slug":"haotian-chu","given":"Haotian","family":"Chu","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Jing Zhou","slug":"jing-zhou","given":"Jing","family":"Zhou","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Fan Yu","slug":"fan-yu","given":"Fan","family":"Yu","affiliation":"Huawei Technologies Co. Ltd"},{"name":"Bei Hua","slug":"bei-hua","given":"Bei","family":"Hua","affiliation":"University of Science and Technology of China"},{"name":"Bin Dong","slug":"bin-dong","given":"Bin","family":"Dong","affiliation":"Peking University"},{"name":"Lei Chen","slug":"lei-chen","given":"Lei","family":"Chen","affiliation":"Hong Kong University of Science and Technology"}],"abstract":"In recent years, deep learning technology has been used to solve partial differential equations (PDEs), among which the physics-informed neural networks (PINNs)method emerges to be a promising method for solving both forward and inverse PDE problems. PDEs with a point source that is expressed as a Dirac delta function in the governing equations are mathematical models of many physical processes. However, they cannot be solved directly by conventional PINNs method due to the singularity brought by the Dirac delta function. In this paper, we propose a universal solution to tackle this problem by proposing three novel techniques.  Firstly the Dirac delta function is modeled as a continuous probability density function to eliminate the singularity at the point source; secondly a lower bound constrained uncertainty weighting algorithm is proposed to balance the physics-informed loss terms of point source area and the remaining areas; and thirdly a multi-scale deep neural network with periodic activation function is used to improve the accuracy and convergence speed. We evaluate the proposed method with three representative PDEs, and the experimental results show that our method outperforms existing deep learning based methods with respect to the accuracy, the efficiency and the versatility.","doi":"10.24963/ijcai.2022/533","url":"https://ijcai.org/proceedings/2022/533","pdf":"https://ijcai.org/proceedings/2022/0533.pdf","markdown":"https://ijcai.org/proceedings/2022/533.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/533","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Multidisciplinary Topics and Applications","track":"Main Track","number":533,"pages":{"first":3839,"last":3846},"keywords":["Multidisciplinary Topics and Applications: Physical Science","Machine Learning: Multi-task and Transfer Learning","Machine Learning: Unsupervised Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986482","length_sec":83,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984625","length_sec":341,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Bekele, 2020] Yared W Bekele. Physics-informed deep learning for flow and deformation in poroelastic media. arXiv preprint arXiv:2010.15426, 2020.","title":"Physics-informed deep learning for flow and deformation in poroelastic media","authors":"W Yared, Bekele","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bischof and Kraus, 2021] Rafael Bischof and Michael Kraus. Multi-objective loss balancing for physics- informed deep learning. arXiv preprint arXiv:2110.09813, 2021.","title":"Multi-objective loss balancing for physicsinformed deep learning","authors":"Kraus Bischof, Rafael Bischof, Michael Kraus","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2021] Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Karniadakis. Physics- informed neural networks for heat transfer problems. Jour- nal of Heat Transfer, 143(6):060801, 2021.","title":"Physicsinformed neural networks for heat transfer problems","authors":"Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, George Em Karniadakis","year":"2021","venue":"Journal of Heat Transfer","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gedney, 2011] Stephen D Gedney. Introduction to the finite-difference time-domain (fdtd) method for electro- magnetics. Synthesis Lectures on Computational Electro- magnetics, 6(1):1-250, 2011.","title":"Introduction to the finite-difference time-domain (fdtd) method for electromagnetics","authors":"Stephen D Gedney","year":"2011","venue":"Synthesis Lectures on Computational Electromagnetics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hennigh et al., 2021] Oliver Hennigh, Susheela Narasimhan, Mohammad Amin Nabian, Akshay Subra- maniam, Kaustubh Tangsali, Zhiwei Fang, Max Rietmann, Wonmin Byeon, and Sanjay Choudhry. Nvidia simnet™: An ai-accelerated multi-physics simulation framework. In International Conference on Computational Science, pages 447-461. Springer, 2021.","title":"Nvidia simnet™: An ai-accelerated multi-physics simulation framework","authors":"Oliver Hennigh, Susheela Narasimhan, Mohammad Amin Nabian, Akshay Subramaniam, Kaustubh Tangsali, Zhiwei Fang, Max Rietmann, Wonmin Byeon, Sanjay Choudhry","year":"2021","venue":"International Conference on Computational Science","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Heydari et al., 2019] A Ali Heydari, Craig A Thompson, and Asif Mehmood. Softadapt: Techniques for adap- tive loss weighting of neural networks with multi-part loss functions. arXiv preprint arXiv:1912.12355, 2019.","title":"Softadapt: Techniques for adaptive loss weighting of neural networks with multi-part loss functions","authors":"Heydari","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jin et al., 2021] Xiaowei Jin, Shengze Cai, Hui Li, and George Em Karniadakis. Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incom- pressible navier-stokes equations. Journal of Computa- tional Physics, 426:109951, 2021.","title":"Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations","authors":"Xiaowei Jin, Shengze Cai, Hui Li, George Em Karniadakis","year":"2021","venue":"Journal of Computational Physics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kendall et al., 2018] Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482-7491, 2018.","title":"Multi-task learning using uncertainty to weigh losses for scene geometry and semantics","authors":"Alex Kendall, Yarin Gal, Roberto Cipolla","year":"2018","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2020] Zongyi Li, Nikola Kovachki, Kamyar Az- izzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural op- erator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.","title":"Fourier neural operator for parametric partial differential equations","authors":"Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, Anima Anandkumar","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2020] Ziqi Liu, Wei Cai, and Zhi-Qin John Xu. Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains. arXiv preprint arXiv:2007.11207, 2020.","title":"Multi-scale deep neural network (mscalednn) for solving poisson-boltzmann equation in complex domains","authors":"Ziqi Liu, Wei Cai, Zhi-Qin John Xu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lu et al., 2019] Lu Lu, Pengzhan Jin, and George Em Kar- niadakis. Deeponet: Learning nonlinear operators for identifying differential equations based on the univer- sal approximation theorem of operators. arXiv preprint arXiv:1910.03193, 2019.","title":"Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators","authors":"Lu Lu, Pengzhan Jin, George Em Karniadakis","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[McClenny and Braga-Neto, 2020] Levi McClenny and Ulisses Braga-Neto. Self-adaptive physics-informed neural networks using a soft attention mechanism. arXiv preprint arXiv:2009.04544, 2020.","title":"Self-adaptive physics-informed neural networks using a soft attention mechanism","authors":"Braga-Neto ; Mcclenny, Levi Mcclenny, Ulisses Braga-Neto","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Moseley et al., 2020] Ben Moseley, Andrew Markham, and Tarje Nissen-Meyer. Solving the wave equation with physics-informed deep learning. arXiv preprint arXiv:2006.11894, 2020.","title":"Solving the wave equation with physics-informed deep learning","authors":"Ben Moseley, Andrew Markham, Tarje Nissen-Meyer","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pinkus, 1999] Allan Pinkus. Approximation theory of the mlp model. Acta Numerica 1999: Volume 8, 8:143-195, 1999.","title":"Approximation theory of the mlp model","authors":"Allan Pinkus","year":"1999","venue":"Acta Numerica","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Raissi et al., 2019] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and in- verse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686- 707, 2019.","title":"Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations","authors":"Maziar Raissi, Paris Perdikaris, George E Karniadakis","year":"2019","venue":"Journal of Computational Physics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sirignano and Spiliopoulos, 2018] Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algo- rithm for solving partial differential equations. Journal of computational physics, 375:1339-1364, 2018.","title":"Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations","authors":"Spiliopoulos Sirignano","year":"2018","venue":"Journal of computational physics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sitzmann et al., 2020] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wet- zstein. Implicit neural representations with periodic ac- tivation functions. Advances in Neural Information Pro- cessing Systems, 33, 2020.","title":"Implicit neural representations with periodic activation functions","authors":"Julien Vincent Sitzmann, Alexander Martel, David Bergman, Gordon Lindell, Wetzstein","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sullivan, 2013] Dennis M Sullivan. Electromagnetic simu- lation using the FDTD method. John Wiley & Sons, 2013.","title":"Electromagnetic simulation using the FDTD method","authors":"Dennis M Sullivan","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2020] Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tan- gent kernel perspective. arXiv preprint arXiv:2007.14527, 2020.","title":"When and why pinns fail to train: A neural tangent kernel perspective","authors":"Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2021] Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055-A3081, 2021.","title":"Understanding and mitigating gradient flow pathologies in physics-informed neural networks","authors":"Wang","year":"2021","venue":"SIAM Journal on Scientific Computing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Weinan and Yu, 2018] E Weinan and Bing Yu. The deep ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Math- ematics and Statistics, 6(1):1-12, 2018.","title":"The deep ritz method: a deep learning-based numerical algorithm for solving variational problems","authors":"Yu ; ] E Weinan, Bing Weinan, Yu","year":"2018","venue":"Communications in Mathematics and Statistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xu et al., 2019] Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv:1901.06523, 2019.","title":"Frequency principle: Fourier analysis sheds light on deep neural networks","authors":"John Zhi-Qin, Yaoyu Xu, Tao Zhang, Yanyang Luo, Zheng Xiao, Ma","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2021] Pan Zhang, Yanyan Hu, Yuchen Jin, Shaogui Deng, Xuqing Wu, and Jiefu Chen. A maxwell's equations based deep learning method for time domain electromagnetic simulations. IEEE Journal on Multi- scale and Multiphysics Computational Techniques, 6:35- 40, 2021.","title":"A maxwell's equations based deep learning method for time domain electromagnetic simulations","authors":"Zhang","year":"2021","venue":"IEEE Journal on Multiscale and Multiphysics Computational Techniques","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}