{"type":"ScholarlyArticle","title":"Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks?","authors":[{"name":"Bing Liu","slug":"bing-liu","given":"Bing","family":"Liu","affiliation":"College of Computer Science and Technology , Jilin University , China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education , China"},{"name":"Wei Luo","slug":"wei-luo","given":"Wei","family":"Luo","affiliation":"School of Information Technology , Deakin University , Geelong , Australia"},{"name":"Gang Li","slug":"gang-li","given":"Gang","family":"Li","affiliation":"School of Information Technology , Deakin University , Geelong , Australia"},{"name":"Jing Huang","slug":"jing-huang","given":"Jing","family":"Huang","affiliation":"College of Computer Science and Technology , Jilin University , China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education , China"},{"name":"Bo Yang","slug":"bo-yang","given":"Bo","family":"Yang","affiliation":"College of Computer Science and Technology , Jilin University , China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education , China"}],"abstract":"As deep learning gains popularity in modelling dynamical systems, we expose an underappreciated misunderstanding relevant to modelling dynamics on networks. Strongly influenced by graph neural networks, latent vertex embeddings are naturally adopted in many neural dynamical network models. However, we show that embeddings tend to induce a model that fits observations well but simultaneously has incorrect dynamical behaviours. Recognising that previous studies narrowly focus on short-term predictions during the transient phase of a flow, we propose three tests for correct long-term behaviour, and illustrate how an embedding-based dynamical model fails these tests, and analyse the causes, particularly through the lens of topological conjugacy. In doing so, we show that the difficulties can be avoided by not using embedding. We propose a simple embedding-free alternative based on parametrising two additive vector-field components. Through extensive experiments, we verify that the proposed model can reliably recover a broad class of dynamics on different network topologies from time series data.","doi":"10.24963/ijcai.2023/242","url":"https://ijcai.org/proceedings/2023/242","pdf":"https://ijcai.org/proceedings/2023/0242.pdf","markdown":"https://ijcai.org/proceedings/2023/242.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/242","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Data Mining","track":"Main Track","number":242,"pages":{"first":2178,"last":2186},"keywords":["Machine Learning: ML: Time series and data streams","Data Mining: DM: Networks"],"videos":[],"references":[{"position":1,"raw":"[Barabási and Albert, 1999] Albert-László Barabási and Réka Albert. Emergence of scaling in random networks. science, 286(5439):509-512, 1999.","title":"Emergence of scaling in random networks","authors":"Albert-László Barabási, Réka Albert","year":"1999","venue":"science","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barzel and Barabási, 2013] Baruch Barzel and Albert- László Barabási. Universality in network dynamics. Nature physics, 9(10):673-681, 2013.","title":"Universality in network dynamics","authors":"Barabási Barzel, Baruch Barzel, Albert-László Barabási","year":"2013","venue":"Nature physics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Barzel et al., 2015] Baruch Barzel, Yang-Yu Liu, and Albert-László Barabási. Constructing minimal models for complex system dynamics. Nature communications, 6(1):1-8, 2015.","title":"Constructing minimal models for complex system dynamics","authors":"Baruch Barzel, Yang-Yu Liu, Albert-László Barabási","year":"2015","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blumer et al., 1987] Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, and Manfred K Warmuth. Occam's razor. Information processing letters, 24(6):377-380, 1987.","title":"Occam's razor","authors":"Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, Manfred K Warmuth","year":"1987","venue":"Information processing letters","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brunton et al., 2016] Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical sys- tems. Proceedings of the national academy of sciences, 113(15):3932-3937, 2016.","title":"Discovering governing equations from data by sparse identification of nonlinear dynamical systems","authors":"Joshua L Steven L Brunton, Nathan Proctor, Kutz","year":"2016","venue":"Proceedings of the national academy of sciences","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chamberlain et al., 2021] Ben Chamberlain, James Row- bottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, and Emanuele Rossi. Grand: Graph neural diffu- sion. In International Conference on Machine Learning, pages 1407-1418. PMLR, 2021.","title":"Grand: Graph neural diffusion","authors":"Ben Chamberlain, James Rowbottom, Maria I Gorinova, Michael Bronstein, Stefan Webb, Emanuele Rossi","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2014] Xu Chen, Xiuyuan Cheng, and Stéphane Mallat. Unsupervised deep haar scattering on graphs. Advances in Neural Information Processing Systems, 27, 2014.","title":"Unsupervised deep haar scattering on graphs","authors":"Xiuyuan Xu Chen, Stéphane Cheng, Mallat","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2018] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary dif- ferential equations. Advances in neural information pro- cessing systems, 31, 2018.","title":"Neural ordinary differential equations","authors":"Yulia Ricky Tq Chen, Jesse Rubanova, David K Bettencourt, Duvenaud","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Danz et al., 2021] Thomas Danz, Till Domröse, and Claus Ropers. Ultrafast nanoimaging of the order parameter in a structural phase transition. Science, 371(6527):371-374, 2021.","title":"Ultrafast nanoimaging of the order parameter in a structural phase transition","authors":"Thomas Danz, Till Domröse, Claus Ropers","year":"2021","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dupont et al., 2019] Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural odes. Advances in Neu- ral Information Processing Systems, 32, 2019.","title":"Augmented neural odes","authors":"Emilien Dupont, Arnaud Doucet, Yee Whye Teh","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fisher and Hasselblatt, 2019] Todd Fisher and Boris Has- selblatt. Hyperbolic flows. EMS Press, 2019.","title":"Hyperbolic flows","authors":"Todd Fisher, Boris","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fortunato, 2010] Santo Fortunato. Community detection in graphs. Physics reports, 486(3-5):75-174, 2010.","title":"Santo Fortunato. Community detection in graphs","authors":"Fortunato","year":"2010","venue":"Physics reports","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gao et al., 2016] Jianxi Gao, Baruch Barzel, and Albert- László Barabási. Universal resilience patterns in complex networks. Nature, 530(7590):307-312, 2016.","title":"Universal resilience patterns in complex networks","authors":"Jianxi Gao, Baruch Barzel, Albert-László Barabási","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gilbert, 1959] Edgar N Gilbert. Random graphs. The An- nals of Mathematical Statistics, 30(4):1141-1144, 1959.","title":"Random graphs","authors":"Gilbert Edgar","year":"1959","venue":"The Annals of Mathematical Statistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Guo et al., 2022] Jia-Nan Guo, Peiyan Zhang, C. Li, Xing Xie, Yan Zhang, and Sunghun Kim. Evolutionary prefer- ence learning via graph nested gru ode for session-based recommendation. Proceedings of the 31st ACM Interna- tional Conference on Information & Knowledge Manage- ment, 2022.","title":"Evolutionary preference learning via graph nested gru ode for session-based recommendation","authors":"Jia-Nan Guo, Peiyan Zhang, C Li, Xing Xie, Yan Zhang, Sunghun Kim","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hofmann et al., 2008] Thomas Hofmann, Bernhard Schölkopf, and Alexander J Smola. Kernel methods in machine learning. The annals of statistics, 36(3):1171- 1220, 2008.","title":"Kernel methods in machine learning","authors":"Thomas Hofmann, Bernhard Schölkopf, Alexander J Smola","year":"2008","venue":"The annals of statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hornik et al., 1989] Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359-366, 1989.","title":"Multilayer feedforward networks are universal approximators","authors":"Kurt Hornik, Maxwell Stinchcombe, Halbert White","year":"1989","venue":"Neural networks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hwang et al., 2021] JeeHyun Hwang, Jeongwhan Choi, Hwan-Kyu Choi, Kookjin Lee, Dongeun Lee, and Noseong Park. Climate modeling with neural diffusion equations. 2021 IEEE International Conference on Data Mining (ICDM), pages 230-239, 2021.","title":"Climate modeling with neural diffusion equations","authors":"Hwang","year":"2021","venue":"IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kim et al., 2021] Suyong Kim, Weiqi Ji, Sili Deng, Yingbo Ma, and Christopher Rackauckas. Stiff neural ordinary differential equations. Chaos: An Interdisciplinary Jour- nal of Nonlinear Science, 31(9):093122, 2021.","title":"Stiff neural ordinary differential equations","authors":"Suyong Kim, Weiqi Ji, Sili Deng, Yingbo Ma, Christopher Rackauckas","year":"2021","venue":"Chaos: An Interdisciplinary Journal of Nonlinear Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lancichinetti et al., 2008] Andrea Lancichinetti, Santo For- tunato, and Filippo Radicchi. Benchmark graphs for test- ing community detection algorithms. Physical review E, 78(4):046110, 2008.","title":"Santo Fortunato, and Filippo Radicchi. Benchmark graphs for testing community detection algorithms","authors":"Andrea Lancichinetti","year":"2008","venue":"Physical review E","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lee and Parish, 2021] Kookjin Lee and Eric J Parish. Pa- rameterized neural ordinary differential equations: Appli- cations to computational physics problems. Proceedings of the Royal Society A, 477(2253):20210162, 2021.","title":"Parameterized neural ordinary differential equations: Applications to computational physics problems","authors":"Lee, Parish, Kookjin Lee, Eric J Parish","year":"2021","venue":"Proceedings of the Royal Society A","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lemhadri et al., 2021] Ismael Lemhadri, Feng Ruan, and Rob Tibshirani. Lassonet: Neural networks with feature sparsity. In International Conference on Artificial Intelli- gence and Statistics, pages 10-18. PMLR, 2021.","title":"Lassonet: Neural networks with feature sparsity","authors":"Ismael Lemhadri, Feng Ruan, Rob Tibshirani","year":"2021","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Massaroli et al., 2020] Stefano Massaroli, Michael Poli, Jinkyoo Park, Atsushi Yamashita, and Hajime Asama. Dissecting neural odes. Advances in Neural Information Processing Systems, 33:3952-3963, 2020.","title":"Dissecting neural odes","authors":"Stefano Massaroli, Michael Poli, Jinkyoo Park, Atsushi Yamashita, Hajime Asama","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Milani and Nobili, 1987] Andrea Milani and Anna Maria Nobili. Integration error over very long time spans. Celes- tial mechanics, 43:1-34, 1987.","title":"Integration error over very long time spans","authors":"Andrea Milani, Anna Maria Nobili","year":"1987","venue":"Celestial mechanics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Murphy et al., 2022] Charles Murphy, Vincent Thibeault, Antoine Allard, and Patrick Desrosiers. Duality between predictability and reconstructability in complex systems. arXiv preprint arXiv:2206.04000, 2022.","title":"Duality between predictability and reconstructability in complex systems","authors":"Charles Murphy, Vincent Thibeault, Antoine Allard, Patrick Desrosiers","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Newman, 2018] Mark Newman. Networks. Oxford univer- sity press, 2018.","title":"Networks","authors":"Mark Newman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Takens, 1981] Floris Takens. Detecting strange attractors in turbulence. In Dynamical systems and turbulence, War- wick 1980, pages 366-381. Springer, 1981.","title":"Detecting strange attractors in turbulence","authors":"Floris Takens, Takens","year":"1981","venue":"Dynamical systems and turbulence","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2022] Zhongyang Wang, Junchang Xin, Qi Chen, Zhiqiong Wang, and Xinlei Wang. Ndcn-brain: An extensible dynamic functional brain network model. Diagnostics, 12, 2022.","title":"Ndcn-brain: An extensible dynamic functional brain network model","authors":"Zhongyang Wang, Junchang Xin, Qi Chen, Zhiqiong Wang, Xinlei Wang","year":"2022","venue":"Diagnostics","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Watts and Strogatz, 1998] Duncan J Watts and Steven H Strogatz. Collective dynamics of 'small-world' networks. nature, 393(6684):440-442, 1998.","title":"Collective dynamics of 'small-world' networks","authors":"J Duncan, Steven H Watts, Strogatz","year":"1998","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wolf et al., 1985] Alan Wolf, Jack B Swift, Harry L Swin- ney, and John A Vastano. Determining lyapunov expo- nents from a time series. Physica D: nonlinear phenom- ena, 16(3):285-317, 1985.","title":"Determining lyapunov exponents from a time series","authors":"Alan Wolf, Jack B Swift, Harry L Swinney, John A Vastano","year":"1985","venue":"Physica D: nonlinear phenomena","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yan et al., 2021] Xu Yan, Xiaoliang Fan, Peizhen Yang, Zonghan Wu, Shirui Pan, Longbiao Chen, Yu Zang, and Cheng Wang. Contig: Continuous representation learning on temporal interaction graphs. ArXiv, abs/2110.06088, 2021.","title":"Contig: Continuous representation learning on temporal interaction graphs","authors":"Xiaoliang Xu Yan, Peizhen Fan, Zonghan Yang, Shirui Wu, Longbiao Pan, Yu Chen, Cheng Zang, Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zaheer et al., 2017] Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017.","title":"Deep sets","authors":"Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, Alexander J Smola","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zang and Wang, 2020] Chengxi Zang and Fei Wang. Neu- ral dynamics on complex networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowl- edge Discovery & Data Mining, pages 892-902, 2020.","title":"Neural dynamics on complex networks","authors":"Wang ; Zang, Chengxi Zang, Fei Wang","year":"2020","venue":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2020] Han Zhang, Xi Gao, Jacob Unterman, and Tom Arodz. Approximation capabilities of neural odes and invertible residual networks. In International Confer- ence on Machine Learning, pages 11086-11095. PMLR, 2020.","title":"Approximation capabilities of neural odes and invertible residual networks","authors":"Han Zhang, Xi Gao, Jacob Unterman, Tom Arodz","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}