{"type":"ScholarlyArticle","title":"Real-World Networks Are Low-Dimensional: Theoretical and Practical Assessment","authors":[{"name":"Tobias Friedrich","slug":"tobias-friedrich","given":"Tobias","family":"Friedrich","affiliation":"Hasso Plattner Institute , University of Potsdam , Germany"},{"name":"Andreas Göbel","slug":"andreas-gobel","given":"Andreas","family":"Göbel","affiliation":"Hasso Plattner Institute , University of Potsdam , Germany"},{"name":"Maximilian Katzmann","slug":"maximilian-katzmann","given":"Maximilian","family":"Katzmann","affiliation":"Karlsruhe Institute of Technology , Germany"},{"name":"Leon Schiller","slug":"leon-schiller","given":"Leon","family":"Schiller","affiliation":"ETH , Zürich , Switzerland"}],"abstract":"Recent empirical evidence suggests that real-world networks have very low underlying dimensionality. We provide a theoretical explanation for this phenomenon as well as develop a linear-time algorithm for detecting the underlying dimensionality of such networks.\r\n  \r\n Our theoretical analysis considers geometric inhomogeneous random graphs (GIRGs), a geometric random graph model, which captures a variety of properties observed in real-world networks. These properties include a heterogeneous degree distribution and non-vanishing clustering coefficient, which is the probability that two random neighbors of a vertex are adjacent. Our first result shows that the clustering coefficient of GIRGs scales inverse exponentially with respect to the number of dimensions d, when the latter is at most logarithmic in n, the number of vertices. Hence, for a GIRG to behave like many real-world networks and have a non-vanishing clustering coefficient, it must come from a geometric space of o(log n) dimensions.\r\n  \r\n Our analysis on GIRGs allows us to obtain a linear-time algorithm for determining the dimensionality of a network. Our algorithm bridges the gap between theory and practice, as it comes with a rigorous proof of correctness and yields results comparable to prior empirical approaches, as indicated by our experiments on real-world instances. The efficiency of our algorithm makes it applicable to very large data-sets. We conclude that very low dimensionalities (from 1 to 10) are needed to explain properties of real-world networks.","doi":"10.24963/ijcai.2024/225","url":"https://ijcai.org/proceedings/2024/225","pdf":"https://ijcai.org/proceedings/2024/0225.pdf","markdown":"https://ijcai.org/proceedings/2024/225.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/225","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":225,"pages":{"first":2036,"last":2044},"keywords":["Data Mining: DM: Theoretical foundations of data mining","Data Mining: DM: Networks"],"videos":[],"references":[{"position":1,"raw":"[Aiello et al., 2001] William Aiello, Fan Chung, and Linyuan Lu. A random graph model for power law graphs. Experimental Mathematics, 10(1):53-66, 2001.","title":"A random graph model for power law graphs","authors":"William Aiello, Fan Chung, Linyuan Lu","year":"2001","venue":"Experimental Mathematics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Almagro et al., 2022] Pedro Almagro, Marián Boguñá, and M Ángeles Serrano. Detecting the ultra low dimensional- ity of real networks. Nature communications, 13(1):6096, 2022.","title":"Detecting the ultra low dimensionality of real networks","authors":"Pedro Almagro, Marián Boguñá, M Ángeles Serrano","year":"2022","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bläsius and Fischbeck, 2022] Thomas Bläsius and Philipp Fischbeck. On the external validity of average-case anal- yses of graph algorithms. In 30th Annual European Symposium on Algorithms (ESA 2022). Schloss Dagstuhl- Leibniz-Zentrum für Informatik, 2022.","title":"Thomas Bläsius and Philipp Fischbeck. On the external validity of average-case analyses of graph algorithms","authors":"Fischbeck Bläsius","year":"2022","venue":"30th Annual European Symposium on Algorithms (ESA 2022)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bläsius et al., 2022] Thomas Bläsius, Tobias Friedrich, Maximilian Katzmann, Ulrich Meyer, Manuel Penschuck, and Christopher Weyand. Efficiently generating geometric inhomogeneous and hyperbolic random graphs. Network Science, 10(4):361-380, 2022.","title":"Efficiently generating geometric inhomogeneous and hyperbolic random graphs","authors":"Thomas Bläsius, Tobias Friedrich, Maximilian Katzmann, Ulrich Meyer, Manuel Penschuck, Christopher Weyand","year":"2022","venue":"Network Science","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Boguna et al., 2010] Marian Boguna, Fragkiskos Pa- padopoulos, and Dmitri Krioukov. Sustaining the internet with hyperbolic mapping. Nature Communications, 1(1):62, Dec 2010. arXiv: 1009.0267.","title":"Marian Boguna, Fragkiskos Papadopoulos, and Dmitri Krioukov. Sustaining the internet with hyperbolic mapping","authors":"Boguna","year":"2010","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Boguna et al., 2021] Marian Boguna, Ivan Bonamassa, Manlio De Domenico, Shlomo Havlin, Dmitri Krioukov, and M Ángeles Serrano. Network geometry. Nature Re- views Physics, 3(2):114-135, 2021.","title":"Network geometry","authors":"Marian Boguna, Ivan Bonamassa, Manlio De Domenico, Shlomo Havlin, Dmitri Krioukov, M Ángeles Serrano","year":"2021","venue":"Nature Reviews Physics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Brennan et al., 2020] Matthew Brennan, Guy Bresler, and Dheeraj Nagaraj. Phase transitions for detecting latent ge- ometry in random graphs. Probability Theory and Related Fields, 178(3-4):1215-1289, 2020.","title":"Phase transitions for detecting latent geometry in random graphs","authors":"Matthew Brennan, Guy Bresler, Dheeraj Nagaraj","year":"2020","venue":"Probability Theory and Related Fields","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Brennan et al., 2024] Matthew Brennan, Guy Bresler, and Brice Huang. Threshold for detecting high dimensional geometry in anisotropic random geometric graphs. Ran- dom Structures & Algorithms, 64(1):125-137, 2024.","title":"Threshold for detecting high dimensional geometry in anisotropic random geometric graphs","authors":"Matthew Brennan, Guy Bresler, Brice Huang","year":"2024","venue":"Random Structures & Algorithms","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Bringmann et al., 2017] Karl Bringmann, Ralph Keusch, and Johannes Lengler. Sampling Geometric Inhomoge- neous Random Graphs in Linear Time. In 25th Annual European Symposium on Algorithms (ESA 2017), Leibniz International Proceedings in Informatics (LIPIcs), pages 20:1-20:15, 2017.","title":"Sampling Geometric Inhomogeneous Random Graphs in Linear Time","authors":"Bringmann","year":"2017","venue":"Leibniz International Proceedings in Informatics (LIPIcs)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Bubeck et al., 2016] Sébastien Bubeck, Jian Ding, Ronen Eldan, and Miklós Z Rácz. Testing for high-dimensional geometry in random graphs. Random Structures and Al- gorithms, 49(3):503-532, 2016.","title":"Testing for high-dimensional geometry in random graphs","authors":"Sébastien Bubeck, Jian Ding, Ronen Eldan, Miklós Z Rácz","year":"2016","venue":"Random Structures and Algorithms","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Cam, 1960] Lucien Le Cam. An approximation theorem for the poisson binomial distribution. Pacific Journal of Math- ematics, 10(4):1181-1197, Jan 1960.","title":"An approximation theorem for the poisson binomial distribution","authors":"Lucien Le, Cam","year":"1960","venue":"Pacific Journal of Mathematics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Camastra and Staiano, 2016] Francesco Camastra and An- tonino Staiano. Intrinsic dimension estimation: Advances and open problems. Information Sciences, 328:26-41, 2016.","title":"Francesco Camastra and Antonino Staiano. Intrinsic dimension estimation: Advances and open problems","authors":"Staiano Camastra","year":"2016","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Chung and Lu, 2002a] Fan Chung and Linyuan Lu. The av- erage distances in random graphs with given expected de- grees. Proceedings of the National Academy of Sciences, 99(25):15879-15882, 2002.","title":"The average distances in random graphs with given expected degrees","authors":"Chung, Lu ;, Fan Chung, Linyuan Lu","year":"2002","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Chung and Lu, 2002b] Fan Chung and Linyuan Lu. Con- nected Components in Random Graphs with Given Ex- pected Degree Sequences. Annals of Combinatorics, 6(25):125-145, 2002.","title":"Connected Components in Random Graphs with Given Expected Degree Sequences","authors":"Chung, Lu ;, Fan Chung, Linyuan Lu","year":"2002","venue":"Annals of Combinatorics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Dall and Christensen, 2002] Jesper Dall and Michael Chris- tensen. Random geometric graphs. Physical Review E, 66(1):016121, Jul 2002. arXiv:cond-mat/0203026.","title":"Jesper Dall and Michael Christensen. Random geometric graphs","authors":"Christensen Dall","year":"2002","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Devroye et al., 2011] Luc Devroye, András György, Gábor Lugosi, and Frederic Udina. High-dimensional random geometric graphs and their clique number. Electronic Journal of Probability, 16:2481-2508, Jan 2011.","title":"High-dimensional random geometric graphs and their clique number","authors":"Devroye","year":"2011","venue":"Electronic Journal of Probability","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Eldan and Mikulincer, 2020] Ronen Eldan and Dan Mikulincer. Information and dimensionality of anisotropic random geometric graphs. In Geometric Aspects of Func- tional Analysis: Israel Seminar (GAFA) 2017-2019 Volume I, pages 273-324. Springer, 2020.","title":"Information and dimensionality of anisotropic random geometric graphs","authors":"Mikulincer Eldan, Ronen Eldan, Dan Mikulincer","year":"2020","venue":"Geometric Aspects of Functional Analysis: Israel Seminar (GAFA) 2017-2019","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Faloutsos et al., 1999] Michalis Faloutsos, Petros Falout- sos, and Christos Faloutsos. On power-law relationships of the internet topology. ACM SIGCOMM computer com- munication review, 29(4):251-262, 1999.","title":"On power-law relationships of the internet topology","authors":"Michalis Faloutsos, Petros Faloutsos, Christos Faloutsos","year":"1999","venue":"ACM SIGCOMM computer communication review","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Friedrich et al., 2013] Tobias Friedrich, Thomas Sauer- wald, and Alexandre Stauffer. Diameter and broadcast time of random geometric graphs in arbitrary dimensions. Algorithmica, 67:65-88, 2013.","title":"Diameter and broadcast time of random geometric graphs in arbitrary dimensions","authors":"Friedrich","year":"2013","venue":"Algorithmica","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Friedrich et al., 2023a] Tobias Friedrich, Andreas Göbel, Maximilian Katzmann, and Leon Schiller. Cliques in high- dimensional geometric inhomogeneous random graphs. In Kousha Etessami, Uriel Feige, and Gabriele Puppis, ed- itors, 50th International Colloquium on Automata, Lan- guages, and Programming, ICALP, pages 62:1-62:13, 2023.","title":"Cliques in highdimensional geometric inhomogeneous random graphs","authors":"Friedrich","year":"2023","venue":"50th International Colloquium on Automata, Languages, and Programming, ICALP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Friedrich et al., 2023b] Tobias Friedrich, Andreas Göbel, Maximilian Katzmann, and Leon Schiller. A simple statis- tic for determining the dimensionality of complex net- works. CoRR, abs/2302.06357, 2023.","title":"A simple statistic for determining the dimensionality of complex networks","authors":"Friedrich","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Gu et al., 2021] Weiwei Gu, Aditya Tandon, Yong-Yeol Ahn, and Filippo Radicchi. Principled approach to the se- lection of the embedding dimension of networks. Nature Communications, 12(1):3772, 2021.","title":"Principled approach to the selection of the embedding dimension of networks","authors":"Weiwei Gu, Aditya Tandon, Yong-Yeol Ahn, Filippo Radicchi","year":"2021","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Gupta et al., 2014] Rishi Gupta, Tim Roughgarden, and Co- mandur Seshadhri. Decompositions of triangle-dense graphs. In Proceedings of the 5th conference on Inno- vations in theoretical computer science, pages 471-482, 2014.","title":"Decompositions of triangle-dense graphs","authors":"Rishi Gupta, Tim Roughgarden, Comandur Seshadhri","year":"2014","venue":"Proceedings of the 5th conference on Innovations in theoretical computer science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Keusch, 2018] Ralph Keusch. Geometric Inhomogeneous Random Graphs and Graph Coloring Games. Doctoral thesis, ETH Zurich, 2018.","title":"Geometric Inhomogeneous Random Graphs and Graph Coloring Games","authors":"Ralph Keusch","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Krioukov, 2016] Dmitri Krioukov. Clustering implies geometry in networks. Physical review letters, 116(20):208302, 2016.","title":"Dmitri Krioukov. Clustering implies geometry in networks","authors":"Krioukov","year":"2016","venue":"Physical review letters","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"Leskovec and Krevl, ] Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data. Accessed: 2024-01-12.","title":"SNAP Datasets: Stanford large network dataset collection","authors":"Krevl Leskovec, Jure Leskovec, Andrej Krevl","year":"2024","venue":null,"doi":null,"url":"http://snap.stanford.edu/data","internal_url":null},{"position":27,"raw":"[Levina and Bickel, 2004] Elizaveta Levina and Peter Bickel. Maximum likelihood estimation of intrinsic dimension. Advances in neural information processing systems, 17, 2004.","title":"Maximum likelihood estimation of intrinsic dimension","authors":"Bickel ; ] Elizaveta Levina, Peter Levina, Bickel","year":"2004","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Liu and Rácz, 2023a] Suqi Liu and Miklós Z Rácz. Phase transition in noisy high-dimensional random geometric graphs. Electronic Journal of Statistics, 17(2):3512-3574, 2023.","title":"Phase transition in noisy high-dimensional random geometric graphs","authors":"Rácz Liu, Suqi Liu, Miklós Z Rácz","year":"2023","venue":"Electronic Journal of Statistics","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Liu and Rácz, 2023b] Suqi Liu and Miklós Z Rácz. A prob- abilistic view of latent space graphs and phase transitions. Bernoulli, 29(3):2417-2441, 2023.","title":"A probabilistic view of latent space graphs and phase transitions","authors":"Rácz Liu, Suqi Liu, Miklós Z Rácz","year":"2023","venue":"Bernoulli","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Liu et al., 2022] Siqi Liu, Sidhanth Mohanty, Tselil Schramm, and Elizabeth Yang. Testing thresholds for high-dimensional sparse random geometric graphs. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2022, page 672-677, New York, NY, USA, 2022. Association for Computing Machinery.","title":"Testing thresholds for high-dimensional sparse random geometric graphs","authors":"Siqi Liu, Sidhanth Mohanty, Tselil Schramm, Elizabeth Yang","year":"2022","venue":"Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2022","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Livadiotis, 2014] George Livadiotis. Chi-p distribution: characterization of the goodness of the fitting using lp- norms. Journal of Statistical Distributions and Applica- tions, 1(1):4, Jun 2014.","title":"Chi-p distribution: characterization of the goodness of the fitting using lpnorms","authors":"George Livadiotis, Livadiotis","year":"2014","venue":"Journal of Statistical Distributions and Applications","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[McDiarmid, 1989] Colin McDiarmid. On the method of bounded differences. Surveys in combinatorics, 141(1):148-188, 1989.","title":"On the method of bounded differences","authors":"Colin Mcdiarmid","year":"1989","venue":"Surveys in combinatorics","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Michielan et al., 2022] Riccardo Michielan, Nelly Litvak, and Clara Stegehuis. Detecting hyperbolic geometry in networks: Why triangles are not enough. Physical Review E, 106(5):054303, 2022.","title":"Detecting hyperbolic geometry in networks: Why triangles are not enough","authors":"Riccardo Michielan, Nelly Litvak, Clara Stegehuis","year":"2022","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Nickel and Kiela, 2017] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical rep- resentations. Advances in neural information processing systems, 30, 2017.","title":"Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Rossi and Ahmed, 2015] Ryan A. Rossi and Nesreen K. Ahmed. The network data repository with interactive graph analytics and visualization. In AAAI, 2015.","title":"The network data repository with interactive graph analytics and visualization","authors":"Ryan A Rossi, Nesreen K Ahmed","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Sarveniazi, 2014] Alireza Sarveniazi. An actual survey of dimensionality reduction. American Journal of Computa- tional Mathematics, 2014, 2014.","title":"An actual survey of dimensionality reduction","authors":"Alireza Sarveniazi","year":"2014","venue":"American Journal of Computational Mathematics","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[van der Hofstad et al., 2017] Remco van der Hofstad, A. J. E. M. Janssen, Johan S. H. van Leeuwaarden, and Clara Stegehuis. Local clustering in scale-free networks with hidden variables. Physical Review E, 95(2):022307, Feb 2017.","title":"Local clustering in scale-free networks with hidden variables","authors":"Hofstad Van Der","year":"2017","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Warnke, 2016] Lutz Warnke. On the method of typical bounded differences. Combinatorics, Probability and Computing, 25:269-299, 2016.","title":"On the method of typical bounded differences","authors":"Lutz Warnke","year":"2016","venue":"Combinatorics, Probability and Computing","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yin and Shen, 2018] Zi Yin and Yuanyuan Shen. On the di- mensionality of word embedding. Advances in neural in- formation processing systems, 31, 2018.","title":"On the dimensionality of word embedding","authors":"Yin, Shen ; ] Zi Yin, Yuanyuan Shen","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}