{"type":"ScholarlyArticle","title":"A Property Testing Framework for the Theoretical Expressivity of Graph Kernels","authors":[{"name":"Nils M. Kriege","slug":"nils-m-kriege","given":"Nils M.","family":"Kriege","affiliation":"TU Dortmund University , Dortmund , Germany"},{"name":"Christopher Morris","slug":"christopher-morris","given":"Christopher","family":"Morris","affiliation":"TU Dortmund University , Dortmund , Germany"},{"name":"Anja Rey","slug":"anja-rey","given":"Anja","family":"Rey","affiliation":"TU Dortmund University , Dortmund , Germany"},{"name":"Christian Sohler","slug":"christian-sohler","given":"Christian","family":"Sohler","affiliation":"TU Dortmund University , Dortmund , Germany"}],"abstract":"Graph kernels are applied heavily for the classification of structured data. However, their expressivity is assessed almost exclusively from experimental studies and there is no theoretical justification why one kernel is in general preferable over another. We introduce a theoretical framework for investigating the expressive power of graph kernels, which is inspired by concepts from the area of property testing. We introduce the notion of distinguishability of a graph property by a graph kernel. For several established graph kernels we show that they cannot distinguish essential graph properties. In order to overcome this, we consider a kernel based on k-disc frequencies. We show that this efficiently computable kernel can distinguish fundamental graph properties. Finally, we obtain learning guarantees for nearest neighbor classifiers in our framework.","doi":"10.24963/ijcai.2018/325","url":"https://ijcai.org/proceedings/2018/325","pdf":"https://ijcai.org/proceedings/2018/0325.pdf","markdown":"https://ijcai.org/proceedings/2018/325.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/325","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":325,"pages":{"first":2348,"last":2354},"keywords":["Machine Learning: Classification","Machine Learning: Learning Theory","Machine Learning: Kernel Methods"],"videos":[],"references":[{"position":1,"raw":"[Arvind et al., 2015] V. Arvind, Johannes Köbler, Gaurav Rat- tan, and Oleg Verbitsky. On the power of color refinement. In Proceedings of the 20th International Symposium on Funda- mentals of Computation Theory, pages 339-350, 2015.","title":"On the power of color refinement","authors":"V Arvind, Johannes Köbler, Gaurav Rattan, Oleg Verbitsky","year":"2015","venue":"Proceedings of the 20th International Symposium on Fundamentals of Computation Theory","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Babai and Kucera, 1979] Laszlo Babai and Ludik Kucera. Canonical labelling of graphs in linear average time. In Proceedings of the 20th Annual Symposium on Foundations of Computer Science, pages 39-46, 1979.","title":"Canonical labelling of graphs in linear average time","authors":"Kucera ; ] Laszlo Babai, Ludik Babai, Kucera","year":"1979","venue":"Proceedings of the 20th Annual Symposium on Foundations of Computer Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Benjamini et al., 2010] Itai Benjamini, Oded Schramm, and Asaf Shapira. Every minor-closed property of sparse graphs is testable. Advances in Mathematics, 223(6):2200-2218, 2010.","title":"Every minor-closed property of sparse graphs is testable","authors":"Itai Benjamini, Oded Schramm, Asaf Shapira","year":"2010","venue":"Advances in Mathematics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Borgwardt and Kriegel, 2005] Karsten M. Borgwardt and Hans-Peter Kriegel. Shortest-path kernels on graphs. In Proceedings of the 5th IEEE International Conference on Data Mining, pages 74-81, 2005.","title":"Shortest-path kernels on graphs","authors":"Karsten M Borgwardt, Hans-Peter Kriegel","year":"2005","venue":"Proceedings of the 5th IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cai et al., 1992] Jin-yi Cai, Martin Fürer, and Neil Immerman. An optimal lower bound on the number of variables for graph identification. Combinatorica, 12(4):389-410, 1992.","title":"An optimal lower bound on the number of variables for graph identification","authors":"Jin-Yi Cai, Martin Fürer, Neil Immerman","year":"1992","venue":"Combinatorica","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Costa and De Grave, 2010] Fabrizio Costa and Kurt De Grave. Fast neighborhood subgraph pairwise distance kernel. In Proceedings of the 26th International Conference on Machine Learning, pages 255-262, 2010.","title":"Fast neighborhood subgraph pairwise distance kernel","authors":"Fabrizio Costa, Kurt De, Grave","year":"2010","venue":"Proceedings of the 26th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gärtner et al., 2003] Thomas Gärtner, Peter Flach, and Stefan Wrobel. On graph kernels: Hardness results and efficient alternatives. In Proceedings of 16th Annual Conference on Learning Theory and Kernel Machines, pages 129-143, 2003.","title":"On graph kernels: Hardness results and efficient alternatives","authors":"Thomas Gärtner, Peter Flach, Stefan Wrobel","year":"2003","venue":"Proceedings of 16th Annual Conference on Learning Theory and Kernel Machines","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goldreich and Ron, 2002] Oded Goldreich and Dana Ron. Property testing in bounded degree graphs. Algorithmica, 32:302-343, 2002.","title":"Property testing in bounded degree graphs","authors":"Ron ; Goldreich, Oded Goldreich, Dana Ron","year":"2002","venue":"Algorithmica","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goldreich et al., 1998] Oded Goldreich, Shari Goldwasser, and Dana Ron. Property testing and its connection to learning and approximation. Journal of the ACM, 45(4):653-750, 1998.","title":"Property testing and its connection to learning and approximation","authors":"Oded Goldreich, Shari Goldwasser, Dana Ron","year":"1998","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goldreich, 2017] Oded Goldreich. Introduction to Property Testing. Cambridge University Press, 2017.","title":"Introduction to Property Testing","authors":"Oded Goldreich, Goldreich","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Haussler, 1999] David Haussler. Convolution kernels on dis- crete structures. Technical Report UCS-CRL-99-10, Univer- sity of California at Santa Cruz, 1999.","title":"Convolution kernels on discrete structures","authors":"David Haussler","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hido and Kashima, 2009] Shohei Hido and Hisashi Kashima. A linear-time graph kernel. In Proceedings of the 9th IEEE In- ternational Conference on Data Mining, pages 179-188, 2009.","title":"A linear-time graph kernel","authors":"Kashima ; ] Shohei Hido, Hisashi Hido, Kashima","year":"2009","venue":"Proceedings of the 9th IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Johansson and Dubhashi, 2015] Fredrik Johansson and Dub- hashi Dubhashi. Learning with similarity functions on graphs using matchings of geometric embeddings. In Proceedings of the 21st ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, pages 467-476, 2015.","title":"Fredrik Johansson and Dubhashi Dubhashi. Learning with similarity functions on graphs using matchings of geometric embeddings","authors":"Dubhashi Johansson","year":"2015","venue":"Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kashima et al., 2003] Hisashi Kashima, Koji Tsuda, and Aki- hiro Inokuchi. Marginalized kernels between labeled graphs. In Proceedings of the 20th International Conference on Ma- chine Learning, pages 321-328, 2003.","title":"Marginalized kernels between labeled graphs","authors":"Hisashi Kashima, Koji Tsuda, Akihiro Inokuchi","year":"2003","venue":"Proceedings of the 20th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kersting et al., 2016] Kristian Kersting, Nils M. Kriege, Christopher Morris, Petra Mutzel, and Marion Neumann. Benchmark data sets for graph kernels, 2016. http: //graphkernels.cs.tu-dortmund.de. [Kiefer et al., 2015] Sandra Kiefer, Pascal Schweitzer, and Er- kal Selman. Graphs identified by logics with counting. In Proceedings of the 40th International Symposium on Mathema- tical Foundations of Computer Science, pages 319-330, 2015.","title":"Graphs identified by logics with counting","authors":"Kristian Kersting, Nils M Kriege, Christopher Morris, Petra Mutzel, Marion Neumann, ; Kiefer","year":"2015","venue":"Proceedings of the 40th International Symposium on Mathematical Foundations of Computer Science","doi":null,"url":"http://graphkernels.cs.tu-dortmund","internal_url":null},{"position":16,"raw":"[Kondor and Pan, 2016] Risi Kondor and Horace Pan. The multiscale laplacian graph kernel. In Advances in Neural Information Processing Systems, pages 2982-2990, 2016.","title":"The multiscale laplacian graph kernel","authors":"Risi Kondor, Horace Pan","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kriege et al., 2014] Nils Kriege, Marion Neumann, Kristian Kersting, and Petra Mutzel. Explicit versus implicit graph feature maps: A computational phase transition for walk kernels. In Proceedings of the 14th IEEE International Conference on Data Mining, pages 881-886, 2014.","title":"Explicit versus implicit graph feature maps: A computational phase transition for walk kernels","authors":"Kriege","year":"2014","venue":"Proceedings of the 14th IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kriege et al., 2016] Nils M. Kriege, Giscard. Pierre-Louis, and Richard C. Wilson. On valid optimal assignment kernels and applications to graph classification. In Advances in Neural Information Processing Systems, pages 1615-1623, 2016.","title":"On valid optimal assignment kernels and applications to graph classification","authors":"Nils M Kriege, Giscard Pierre-Louis, Richard C Wilson","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Morris et al., 2016] Christopher Morris, Nils M. Kriege, Kris- tian Kersting, and Petra Mutzel. Faster kernel for graphs with continuous attributes via hashing. In Proceedings of the 16th IEEE Int Conference on Data Mining, pages 1095-1100, 2016.","title":"Faster kernel for graphs with continuous attributes via hashing","authors":"Christopher Morris, Nils M Kriege, Kristian Kersting, Petra Mutzel","year":"2016","venue":"Proceedings of the 16th IEEE Int Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Morris et al., 2017] Christopher Morris, Kristian Kersting, and Petra Mutzel. Glocalized Weisfeiler-Lehman graph kernels: Global-local feature maps of graphs. In Proceedings of 17th IEEE Int Conference on Data Mining, pages 327-336, 2017.","title":"Glocalized Weisfeiler-Lehman graph kernels: Global-local feature maps of graphs","authors":"Christopher Morris, Kristian Kersting, Petra Mutzel","year":"2017","venue":"Proceedings of 17th IEEE Int Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Newman and Sohler, 2013] Ilan Newman and Christian Sohler. Every property of hyperfinite graphs is testable. SIAM Journal on Computing, 42(3):1095-1112, 2013.","title":"Every property of hyperfinite graphs is testable","authors":"Sohler Newman, Ilan Newman, Christian Sohler","year":"2013","venue":"SIAM Journal on Computing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Oneto et al., 2017] Luca Oneto, Nicol Navarin, Michele Donini, Alessandro Sperduti, Fabio Aiolli, and Davide An- guita. Measuring the expressivity of graph kernels through statistical learning theory. Neurocomputing, 2017.","title":"Measuring the expressivity of graph kernels through statistical learning theory","authors":"Luca Oneto, Nicol Navarin, Michele Donini, Alessandro Sperduti, Fabio Aiolli, Davide Anguita","year":"2017","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ramon and Gärtner, 2003] Jan Ramon and Thomas Gärtner. Expressivity versus efficiency of graph kernels. In Proceed- ings of the 1st International Workshop on Mining Graphs, Trees and Sequences, 2003.","title":"Expressivity versus efficiency of graph kernels","authors":"Ramon, Gärtner, Jan Ramon, Thomas Gärtner","year":"2003","venue":"Proceedings of the 1st International Workshop on Mining Graphs, Trees and Sequences","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shervashidze et al., 2009] Nino Shervashidze, S. V. N. Vish- wanathan, Tobias H. Petri, Kurt Mehlhorn, and Karsten M. Borgwardt. Efficient graphlet kernels for large graph compari- son. In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, pages 488-495, 2009.","title":"Efficient graphlet kernels for large graph comparison","authors":"S V N Nino Shervashidze, Tobias H Vishwanathan, Kurt Petri, Karsten M Mehlhorn, Borgwardt","year":"2009","venue":"Proceedings of the 12th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shervashidze et al., 2011] Nino Shervashidze, Pascal Schweitzer, Erik Jan van Leeuwen, Kurt Mehlhorn, and Karsten M. Borgwardt. Weisfeiler-Lehman graph kernels. Journal of Machine Learning Research, 12:2539-2561, 2011.","title":"Weisfeiler-Lehman graph kernels","authors":"Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, Karsten M Borgwardt","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sugiyama and Borgwardt, 2015] Mahito Sugiyama and Karsten M. Borgwardt. Halting in random walk kernels. In Advances in Neural Information Processing Systems, pages 1639-1647, 2015.","title":"Halting in random walk kernels","authors":"Borgwardt Sugiyama, Mahito Sugiyama, Karsten M Borgwardt","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vishwanathan et al., 2010] S. V. N. Vishwanathan, Nicol N. Schraudolph, Risi Kondor, and Karsten M. Borgwardt. Graph kernels. Journal of Machine Learning Research, 11:1201- 1242, 2010.","title":"Graph kernels","authors":"S V N Vishwanathan, Nicol N Schraudolph, Risi Kondor, Karsten M Borgwardt","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yu et al., 2002] Kai Yu, Liang Ji, and Xuegong Zhang. Ker- nel nearest-neighbor algorithm. Neural Processing Letters, 15(2):147-156, 2002.","title":"Kernel nearest-neighbor algorithm","authors":"Kai Yu, Liang Ji, Xuegong Zhang","year":"2002","venue":"Neural Processing Letters","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}