{"type":"ScholarlyArticle","title":"Expanding the Hyperbolic Kernels: A Curvature-aware Isometric Embedding View","authors":[{"name":"Meimei Yang","slug":"meimei-yang","given":"Meimei","family":"Yang","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"},{"name":"Pengfei Fang","slug":"pengfei-fang","given":"Pengfei","family":"Fang","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"},{"name":"Hui Xue","slug":"hui-xue","given":"Hui","family":"Xue","affiliation":"School of Computer Science and Engineering , Southeast University , Nanjing 210096 , China; MOE Key Laboratory of Computer Network and Information Integration ( Southeast University) , China"}],"abstract":"Modeling data relation as a hierarchical structure has proven beneficial for many learning scenarios, and the hyperbolic space, with negative curvature, can encode such data hierarchy without distortion. Several recent studies also show that the representation power of the hyperbolic space can be further improved by endowing the kernel methods. Unfortunately, the known kernel methods, developed in hyperbolic space, are limited by the adaptation capacity or distortion issues. This paper addresses the issues through a novel embedding function. To this end, we propose a curvature-aware isometric embedding, which establishes an isometry from the Poincar\\'e model to a special reproducing kernel Hilbert space (RKHS). Then we can further define a series of kernels on this RKHS, including several positive definite kernels and an indefinite kernel. Thorough experiments are conducted to demonstrate the superiority of our proposals over existing-known hyperbolic and Euclidean kernels in various learning tasks, e.g., graph learning and zero-shot learning.","doi":"10.24963/ijcai.2023/497","url":"https://ijcai.org/proceedings/2023/497","pdf":"https://ijcai.org/proceedings/2023/0497.pdf","markdown":"https://ijcai.org/proceedings/2023/497.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/497","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":497,"pages":{"first":4469,"last":4477},"keywords":["Machine Learning: ML: Geometric learning","Machine Learning: ML: Kernel methods"],"videos":[],"references":[{"position":1,"raw":"[Akata et al., 2016] Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. Label-embedding for image classification. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 38(7):1425-1438, July 2016.","title":"Label-embedding for image classification","authors":"Zeynep Akata, Florent Perronnin, Zaid Harchaoui, Cordelia Schmid","year":"2016","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arcozzi et al., 2007] Nicola Arcozzi, Richard Rochberg, and Eric Sawyer. The diameter space, a restriction of the drury-arveson-hardy space. In Proceedings of the Confer- ence on Function Spaces, pages 21-42. Providence, R.I. : American Mathematical Society, January 2007.","title":"The diameter space, a restriction of the drury-arveson-hardy space","authors":"Nicola Arcozzi, Richard Rochberg, Eric Sawyer","year":"2007","venue":"Proceedings of the Conference on Function Spaces","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bellet et al., 2015] Aurélien Bellet, Amaury Habrard, and Marc Sebban. Metric learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 9(1):1-151, 2015.","title":"Metric learning","authors":"Aurélien Bellet, Amaury Habrard, Marc Sebban","year":"2015","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bishop and Nasrabadi, 2006] Christopher M Bishop and Nasser M Nasrabadi. Pattern Recognition and Machine Learning. Number 4. Springer, 2006.","title":"Pattern Recognition and Machine Learning","authors":"M Christopher, Bishop, M Nasser, Nasrabadi","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bojchevski and Günnemann, 2017] Aleksandar Bojchevski and Stephan Günnemann. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. arXiv preprint arXiv:1707.03815, 2017.","title":"Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking","authors":"Aleksandar Bojchevski, Stephan Günnemann","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cannon et al., 1997] James W Cannon, William J Floyd, Richard Kenyon, Walter R Parry, et al. Hyperbolic ge- ometry. Flavors of Geometry, 31:59-115, 1997.","title":"Hyperbolic geometry","authors":"William J James W Cannon, Richard Floyd, Kenyon, Walter R Parry","year":"1997","venue":"Flavors of Geometry","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chami et al., 2019] Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems, pages 4869-4880, 2019.","title":"Hyperbolic graph convolutional neural networks","authors":"Ines Chami, Zhitao Ying, Christopher Ré, Jure Leskovec","year":"2019","venue":"Proceedings of the Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2018] Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, and Shih-Fu Chang. Zero-shot visual recog- nition using semantics-preserving adversarial embedding networks. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 1043-1052, 2018.","title":"Zero-shot visual recognition using semantics-preserving adversarial embedding networks","authors":"Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, Shih-Fu Chang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cho et al., 2019] Hyunghoon Cho, Benjamin DeMeo, Jian Peng, and Bonnie Berger. Large-margin classification in hyperbolic space. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 1832-1840, 2019.","title":"Large-margin classification in hyperbolic space","authors":"Cho","year":"2019","venue":"Proceedings of the International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Coxeter and Greitzer, 1967] Harold Scott Macdonald Cox- eter and Samuel L Greitzer. Geometry revisited, vol- ume 19. Maa, 1967.","title":"Harold Scott Macdonald Coxeter and Samuel L Greitzer","authors":"Greitzer Coxeter","year":"1967","venue":"Geometry revisited","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Cucerzan, 2007] Silviu Cucerzan. Large-scale named en- tity disambiguation based on wikipedia data. In Proceed- ings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 708-716, 2007.","title":"Large-scale named entity disambiguation based on wikipedia data","authors":"Silviu Cucerzan","year":"2007","venue":"Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Duren and Weir, 2007] Peter Duren and Rachel Weir. The pseudohyperbolic metric and bergman spaces in the ball. Transactions of the American Mathematical Society, 359(1):63-76, 2007.","title":"The pseudohyperbolic metric and bergman spaces in the ball","authors":"Weir Duren, Peter Duren, Rachel Weir","year":"2007","venue":"Transactions of the American Mathematical Society","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Fang et al., 2021] Pengfei Fang, Mehrtash Harandi, and Lars Petersson. Kernel methods in hyperbolic spaces. In Proceedings of the IEEE International Conference on Computer Vision, pages 10665-10674, October 2021.","title":"Kernel methods in hyperbolic spaces","authors":"Pengfei Fang, Mehrtash Harandi, Lars Petersson","year":"2021","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Fang et al., 2023] Pengfei Fang, Mehrtash Harandi, Trung Le, and Dinh Phung. Hyperbolic geometry in computer vision: A survey. arXiv preprint arXiv:2304.10764, 2023.","title":"Hyperbolic geometry in computer vision: A survey","authors":"Pengfei Fang, Mehrtash Harandi, Trung Le, Dinh Phung","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Frome et al., 2013] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. Devise: A deep visual-semantic em- bedding model. Advances in neural information process- ing systems, 26, 2013.","title":"Devise: A deep visual-semantic embedding model","authors":"Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'aurelio Ranzato, Tomas Mikolov","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ganea et al., 2018a] Octavian Ganea, Gary Becigneul, and Thomas Hofmann. Hyperbolic entailment cones for learn- ing hierarchical embeddings. In Proceedings of the Inter- national Conference on Machine Learning, pages 1632- 1641, 2018.","title":"Hyperbolic entailment cones for learning hierarchical embeddings","authors":"Ganea","year":"2018","venue":"Proceedings of the International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ganea et al., 2018b] Octavian Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic neural networks. In Pro- ceedings of the Advances in Neural Information Process- ing Systems, volume 31, pages 5345-5355, 2018.","title":"Hyperbolic neural networks","authors":"Ganea","year":"2018","venue":"Proceedings of the Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gulcehre et al., 2018] Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, et al. Hyperbolic attention networks. arXiv preprint arXiv:1805.09786, 2018.","title":"Hyperbolic attention networks","authors":"Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jayasumana et al., 2015] Sadeep Jayasumana, Richard Hartley, Mathieu Salzmann, Hongdong Li, and Mehrtash Harandi. Kernel methods on riemannian manifolds with gaussian RBF kernels. PAMI, 2015.","title":"Kernel methods on riemannian manifolds with gaussian RBF kernels","authors":"Jayasumana","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Khrulkov et al., 2020] Valentin Khrulkov, Leyla Mir- vakhabova, Evgeniya Ustinova, Ivan Oseledets, and Victor Lempitsky. Hyperbolic image embeddings. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6418-6428, 2020.","title":"Hyperbolic image embeddings","authors":"Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan Oseledets, Victor Lempitsky","year":"2020","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lampert et al., 2013] Christoph H Lampert, Hannes Nick- isch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 36(3):453-465, 2013.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2019] Kai Li, Martin Renqiang Min, and Yun Fu. Rethinking zero-shot learning: A conditional visual clas- sification perspective. In Proceedings of the IEEE Interna- tional Conference on Computer Vision, pages 3583-3592, 2019.","title":"Rethinking zero-shot learning: A conditional visual classification perspective","authors":"Kai Li, Martin Renqiang Min, Yun Fu","year":"2019","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Loosli et al., 2015] Gaëlle Loosli, Stéphane Canu, and Cheng Soon Ong. Learning SVM in Kreȋn spaces. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 38(6):1204-1216, 2015.","title":"Learning SVM in Kreȋn spaces","authors":"Gaëlle Loosli, Stéphane Canu, Soon Cheng, Ong","year":"2015","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nickel and Kiela, 2017] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical rep- resentations. In Proceedings of the Advances in Neural Information Processing Systems, pages 6338-6347, 2017.","title":"Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"Proceedings of the Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ong et al., 2004] Cheng Soon Ong, Xavier Mary, Stéphane Canu, and Alexander J Smola. Learning with non-positive kernels. In Proceedings of the International Conference on Machine Learning, page 81. ACM, 2004.","title":"Learning with non-positive kernels","authors":"Soon Cheng, Xavier Ong, Stéphane Mary, Alexander J Canu, Smola","year":"2004","venue":"Proceedings of the International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Platt and others, 1999] John Platt et al. Probabilistic out- puts for support vector machines and comparisons to reg- ularized likelihood methods. Advances in Large Margin Classifiers, 10(3):61-74, 1999.","title":"Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods","authors":"John Platt","year":"1999","venue":"Advances in Large Margin Classifiers","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ratcliffe, 1994] John G Ratcliffe. Foundations of Hyper- bolic Manifolds. Springer, 1994.","title":"Foundations of Hyperbolic Manifolds","authors":"John G Ratcliffe","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Rochberg, 2019] Richard Rochberg. Complex hyperbolic geometry and hilbert spaces with complete pick kernels. Journal of Functional Analysis, 276(5):1622 -1679, 2019.","title":"Complex hyperbolic geometry and hilbert spaces with complete pick kernels","authors":"Richard Rochberg","year":"2019","venue":"Journal of Functional Analysis","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rozemberczki et al., 2019] Benedek Rozemberczki, Ryan Davies, Rik Sarkar, and Charles Sutton. Gemsec: Graph embedding with self clustering. In Proceedings of the IEEE/ACM International Conference on Advances in So- cial Networks Analysis and Mining, pages 65-72. ACM, 2019.","title":"Gemsec: Graph embedding with self clustering","authors":"Ryan Benedek Rozemberczki, Rik Davies, Charles Sarkar, Sutton","year":"2019","venue":"Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Shchur et al., 2018] Oleksandr Shchur, Maximil- ian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868, 2018.","title":"Pitfalls of graph neural network evaluation","authors":"Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, Stephan Günnemann","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shimizu et al., 2020] Ryohei Shimizu, Yusuke Mukuta, and Tatsuya Harada. Hyperbolic neural networks++. arXiv preprint arXiv:2006.08210, 2020.","title":"Hyperbolic neural networks++","authors":"Ryohei Shimizu, Yusuke Mukuta, Tatsuya Harada","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Smola and Schölkopf, 1998] Alex J Smola and Bernhard Schölkopf. Learning with kernels, volume 4. Citeseer, 1998.","title":"Learning with kernels","authors":"Alex J Smola, Bernhard Schölkopf","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ungar, 1998] Abraham A Ungar. From pythagoras to ein- stein: the hyperbolic pythagorean theorem. Foundations of physics, 28(8):1283-1321, 1998.","title":"From pythagoras to einstein: the hyperbolic pythagorean theorem","authors":"A Abraham, Ungar","year":"1998","venue":"Foundations of physics","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Ungar, 2022] Abraham Albert Ungar. Analytic hyperbolic geometry and albert einstein's special theory of relativity , world sci. Publ. Co., Singapore, 2022.","title":"Analytic hyperbolic geometry and albert einstein's special theory of relativity","authors":"Abraham Albert, Ungar","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. Technical Report CNS-TR- 2010-001, California Institute of Technology, 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xia et al., 2021] Feng Xia, Ke Sun, Shuo Yu, Abdul Aziz, Liangtian Wan, Shirui Pan, and Huan Liu. Graph learning: A survey. IEEE Transactions on Artificial Intelligence, 2(2):109-127, 2021.","title":"Graph learning: A survey","authors":"Feng Xia, Ke Sun, Shuo Yu, Abdul Aziz, Liangtian Wan, Shirui Pan, Huan Liu","year":"2021","venue":"IEEE Transactions on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xian et al., 2016] Yongqin Xian, Zeynep Akata, Gaurav Sharma, Quynh Nguyen, Matthias Hein, and Bernt Schiele. Latent embeddings for zero-shot classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 69-77, 2016.","title":"Latent embeddings for zero-shot classification","authors":"Xian","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xian et al., 2018] Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning- a comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2251-2265, 2018.","title":"Zero-shot learninga comprehensive evaluation of the good, the bad and the ugly","authors":"Xian","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Xu et al., 2017] Haiming Xu, Hui Xue, Xiaohong Chen, and Yunyun Wang. Solving indefinite kernel support vec- tor machine with difference of convex functions program- ming. In Proceedings of the AAAI Conference on Artificial Intelligence, 2017.","title":"Solving indefinite kernel support vector machine with difference of convex functions programming","authors":"Haiming Xu, Hui Xue, Xiaohong Chen, Yunyun Wang","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang and Shi, 2019] Fei Zhang and Guangming Shi. Co- representation network for generalized zero-shot learning. In Proceedings of the International Conference on Ma- chine Learning, pages 7434-7443. PMLR, 2019.","title":"Corepresentation network for generalized zero-shot learning","authors":"Shi Zhang, Fei Zhang, Guangming Shi","year":"2019","venue":"Proceedings of the International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang et al., 2017] Li Zhang, Tao Xiang, and Shaogang Gong. Learning a deep embedding model for zero-shot learning. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 2021-2030, 2017.","title":"Learning a deep embedding model for zero-shot learning","authors":"Li Zhang, Tao Xiang, Shaogang Gong","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhao et al., 2006] Bin Zhao, Prithviraj Sen, and Lise Getoor. Entity and elationship labeling in affiliation net- works. In ICML Workshop on Statistical Network Analy- sis, 2006.","title":"Entity and elationship labeling in affiliation networks","authors":"Zhao","year":"2006","venue":"ICML Workshop on Statistical Network Analysis","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}