{"type":"ScholarlyArticle","title":"Clustering-Based Relational Unsupervised Representation Learning with an Explicit Distributed Representation","authors":[{"name":"Sebastijan Dumancic","slug":"sebastijan-dumancic","given":"Sebastijan","family":"Dumancic","affiliation":"Computer Science Department , KU Leuven , Belgium"},{"name":"Hendrik Blockeel","slug":"hendrik-blockeel","given":"Hendrik","family":"Blockeel","affiliation":"Computer Science Department , KU Leuven , Belgium"}],"abstract":"The goal of unsupervised representation learning is to extract a new representation of data, such that solving many different tasks becomes easier. Existing methods typically focus on vectorized data and offer little support for relational data, which additionally describes relationships among instances. In this work we introduce an approach for relational unsupervised representation learning. Viewing a relational dataset as a hypergraph, new features are obtained by clustering vertices and hyperedges. To find a representation suited for many relational learning tasks, a wide range of similarities between relational objects is considered, e.g. feature and structural similarities. We experimentally evaluate the proposed approach and show that models learned on such latent representations perform better, have lower complexity, and outperform the existing approaches on classification tasks.","doi":"10.24963/ijcai.2017/226","url":"https://ijcai.org/proceedings/2017/226","pdf":"https://ijcai.org/proceedings/2017/0226.pdf","markdown":"https://ijcai.org/proceedings/2017/226.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/226","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":226,"pages":{"first":1631,"last":1637},"keywords":["Machine Learning: Relational Learning","Machine Learning: Machine Learning","Machine Learning: Deep Learning","Machine Learning: Feature Selection/Construction"],"videos":[],"references":[{"position":1,"raw":"[Arbelaitz et al., 2013] Olatz Arbelaitz, Ibai Gurrutxaga, Javier Muguerza, JesúS M. PéRez, and IñIgo Perona. An extensive comparative study of cluster validity indices. Pat- tern Recogn., 46(1):243-256, January 2013.","title":"An extensive comparative study of cluster validity indices","authors":"Olatz Arbelaitz, Ibai Gurrutxaga, Javier Muguerza, M Jesús, Iñigo Pérez, Perona","year":"2013","venue":"Pattern Recogn","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2007] Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. Greedy layer-wise training of deep networks. In NIPS. MIT Press, 2007.","title":"Greedy layer-wise training of deep networks","authors":"Yoshua Bengio, Pascal Lamblin, Dan Popovici, Hugo Larochelle","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798-1828, August 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bengio, 2009] Yoshua Bengio. Learning deep architectures for ai. Found. Trends Mach. Learn., 2(1):1-127, January 2009.","title":"Learning deep architectures for ai","authors":"Yoshua Bengio","year":"2009","venue":"Found. Trends Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Blockeel and De Raedt, 1998] Hendrik Blockeel and Luc De Raedt. Top-down induction of first-order logical de- cision trees. Artificial Intelligence, 101(12):285 -297, 1998.","title":"Top-down induction of first-order logical decision trees","authors":"De Blockeel, Hendrik Raedt, Luc Blockeel, Raedt De","year":"1998","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bordes et al., 2011] Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. Learning structured embed- dings of knowledge bases. In Proceedings of the Twenty- Fifth AAAI Conference on Artificial Intelligence, AAAI'11, pages 301-306. AAAI Press, 2011.","title":"Learning structured embeddings of knowledge bases","authors":"Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio","year":"2011","venue":"Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI'11","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Al- berto Garcia-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS'13, pages 2787-2795, USA, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Durán, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS'13","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Curran Associates Inc. Coates et al., 2011] A. Coates, H. Lee, and A.Y. Ng. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, vol- ume 15 of JMLR Workshop and Conference Proceedings, pages 215-223. JMLR W&CP, 2011.","title":"An analysis of single-layer networks in unsupervised feature learning","authors":"A Coates, H Lee, A Y Ng","year":"2011","venue":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Craven and Slattery, 2001] Mark Craven and Seán Slattery. Relational learning with statistical predicate invention: Bet- ter models for hypertext. Mach. Learn., 43(1-2):97-119, April 2001.","title":"Relational learning with statistical predicate invention: Better models for hypertext","authors":"Slattery Craven, Mark Craven, Seán Slattery","year":"2001","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dumančić and Blockeel, 2017] Sebastijan Dumančić and Hendrik Blockeel. An expressive dissimilarity measure for relational clustering using neighbourhood trees. Ma- chine learning, 2017. To appear.","title":"Sebastijan Dumančić and Hendrik Blockeel. An expressive dissimilarity measure for relational clustering using neighbourhood trees","authors":"Blockeel Dumančić","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fortunato, 2010] Santo Fortunato. Community detection in graphs. Physics Reports, 486(3-5):75 -174, 2010.","title":"Santo Fortunato. Community detection in graphs","authors":"Fortunato","year":"2010","venue":"Physics Reports","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hinton and Salakhutdinov, 2006] G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, 2006.","title":"Reducing the dimensionality of data with neural networks","authors":"G E Hinton, R R Salakhutdinov","year":"2006","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Karypis and Kumar, 1998] George Karypis and Vipin Ku- mar. A fast and high quality multilevel scheme for partition- ing irregular graphs. SIAM J. Sci. Comput., 20(1):359-392, December 1998.","title":"George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs","authors":"Kumar Karypis","year":"1998","venue":"SIAM J. Sci. Comput","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kok and Domingos, 2007] Stanley Kok and Pedro Domin- gos. Statistical predicate invention. In Proceedings of the 24th International Conference on Machine Learning, ICML '07, pages 433-440, New York, NY, USA, 2007. ACM.","title":"Statistical predicate invention","authors":"Domingos ; Kok, Pedro Kok, Domingos","year":"2007","venue":"Proceedings of the 24th International Conference on Machine Learning, ICML '07","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kok and Domingos, 2008] Stanley Kok and Pedro Domin- gos. Extracting semantic networks from text via relational clustering. In Proceedings of the 2008 European Confer- ence on Machine Learning and Knowledge Discovery in Databases -Part I, ECML PKDD '08, pages 624-639, Berlin, Heidelberg, 2008. Springer-Verlag.","title":"Extracting semantic networks from text via relational clustering","authors":"Domingos Kok, Stanley Kok, Pedro Domingos","year":"2008","venue":"Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases -Part I, ECML PKDD '08","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kramer, 1995] Stefan Kramer. Predicate Invention: A Com- prehensive View. Technical report, 1995.","title":"Predicate Invention: A Comprehensive View","authors":"Stefan Kramer","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nickel et al., 2016] Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. A review of rela- tional machine learning for knowledge graphs. Proceedings of the IEEE, 104(1):11-33, 2016.","title":"A review of relational machine learning for knowledge graphs","authors":"Maximilian Nickel, Kevin Murphy, Evgeniy Volker Tresp, Gabrilovich","year":"2016","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Niepert et al., 2016] Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In Proceedings of the 33nd Interna- tional Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, pages 2014-2023, 2016.","title":"Learning convolutional neural networks for graphs","authors":"Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov","year":"2016","venue":"Proceedings of the 33nd International Conference on Machine Learning, ICML 2016","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Niepert, 2016] Mathias Niepert. Discriminative gaifman models. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Pro- cessing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 3405-3413, 2016.","title":"Discriminative gaifman models","authors":"Mathias Niepert","year":"2016","venue":"Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Popescul and Ungar, 2004] Alexandrin Popescul and Lyle H. Ungar. Cluster-based concept invention for statistical rela- tional learning. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '04, pages 665-670, New York, NY, USA, 2004. ACM.","title":"Alexandrin Popescul and Lyle H. Ungar. Cluster-based concept invention for statistical relational learning","authors":"Ungar Popescul","year":"2004","venue":"Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '04","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ranzato et al., 2007] Marc'Aurelio Ranzato, Y-Lan Boureau, and Yann LeCun. Sparse feature learning for deep belief networks. In NIPS, pages 1185-1192.","title":"Sparse feature learning for deep belief networks","authors":"Ranzato","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"Curran Associates, Inc., 2007.","title":null,"authors":"Curran Associates, Inc","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Richardson and Domingos, 2006] Matthew Richardson and Pedro Domingos. Markov logic networks. Mach. Learn., 62(1-2):107-136, February 2006.","title":"Markov logic networks","authors":"Matthew Richardson, Pedro Domingos","year":"2006","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rousseeuw, 1987] Peter Rousseeuw. Silhouettes: A graphi- cal aid to the interpretation and validation of cluster analy- sis. J. Comput. Appl. Math., 20(1):53-65, November 1987.","title":"Silhouettes: A graphical aid to the interpretation and validation of cluster analysis","authors":"Peter Rousseeuw","year":"1987","venue":"J. Comput. Appl. Math","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vendramin et al., 2010] Lucas Vendramin, Ricardo J. G. B. Campello, and Eduardo R. Hruschka. Relative clustering validity criteria: A comparative overview. Statistical Anal- ysis and Data Mining, 3(4):209-235, 2010.","title":"Relative clustering validity criteria: A comparative overview","authors":"Lucas Vendramin, Ricardo J G B Campello, Eduardo R Hruschka","year":"2010","venue":"Statistical Analysis and Data Mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}