{"type":"ScholarlyArticle","title":"Hierarchical Linear Disentanglement of Data-Driven Conceptual Spaces","authors":[{"name":"Rana Alshaikh","slug":"rana-alshaikh","given":"Rana","family":"Alshaikh","affiliation":"Cardiff University , UK"},{"name":"Zied Bouraoui","slug":"zied-bouraoui","given":"Zied","family":"Bouraoui","affiliation":"CRIL , Univ Artois & CNRS , France"},{"name":"Steven Schockaert","slug":"steven-schockaert","given":"Steven","family":"Schockaert","affiliation":"Cardiff University , UK"}],"abstract":"Conceptual spaces are geometric meaning representations in which similar entities are represented by similar vectors. They are widely used in cognitive science, but there has been relatively little work on learning such representations from data. In particular, while standard representation learning methods can be used to induce vector space embeddings from text corpora, these differ from conceptual spaces in two crucial ways.  First, the dimensions of a conceptual space correspond to salient semantic features, known as quality dimensions, whereas the dimensions of learned vector space embeddings typically lack any clear interpretation. This has been partially addressed in previous work, which has shown that it is possible to identify directions in learned vector spaces which capture semantic features. Second, conceptual spaces are normally organised into a set of domains, each of which is associated with a separate vector space. In contrast, learned embeddings represent all entities in a single vector space. Our hypothesis in this paper is that such single-space representations are sub-optimal for learning quality dimensions, due to the fact that semantic features are often only relevant to a subset of the entities. We show that this issue can be mitigated by identifying features in a hierarchical fashion. Intuitively, the top-level features split the vector space into different domains, making it possible to subsequently identify domain-specific quality dimensions.","doi":"10.24963/ijcai.2020/494","url":"https://ijcai.org/proceedings/2020/494","pdf":"https://ijcai.org/proceedings/2020/0494.pdf","markdown":"https://ijcai.org/proceedings/2020/494.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/494","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":494,"pages":{"first":3573,"last":3579},"keywords":["Machine Learning: Interpretability","Humans and AI: Cognitive Modeling","Natural Language Processing: Natural Language Processing"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24340/file_24340.m3u8","length_sec":301,"provider":"hls"}],"references":[{"position":1,"raw":"childish, adventures, fare], [musicals, dance, sings], [magical, magic, enchanting], [heartwarming, sentimental, warm] [gore, gory] [bloody, zombie, massacre], [killings, serial, maniac], [hardcore, carnage, gratuitous], [supernatural] [sci, science] [creatures, humans, eaten], [franchise, sequel, sequels], [cgi, technology, computer, graphics], [mythology, kingdom, ancient], [doctor, blast, mindless, brain], [scifi, futuristic, outer, aliens], [animation, animators, pixar],","title":"magical, magic, enchanting], [heartwarming, sentimental, warm] [gore, gory] [bloody, zombie, massacre","authors":null,"year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"musicians, concert, concerts, albums], [biography], documentary, [individuals, perspective, focuses, individual], [educational], interviews] inspiring, awe, captured, appreciation], [artists, artist], [facts, research, account, thousands], [recording, recordings]","title":"musicians, concert, concerts, albums","authors":null,"year":null,"venue":"documentary, [individuals, perspective, focuses, individual","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ager et al., 2018] Thomas Ager, Ondrej Kuzelka, and Steven Schockaert. Modelling salient features as direc- tions in fine-tuned semantic spaces. In Proc. CoNLL, pages 530-540, 2018.","title":"Modelling salient features as directions in fine-tuned semantic spaces","authors":"Thomas Ager, Ondrej Kuzelka, Steven Schockaert","year":"2018","venue":"Proc. CoNLL","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Agres et al., 2015] Kat Agres, Stephen McGregor, Matthew Purver, and Geraint A. Wiggins. Conceptualizing creativ- ity: From distributional semantics to conceptual spaces. In Proc. ICCC, pages 118-125, 2015.","title":"Conceptualizing creativity: From distributional semantics to conceptual spaces","authors":"Agres","year":"2015","venue":"Proc. ICCC","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Alshaikh et al., 2019] Rana Alshaikh, Zied Bouraoui, and Steven Schockaert. Learning conceptual spaces with dis- entangled facets. In Proc. CoNLL, 2019.","title":"Learning conceptual spaces with disentangled facets","authors":"Rana Alshaikh, Zied Bouraoui, Steven Schockaert","year":"2019","venue":"Proc. CoNLL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bingham and Mannila, 2001] Ella Bingham and Heikki Mannila. Random projection in dimensionality reduction: applications to image and text data. In Proc. SIGKDD, pages 245-250, 2001.","title":"Random projection in dimensionality reduction: applications to image and text data","authors":"Mannila Bingham, Ella Bingham, Heikki Mannila","year":"2001","venue":"Proc. SIGKDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Camacho-Collados et al., 2016] José Camacho-Collados, Mohammad Taher Pilehvar, and Roberto Navigli. Nasari: Integrating explicit knowledge and corpus statistics for a multilingual representation of concepts and entities. Artificial Intelligence, 240:36-64, 2016.","title":"José Camacho-Collados, Mohammad Taher Pilehvar, and Roberto Navigli. Nasari: Integrating explicit knowledge and corpus statistics for a multilingual representation of concepts and entities","authors":"Camacho-Collados","year":"2016","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chella, 2015] Antonio Chella. A cognitive architecture for music perception exploiting conceptual spaces. In Frank Zenker and Peter Gärdenfors, editors, Applications of Con- ceptual Spaces, pages 187-203. Springer International Publishing, 2015.","title":"A cognitive architecture for music perception exploiting conceptual spaces","authors":"Antonio Chella","year":"2015","venue":"Applications of Conceptual Spaces","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. In Proc. NIPS, pages 2172-2180, 2016.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":"2016","venue":"Proc. NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Deerwester et al., 1990] S. Deerwester, S.Dumais, G. Fur- nas, T. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American Society for In- formation Science, 41:391-407, 1990.","title":"Indexing by latent semantic analysis","authors":"S Deerwester, S Dumais, G Furnas, T Landauer, R Harshman","year":"1990","venue":"Journal of the American Society for Information Science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Derrac and Schockaert, 2015] Joaquín Derrac and Steven Schockaert. Inducing semantic relations from conceptual spaces: a data-driven approach to plausible reasoning. Ar- tificial Intelligence, 228:66-94, 2015.","title":"Joaquín Derrac and Steven Schockaert. Inducing semantic relations from conceptual spaces: a data-driven approach to plausible reasoning","authors":"Schockaert Derrac","year":"2015","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Esmaeili et al., 2019] B. Esmaeili, H. Wu, S. Jain, A. Bozkurt, N. Siddharth, B. Paige, D. H. Brooks, J. G. Dy, and Jan-Willem van de Meent. Structured disentangled representations. In Proc. AISTATS, pages 2525-2534, 2019.","title":"Structured disentangled representations","authors":"B Esmaeili, H Wu, S Jain, A Bozkurt, N Siddharth, B Paige, D H Brooks, J G Dy, Jan-Willem Van De Meent","year":"2019","venue":"Proc. AISTATS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Forth et al., 2010] J. Forth, G. A Wiggins, and A. McLean. Unifying conceptual spaces: Concept formation in musical creative systems. Minds and Machines, 20:503-532, 2010.","title":"Unifying conceptual spaces: Concept formation in musical creative systems","authors":"J Forth, G A Wiggins, A Mclean","year":"2010","venue":"Minds and Machines","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Frey and Dueck, 2007] Brendan J Frey and Delbert Dueck. Clustering by passing messages between data points. Sci- ence, 315:972-976, 2007.","title":"Clustering by passing messages between data points","authors":"J Brendan, Delbert Frey, Dueck","year":"2007","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gabrilovich and Markovitch, 2007] E. Gabrilovich and S. Markovitch. Computing semantic relatedness using wikipedia-based explicit semantic analysis. In Proc. IJCAI, pages 1606-1611, 2007.","title":"Computing semantic relatedness using wikipedia-based explicit semantic analysis","authors":"E Gabrilovich, S Markovitch","year":"2007","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/259"},{"position":16,"raw":"[Gärdenfors, 1997] Peter Gärdenfors. Symbolic, conceptual and subconceptual representations. In Human and Ma- chine Perception, pages 255-270. Springer, 1997.","title":"Peter Gärdenfors. Symbolic, conceptual and subconceptual representations","authors":"Gärdenfors","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gärdenfors, 2000] Peter Gärdenfors. Conceptual Spaces: The Geometry of Thought. MIT press, 2000.","title":"Conceptual Spaces: The Geometry of Thought","authors":"Peter Gärdenfors","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Goodfellow et al., 2014] I.","title":null,"authors":"Goodfellow","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Proc. NIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"J Goodfellow, M Pouget-Abadie, B Mirza, D Xu, S Warde-Farley, A Ozair, Y Courville, Bengio","year":"2014","venue":"Proc. NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Gupta et al., 2015] Abhijeet Gupta, Gemma Boleda, Marco Baroni, and Sebastian Padó. Distributional vectors encode referential attributes. In Proc. EMNLP, pages 12-21, 2015.","title":"Distributional vectors encode referential attributes","authors":"Abhijeet Gupta, Gemma Boleda, Marco Baroni, Sebastian Padó","year":"2015","venue":"Proc. EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Jain et al., 2018] Sarthak Jain, Edward Banner, Jan-Willem van de Meent, Iain J Marshall, and Byron C Wallace. Learning disentangled representations of texts with appli- cation to biomedical abstracts. In Proc. EMNLP, pages 4683-4693, 2018.","title":"Learning disentangled representations of texts with application to biomedical abstracts","authors":"Sarthak Jain, Edward Banner, Jan-Willem Van De Meent, Iain J Marshall, Byron C Wallace","year":"2018","venue":"Proc. EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Jameel et al., 2017] S. Jameel, Z. Bouraoui, and S. Schock- aert. Member: Max-margin based embeddings for entity retrieval. In Proc. ACM SIGIR, pages 783-792, 2017.","title":"Member: Max-margin based embeddings for entity retrieval","authors":"S Jameel, Z Bouraoui, S Schockaert","year":"2017","venue":"Proc. ACM SIGIR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kim and de Marneffe, 2013] Joo-Kyung Kim and Marie- Catherine de Marneffe. Deriving adjectival scales from continuous space word representations. In Proc. EMNLP, pages 1625-1630, 2013.","title":"Deriving adjectival scales from continuous space word representations","authors":"De Kim, ; Joo-Kyung Marneffe, Marie-Catherine Kim, De Marneffe","year":"2013","venue":"Proc. EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Le and Mikolov, 2014] Quoc Le and Tomas Mikolov. Dis- tributed representations of sentences and documents. In Proc. ICML, pages 1188-1196, 2014.","title":"Quoc Le and Tomas Mikolov. Distributed representations of sentences and documents","authors":"Mikolov Le","year":"2014","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Locatello et al., 2018] F. Locatello, S. Bauer, M. Lucic, S. Gelly, B. Schölkopf, and O. Bachem. Challenging com- mon assumptions in the unsupervised learning of disentan- gled representations. CoRR, abs/1811.12359, 2018.","title":"Challenging common assumptions in the unsupervised learning of disentangled representations","authors":"F Locatello, S Bauer, M Lucic, S Gelly, B Schölkopf, O Bachem","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Miao et al., 2016] Yishu Miao, Lei Yu, and Phil Blunsom. Neural variational inference for text processing. In Proc. ICML, pages 1727-1736, 2016.","title":"Neural variational inference for text processing","authors":"Yishu Miao, Lei Yu, Phil Blunsom","year":"2016","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In Proc. NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Proc. NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Norouzi et al., 2014] M. Norouzi, T. Mikolov, S. Bengio, Y.Singer, J. Shlens, A. Frome, G.S Corrado, and J. Dean. Zero-shot learning by convex combination of semantic embeddings. In Proc. ICLR, 2014.","title":"Zero-shot learning by convex combination of semantic embeddings","authors":"M Norouzi, T Mikolov, S Bengio, Y Singer, J Shlens, A Frome, G S Corrado, J Dean","year":"2014","venue":"Proc. ICLR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nosofsky, 1986] Robert M Nosofsky. Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology, 115:39, 1986.","title":"Attention, similarity, and the identification-categorization relationship","authors":"M Robert, Nosofsky","year":"1986","venue":"Journal of Experimental Psychology","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ruder et al., 2016] Sebastian Ruder, Parsa Ghaffari, and John G. Breslin. A hierarchical model of reviews for aspect-based sentiment analysis. In Proc. EMNLP, pages 999-1005, 2016.","title":"A hierarchical model of reviews for aspect-based sentiment analysis","authors":"Sebastian Ruder, Parsa Ghaffari, John G Breslin","year":"2016","venue":"Proc. EMNLP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shepard, 1957] Roger N Shepard. Stimulus and response generalization: A stochastic model relating generaliza- tion to distance in psychological space. Psychometrika, 22:325-345, 1957.","title":"Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space","authors":"N Roger, Shepard","year":"1957","venue":"Psychometrika","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}