{"type":"ScholarlyArticle","title":"Embedding-based Representation of Categorical Data by Hierarchical Value Coupling Learning","authors":[{"name":"Songlei Jian","slug":"songlei-jian","given":"Songlei","family":"Jian","affiliation":"* Advanced Analytics Institute , University of Technology Sydney , Australia; † Science and Technology on Parallel and Distributed Processing Laboratory ,; ‡ State Key Laboratory of High Performance Computing , College of Computer , National University of Defense Technology , China"},{"name":"Longbing Cao","slug":"longbing-cao","given":"Longbing","family":"Cao","affiliation":"* Advanced Analytics Institute , University of Technology Sydney , Australia"},{"name":"Guansong Pang","slug":"guansong-pang","given":"Guansong","family":"Pang","affiliation":"* Advanced Analytics Institute , University of Technology Sydney , Australia"},{"name":"Kai Lu","slug":"kai-lu","given":"Kai","family":"Lu","affiliation":"† Science and Technology on Parallel and Distributed Processing Laboratory ,; ‡ State Key Laboratory of High Performance Computing , College of Computer , National University of Defense Technology , China"},{"name":"Hang Gao","slug":"hang-gao","given":"Hang","family":"Gao","affiliation":"† Science and Technology on Parallel and Distributed Processing Laboratory ,"}],"abstract":"Learning the representation of categorical data with hierarchical value coupling relationships is very challenging but critical for the effective analysis and learning of such data. This paper proposes a novel coupled unsupervised categorical data representation (CURE) framework and its instantiation, i.e., a coupled data embedding (CDE) method, for representing categorical data by hierarchical value-to-value cluster coupling learning. Unlike existing embedding- and similarity-based representation methods which can capture only a part or none of these complex couplings, CDE explicitly incorporates the hierarchical couplings into its embedding representation. CDE first learns two complementary feature value couplings which are then used to cluster values with different granularities. It further models the couplings in value clusters within the same granularity and with different granularities to embed feature values into a new numerical space with independent dimensions. Substantial experiments show that CDE significantly outperforms three popular unsupervised embedding methods and three state-of-the-art similarity-based representation methods.","doi":"10.24963/ijcai.2017/269","url":"https://ijcai.org/proceedings/2017/269","pdf":"https://ijcai.org/proceedings/2017/0269.pdf","markdown":"https://ijcai.org/proceedings/2017/269.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/269","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":269,"pages":{"first":1937,"last":1943},"keywords":["Machine Learning: Data Mining","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Ahmad and Dey, 2007] Amir Ahmad and Lipika Dey. A k- mean clustering algorithm for mixed numeric and categor- ical data. Data & Knowledge Engineering, 63(2):503-527, 2007.","title":"A kmean clustering algorithm for mixed numeric and categorical data","authors":"Ahmad, Dey ;, Amir Ahmad, Lipika Dey","year":"2007","venue":"Data & Knowledge Engineering","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aizawa, 2003] Akiko Aizawa. An information-theoretic perspective of tf-idf measures. Information Processing & Management, 39(1):45-65, 2003.","title":"An information-theoretic perspective of tf-idf measures","authors":"Akiko Aizawa","year":"2003","venue":"Information Processing & Management","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798-1828, 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blei et al., 2003] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3(Jan):993-1022, 2003.","title":"Latent dirichlet allocation","authors":"Andrew Y David M Blei, Michael I Ng, Jordan","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2012] Longbing Cao, Yuming Ou, and S Yu Philip. Coupled behavior analysis with applications. IEEE Transactions on Knowledge and Data Engineering, 24(8):1378-1392, 2012.","title":"Coupled behavior analysis with applications","authors":"Cao","year":"2012","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cao, 2014] Longbing Cao. Non-iidness learning in behav- ioral and social data. The Computer Journal, 57(9):1358- 1370, 2014.","title":"Longbing Cao. Non-iidness learning in behavioral and social data","authors":"Cao","year":"2014","venue":"The Computer Journal","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cao, 2015] Longbing Cao. Coupling learning of complex interactions. Information Processing & Management, 51(2):167-186, 2015.","title":"Longbing Cao. Coupling learning of complex interactions","authors":"Cao","year":"2015","venue":"Information Processing & Management","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cohen et al., 2013] Jacob Cohen, Patricia Cohen, Stephen G West, and Leona S Aiken. Applied mul- tiple regression/correlation analysis for the behavioral sciences. Routledge, 2013.","title":"Applied multiple regression/correlation analysis for the behavioral sciences","authors":"Jacob Cohen, Patricia Cohen, Stephen G West, Leona S Aiken","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cox and Cox, 2000] Trevor F Cox and Michael AA Cox. Multidimensional Scaling. CRC press, 2000.","title":"Multidimensional Scaling","authors":"F Trevor, Michael Aa Cox, Cox","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Deerwester et al., 1990] Scott Deerwester, Susan T Du- mais, George W Furnas, Thomas K Landauer, and Richard Harshman. Indexing by latent semantic analysis. Jour- nal of the American Society for Information Science, 41(6):391, 1990.","title":"Indexing by latent semantic analysis","authors":"Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, Richard Harshman","year":"1990","venue":"Journal of the American Society for Information Science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Estévez et al., 2009] Pablo A Estévez, Michel Tesmer, Claudio A Perez, and Jacek M Zurada. Normalized mu- tual information feature selection. IEEE Transactions on Neural Networks, 20(2):189-201, 2009.","title":"Normalized mutual information feature selection","authors":"Pablo A Estévez, Michel Tesmer, Claudio A Perez, Jacek M Zurada","year":"2009","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Foss and Zaïane, 2002] Andrew Foss and Osmar R Zaïane. A parameterless method for efficiently discovering clus- ters of arbitrary shape in large datasets. In Proceedings of ICDM, pages 179-186. IEEE, 2002.","title":"A parameterless method for efficiently discovering clusters of arbitrary shape in large datasets","authors":"Andrew Foss, Osmar R Zaïane","year":"2002","venue":"Proceedings of ICDM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hinton and Roweis, 2002] Geoffrey E Hinton and Sam T Roweis. Stochastic neighbor embedding. In Proceedings of NIPS, pages 833-840, 2002.","title":"Stochastic neighbor embedding","authors":"Geoffrey E Hinton, Sam T Roweis","year":"2002","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hofmann, 1999] Thomas Hofmann. Probabilistic latent se- mantic indexing. In Proceedings of SIGIR, pages 50-57. ACM, 1999.","title":"Probabilistic latent semantic indexing","authors":"Thomas Hofmann","year":"1999","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ienco et al., 2012] Dino Ienco, Ruggero G Pensa, and Rosa Meo. From context to distance: Learning dissimilarity for categorical data clustering. ACM Transactions on Knowl- edge Discovery from Data, 6(1):1, 2012.","title":"From context to distance: Learning dissimilarity for categorical data clustering","authors":"Dino Ienco, G Ruggero, Rosa Pensa, Meo","year":"2012","venue":"ACM Transactions on Knowledge Discovery from Data","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jia et al., 2016] Hong Jia, Yiu-ming Cheung, and Jiming Liu. A new distance metric for unsupervised learning of categorical data. IEEE Transactions on Neural Networks and Learning Systems, 27(5):1065-1079, 2016.","title":"A new distance metric for unsupervised learning of categorical data","authors":"Jia","year":"2016","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jolliffe, 2002] Ian Jolliffe. Principal component analysis. Wiley Online Library, 2002.","title":"Principal component analysis","authors":"Ian Jolliffe","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. In Proceedings of NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pang et al., 2016a] Guansong Pang, Longbing Cao, and Ling Chen. Outlier detection in complex categorical data by modelling the feature value couplings. In Proceedings of IJCAI, pages 1902-1908. AAAI, 2016.","title":"Outlier detection in complex categorical data by modelling the feature value couplings","authors":"Pang","year":"2016","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pang et al., 2016b] Guansong Pang, Longbing Cao, Ling Chen, and Huan Liu. Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings. In Proceedings of ICDM, pages 410-419. IEEE, 2016.","title":"Unsupervised feature selection for outlier detection by modelling hierarchical value-feature couplings","authors":"Pang","year":"2016","venue":"Proceedings of ICDM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pang et al., 2016c] Guansong Pang, Kai Ming Ting, David Albrecht, and Huidong Jin. Zero++: Harnessing the power of zero appearances to detect anomalies in large-scale data sets. Journal of Artificial Intelligence Research, 57:593- 620, 2016.","title":"Zero++: Harnessing the power of zero appearances to detect anomalies in large-scale data sets","authors":"Pang","year":"2016","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Powers, 2011] David Martin Powers. Evaluation: from pre- cision, recall and f-measure to roc, informedness, marked- ness and correlation. 2011.","title":"Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation","authors":"David Martin, Powers","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2015] Can Wang, Xiangjun Dong, Fei Zhou, Longbing Cao, and Chi-Hung Chi. Coupled attribute sim- ilarity learning on categorical data. IEEE Transactions on Neural Networks and Learning Systems, 26(4):781-797, 2015.","title":"Coupled attribute similarity learning on categorical data","authors":"Wang","year":"2015","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wilson and Chew, 2010] Andrew T Wilson and Peter A Chew. Term weighting schemes for latent dirichlet alloca- tion. In The 2010 Annual Conference of the North Amer- ican Chapter of the Association for Computational Lin- guistics, pages 465-473. Association for Computational Linguistics, 2010.","title":"Term weighting schemes for latent dirichlet allocation","authors":"T Andrew, Peter A Wilson, Chew","year":"2010","venue":"The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu and Liu, 2003] Lei Yu and Huan Liu. Feature selection for high-dimensional data: A fast correlation-based filter solution. In Proceedings of ICML, volume 3, pages 856- 863, 2003.","title":"Feature selection for high-dimensional data: A fast correlation-based filter solution","authors":"Yu, Liu ; ] Lei Yu, Huan Liu","year":"2003","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2015] Kai Zhang, Qiaojun Wang, Zhengzhang Chen, Ivan Marsic, Vipin Kumar, Guofei Jiang, and Jie Zhang. From categorical to numerical: Multiple transitive distance learning and embedding. In Proceedings of SDM. SIAM, 2015.","title":"From categorical to numerical: Multiple transitive distance learning and embedding","authors":"Kai Zhang, Qiaojun Wang, Zhengzhang Chen, Ivan Marsic, Vipin Kumar, Guofei Jiang, Jie Zhang","year":"2015","venue":"Proceedings of SDM. SIAM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}