{"type":"ScholarlyArticle","title":"Distributed Collaborative Feature Selection Based on Intermediate Representation","authors":[{"name":"Xiucai Ye","slug":"xiucai-ye","given":"Xiucai","family":"Ye","affiliation":"Department of Computer Science , University of Tsukuba"},{"name":"Hongmin Li","slug":"hongmin-li","given":"Hongmin","family":"Li","affiliation":"Department of Computer Science , University of Tsukuba"},{"name":"Akira Imakura","slug":"akira-imakura","given":"Akira","family":"Imakura","affiliation":"Department of Computer Science , University of Tsukuba"},{"name":"Tetsuya Sakurai","slug":"tetsuya-sakurai","given":"Tetsuya","family":"Sakurai","affiliation":"Department of Computer Science , University of Tsukuba"}],"abstract":"Feature selection is an efficient dimensionality reduction technique for artificial intelligence and machine learning. Many feature selection methods learn the data structure to select the most discriminative features for distinguishing different classes. However, the data is sometimes distributed in multiple parties and sharing the original data is difficult due to the privacy requirement. As a result, the data in one party may be lack of useful information to learn the most discriminative features. In this paper, we propose a novel distributed method which allows collaborative feature selection for multiple parties without revealing their original data. In the proposed method, each party finds the intermediate representations from the original data, and shares the intermediate representations for collaborative feature selection. Based on the shared intermediate representations, the original data from multiple parties are transformed to the same low dimensional space. The feature ranking of the original data is learned by imposing row sparsity on the transformation matrix simultaneously. Experimental results on real-world datasets demonstrate the effectiveness of the proposed method.","doi":"10.24963/ijcai.2019/575","url":"https://ijcai.org/proceedings/2019/575","pdf":"https://ijcai.org/proceedings/2019/0575.pdf","markdown":"https://ijcai.org/proceedings/2019/575.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/575","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":575,"pages":{"first":4142,"last":4149},"keywords":["Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning: Unsupervised Learning","Machine Learning: Dimensionality Reduction and Manifold Learning"],"videos":[],"references":[{"position":1,"raw":"[Banerjee and Chakravarty, 2011] Madhushri Banerjee and Sumit Chakravarty. Privacy preserving feature selection for distributed data using virtual dimension. In Proceed- ings of ACM Conference on Information and Knowledge Management, pages 2281-2284, 2011.","title":"Privacy preserving feature selection for distributed data using virtual dimension","authors":"Chakravarty Banerjee, Madhushri Banerjee, Sumit Chakravarty","year":"2011","venue":"Proceedings of ACM Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2010] Deng Cai, Chiyuan Zhang, and Xiaofei He. Unsupervised feature selection for multi-cluster data. In Proceedings of ACM SIGKDD international conference on Knowledge discovery and data mining, pages 333-342, 2010.","title":"Unsupervised feature selection for multi-cluster data","authors":"Deng Cai, Chiyuan Zhang, Xiaofei He","year":"2010","venue":"Proceedings of ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chaudhuri et al., 2011] Kamalika Chaudhuri, Claire Mon- teleoni, and Anand D. Sarwate. Differentially private em- pirical risk minimization. Journal of Machine Learning Research, 9:1069-1109, 2011.","title":"Differentially private empirical risk minimization","authors":"Kamalika Chaudhuri, Claire Monteleoni, Anand D Sarwate","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guyon and Elisseeff, 1997] Isabelle Guyon and André Elis- seeff. An introduction to variable and feature selec- tion. Journal of Machine Learning Research, 3:1157- 1182, 1997.","title":"Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection","authors":"Elisseeff Guyon","year":"1997","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hall et al., 2012] Rob Hall, Rinaldo Alessandro, and Wasserman Larry. Random differential privacy. Journal of Privacy and Confidentiality, 4(2):43-59, 2012.","title":"Random differential privacy","authors":"Rob Hall, Rinaldo Alessandro, Wasserman Larry","year":"2012","venue":"Journal of Privacy and Confidentiality","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He and Niyogi, 2004] Xiaofei He and Partha Niyogi. Local- ity preserving projections. Neural information processing systems, 16:153, 2004.","title":"Locality preserving projections","authors":"Xiaofei He, Partha Niyogi","year":"2004","venue":"Neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2006] Xiaofei He, Deng Cai, , and Partha Niyogi. Laplacian score for feature selection. In Advances in neu- ral information processing systems, pages 507-514, 2006.","title":"Laplacian score for feature selection","authors":"Xiaofei He, Deng Cai, Partha Niyogi","year":"2006","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hou et al., 2011] Chenping Hou, Feiping Nie, Dongyun Yi, and Yi Wu. Feature selection via joint embedding learn- ing and sparse regression. In Proceedings of International Joint Conference on Artificial Intelligence, pages 1324- 1329, 2011.","title":"Feature selection via joint embedding learning and sparse regression","authors":"Hou","year":"2011","venue":"Proceedings of International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2011/224"},{"position":9,"raw":"[Imakura and Sakurai, 2019] Akira Imakura and Tetsuya Sakurai. Data collaboration analysis for distributed datasets. In arXiv:1902.07535 [cs.LG], 2019.","title":"Data collaboration analysis for distributed datasets","authors":"Sakurai Imakura, Akira Imakura, Tetsuya Sakurai","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Imakura et al., 2019] Akira Imakura, Momo Matsuda, Xiu- cai Ye, and Tetsuya Sakurai. Complex moment-based su- pervised eigenmap for dimensionality reduction. In The Thirty-Third AAAI Conference on Artificial Intelligence, 2019.","title":"Complex moment-based supervised eigenmap for dimensionality reduction","authors":"Akira Imakura, Momo Matsuda, Xiucai Ye, Tetsuya Sakurai","year":"2019","venue":"The Thirty-Third AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ito and Murota, 2016] Shinji Ito and Kazuo Murota. An algorithm for the generalized eigenvalue problem for nonsquare matrix pencils by minimal perturbation ap- proach. SIAM Journal on Matrix Analysis and Applica- tions, 37:409-419, 2016.","title":"An algorithm for the generalized eigenvalue problem for nonsquare matrix pencils by minimal perturbation approach","authors":"Murota ; ] Shinji Ito, Kazuo Ito, Murota","year":"2016","venue":"SIAM Journal on Matrix Analysis and Applications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jafer et al., 2014] Yasser Jafer, Stan Matwinand, and Ma- rina Sokolova. Task oriented privacy preserving data pub- lishing using feature selection. In in Advances in Artificial Intelligence -27th Canadian Conference on Artificial In- telligence, pages 143-154, 2014.","title":"Task oriented privacy preserving data publishing using feature selection","authors":"Jafer","year":"2014","venue":"Advances in Artificial Intelligence -27th Canadian Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jain and Zongker, 1997] Anil Jain and Douglas Zongker. Feature selection: Evaluation, application, and small sam- ple performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(2):153-158, 1997.","title":"Feature selection: Evaluation, application, and small sample performance","authors":"Anil Jain, Douglas Zongker","year":"1997","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[LeCun et al., 1998] Yann LeCun, Corinna Cortes, and Christopher J.C. Burges. The mnist database of hand- written digits. In http://yann.lecun.com/exdb/mnist/, 1998.","title":"The mnist database of handwritten digits","authors":"Lecun","year":"1998","venue":null,"doi":null,"url":"http://yann.lecun.com/exdb/mnist/,1998","internal_url":null},{"position":15,"raw":"[Maldonado and Weber, 2009] Sebastián Maldonado and Richard Weber. A wrapper method for feature selection using support vector machines. information sciences. Information Sciences, 179(13):2208-2217, 2009.","title":"A wrapper method for feature selection using support vector machines","authors":"Weber ; Maldonado, Sebastián Maldonado, Richard Weber","year":"2009","venue":"information sciences. Information Sciences","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H. Ding. Efficient and robust feature selection via joint l2,1-norms minimization. In Advances in neural in- formation processing systems, 2010.","title":"Efficient and robust feature selection via joint l2,1-norms minimization","authors":"Feiping Nie, Heng Huang, Xiao Cai, Chris H Ding","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Roweis and Saul, 2000] Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally lin- ear embedding. Science, 290:2323-2326, 2000.","title":"Nonlinear dimensionality reduction by locally linear embedding","authors":"Sam T Roweis, Lawrence K Saul","year":"2000","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Saeys et al., 2008] Yvan Saeys, Thomas Abeel, and Yves Van de Peer. Robust feature selection using en- semble feature selection techniques. In Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 313-325, 2008.","title":"Robust feature selection using ensemble feature selection techniques","authors":"Yvan Saeys, Thomas Abeel, Yves Van De Peer","year":"2008","venue":"Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sheikh et al., 2010] Rashid Sheikh, Beerendra Kumar, and Durgesh Kumar Mishra. A distributed k-secure sum proto- col for secure multi-party computations. In arXiv preprint arXiv:1003.4071, 2010.","title":"A distributed k-secure sum protocol for secure multi-party computations","authors":"Rashid Sheikh, Beerendra Kumar, Durgesh Kumar, Mishra","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sheikhalishahi and Fabio, 2017] Mina Sheikhalishahi and Martinelli Fabio. Privacy-utility feature selection as a pri- vacy mechanism in collaborative data classification. In Proceedings of IEEE International Conference on En- abling Technologies: Infrastructure for Collaborative En- terprises, pages 244-249, 2017.","title":"Mina Sheikhalishahi and Martinelli Fabio. Privacy-utility feature selection as a privacy mechanism in collaborative data classification","authors":"Fabio Sheikhalishahi","year":"2017","venue":"Proceedings of IEEE International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Strehl and Ghosh, 2002] Alexander Strehl and Joydeep Ghosh. Cluster ensembles-a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, 3:583-617, 2002.","title":"Cluster ensembles-a knowledge reuse framework for combining multiple partitions","authors":"Ghosh Strehl, Alexander Strehl, Joydeep Ghosh","year":"2002","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang and Li, 2014] Jun Yang and Yun Li. Differentially pri- vate feature selection. In Proceedings of International Joint Conference on Neural Networks, pages 4182-4189, 2014.","title":"Differentially private feature selection","authors":"Yang, Li ;, Jun Yang, Yun Li","year":"2014","venue":"Proceedings of International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ye and Sakurai, 2018] Xiucai Ye and Tetsuya Sakurai. Un- supervised feature selection for microarray gene expres- sion data based on discriminative structure learning. Jour- nal of Universal Computer Science, 24(6):725-741, 2018.","title":"Xiucai Ye and Tetsuya Sakurai. Unsupervised feature selection for microarray gene expression data based on discriminative structure learning","authors":"Sakurai Ye","year":"2018","venue":"Journal of Universal Computer Science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ye et al., 2016a] Xiucai Ye, Kaiyang Ji, and Tetsuya Saku- rai. Global discriminant analysis for unsupervised feature selection with local structure preservation. In Proceedings of the Twenty-Ninth International Florida Artificial Intelli- gence Research Society Conference, pages 454-459, 2016.","title":"Global discriminant analysis for unsupervised feature selection with local structure preservation","authors":"Ye","year":"2016","venue":"Proceedings of the Twenty-Ninth International Florida Artificial Intelligence Research Society Conference","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ye et al., 2016b] Xiucai Ye, Kaiyang Ji, and Tetsuya Saku- rai. Unsupervised feature selection with correlation and individuality analysis. International Journal of Machine Learning and Computing, 6(1):36-41, 2016.","title":"Unsupervised feature selection with correlation and individuality analysis","authors":"Ye","year":"2016","venue":"International Journal of Machine Learning and Computing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}