{"type":"ScholarlyArticle","title":"Multi-label Feature Selection via Global Relevance and Redundancy Optimization","authors":[{"name":"Jia Zhang","slug":"jia-zhang","given":"Jia","family":"Zhang","affiliation":"Department of Artificial Intelligence , Xiamen University , China"},{"name":"Yidong Lin","slug":"yidong-lin","given":"Yidong","family":"Lin","affiliation":"School of Mathematical Sciences , Xiamen University , China"},{"name":"Min Jiang","slug":"min-jiang","given":"Min","family":"Jiang","affiliation":"Department of Artificial Intelligence , Xiamen University , China"},{"name":"Shaozi Li","slug":"shaozi-li","given":"Shaozi","family":"Li","affiliation":"Department of Artificial Intelligence , Xiamen University , China"},{"name":"Yong Tang","slug":"yong-tang","given":"Yong","family":"Tang","affiliation":"School of Computer Science , South China Normal University , China"},{"name":"Kay Chen Tan","slug":"kay-chen-tan","given":"Kay Chen","family":"Tan","affiliation":"Department of Computer Science , City University of Hong Kong , Hong Kong"}],"abstract":"Information theoretical based methods have attracted a great attention in recent years, and gained promising results to deal with multi-label data with high dimensionality. However, most of the existing methods are either directly transformed from heuristic single-label feature selection methods or inefficient in exploiting labeling information. Thus, they may not be able to get an optimal feature selection result shared by multiple labels. In this paper, we propose a general global optimization framework, in which feature relevance, label relevance (i.e., label correlation), and feature redundancy are taken into account, thus facilitating multi-label feature selection. Moreover, the proposed method has an excellent mechanism for utilizing inherent properties of multi-label learning. Specially, we provide a formulation to extend the proposed method with label-specific features. Empirical studies on twenty multi-label data sets reveal the effectiveness and efficiency of the proposed method. Our implementation of the proposed method is available online at: https://jiazhang-ml.pub/GRRO-master.zip.","doi":"10.24963/ijcai.2020/348","url":"https://ijcai.org/proceedings/2020/348","pdf":"https://ijcai.org/proceedings/2020/0348.pdf","markdown":"https://ijcai.org/proceedings/2020/348.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/348","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":348,"pages":{"first":2512,"last":2518},"keywords":["Data Mining: Feature Extraction, Selection and Dimensionality Reduction","Machine Learning: Feature Selection; Learning Sparse Models"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24514/file_24514.m3u8","length_sec":282,"provider":"hls"}],"references":[{"position":1,"raw":"[Bermejo et al., 2018] Pablo Bermejo, José A. Gámez, and José Miguel Puerta. Adapting the CMIM algorithm for multilabel feature selection. A comparison with existing methods. Expert Systems, 35(1), 2018.","title":"Adapting the CMIM algorithm for multilabel feature selection. A comparison with existing methods","authors":"Bermejo","year":"2018","venue":"Expert Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boutell et al., 2004] Matthew R. Boutell, Jiebo Luo, X- ipeng Shen, and Christopher M. Brown. Learning multi-label scene classification. Pattern Recognition, 37(9):1757-1771, 2004.","title":"Learning multi-label scene classification","authors":"Matthew R Boutell, Jiebo Luo, Xipeng Shen, Christopher M Brown","year":"2004","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brown et al., 2012] Gavin Brown, Adam Craig Pocock, Ming-Jie Zhao, and Mikel Luján. Conditional likelihood maximisation: A unifying framework for information the- oretic feature selection. Journal of Machine Learning Re- search, 13:27-66, 2012.","title":"Conditional likelihood maximisation: A unifying framework for information theoretic feature selection","authors":"Gavin Brown, Adam Craig Pocock, Ming-Jie Zhao, Mikel Luján","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Demsar, 2006] Janez Demsar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1-30, 2006.","title":"Janez Demsar. Statistical comparisons of classifiers over multiple data sets","authors":"Demsar","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Elisseeff and Weston, 2001] André Elisseeff and Jason We- ston. A kernel method for multi-labelled classification. In Advances in Neural Information Processing Systems 14, Vancouver, Canada, pages 681-687, 2001.","title":"A kernel method for multi-labelled classification","authors":"Weston ; Elisseeff, André Elisseeff, Jason Weston","year":"2001","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jian et al., 2016] Ling Jian, Jundong Li, Kai Shu, and Huan Liu. Multi-label informed feature selection. In Proceed- ings of the 25th International Joint Conference on Artifi- cial Intelligence, New York, NY, pages 1627-1633, 2016.","title":"Multi-label informed feature selection","authors":"Ling Jian, Jundong Li, Kai Shu, Huan Liu","year":"2016","venue":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2016/233"},{"position":7,"raw":"[Krawczyk, 2016] Bartosz Krawczyk. Learning from im- balanced data: Open challenges and future directions. Progress in Artificial Intelligence, 5(4):221-232, 2016.","title":"Learning from imbalanced data: Open challenges and future directions","authors":"Bartosz Krawczyk","year":"2016","venue":"Progress in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lee and Kim, 2013] Jae-Sung Lee and Dae-Won Kim. Fea- ture selection for multi-label classification using multi- variate mutual information. Pattern Recognition Letters, 34(3):349-357, 2013.","title":"Feature selection for multi-label classification using multivariate mutual information","authors":"Jae-Sung Lee, Dae-Won Kim","year":"2013","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lee and Kim, 2015] Jae-Sung Lee and Dae-Won Kim. Fast multi-label feature selection based on information- theoretic feature ranking. Pattern Recognition, 48(9):2761-2771, 2015.","title":"Fast multi-label feature selection based on informationtheoretic feature ranking","authors":"Jae-Sung Lee, Dae-Won Kim","year":"2015","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lee and Kim, 2017] Jae-Sung Lee and Dae-Won Kim. S- CLS: Multi-label feature selection based on scalable crite- rion for large label set. Pattern Recognition, 66:342-352, 2017.","title":"S-CLS: Multi-label feature selection based on scalable criterion for large label set","authors":"Lee, Kim ; ] Jae-Sung Lee, Dae-Won Kim","year":"2017","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2015] Yu-Kun Li, Min-Ling Zhang, and Xin Geng. Leveraging implicit relative labeling-importance informa- tion for effective multi-label learning. In Proceedings of the 2015 IEEE International Conference on Data Mining, Atlantic City, NJ, pages 251-260, 2015.","title":"Leveraging implicit relative labeling-importance information for effective multi-label learning","authors":"Yu-Kun Li, Min-Ling Zhang, Xin Geng","year":"2015","venue":"Proceedings of the 2015 IEEE International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018] Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. Feature selection: A data perspective. ACM Comput- ing Surveys, 50(6):94:1-94:45, 2018.","title":"Feature selection: A data perspective","authors":"Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, Huan Liu","year":"2018","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lim and Kim, 2016] Hyunki Lim and Dae-Won Kim. Con- vex optimization approach for multi-label feature selection based on mutual information. In Proceedings of the 23rd International Conference on Pattern Recognition, Cancún, Mexico, pages 1512-1517, 2016.","title":"Convex optimization approach for multi-label feature selection based on mutual information","authors":"Kim ; ] Hyunki Lim, Dae-Won Lim, Kim","year":"2016","venue":"Proceedings of the 23rd International Conference on Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2015] Yaojin Lin, Qinghua Hu, Jinghua Liu, and Jie Duan. Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing, 168:92-103, 2015.","title":"Multi-label feature selection based on max-dependency and min-redundancy","authors":"Lin","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2016] Yaojin Lin, Qinghua Hu, Jia Zhang, and Xindong Wu. Multi-label feature selection with stream- ing labels. Information Sciences, 372:256-275, 2016.","title":"Multi-label feature selection with streaming labels","authors":"Yaojin Lin, Qinghua Hu, Jia Zhang, Xindong Wu","year":"2016","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Peng et al., 2005] Hanchuan Peng, Fuhui Long, and Chris H. Q. Ding. Feature selection based on mutual infor- mation: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1226-1238, 2005.","title":"Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy","authors":"Peng","year":"2005","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Schapire and Singer, 2000] Robert E. Schapire and Yoram Singer. Boostexter: A boosting-based system for text cat- egorization. Machine Learning, 39(2/3):135-168, 2000.","title":"Boostexter: A boosting-based system for text categorization","authors":"Robert E Schapire, Yoram Singer","year":"2000","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sechidis et al., 2014] Konstantinos Sechidis, Nikolaos Nikolaou, and Gavin Brown. Information theoretic feature selection in multi-label data through composite likelihood. In Structural, Syntactic, and Statistical Pattern Recognition -Joint IAPR International Workshop, Joensuu, Finland, pages 143-152, 2014.","title":"Information theoretic feature selection in multi-label data through composite likelihood","authors":"Konstantinos Sechidis, Nikolaos Nikolaou, Gavin Brown","year":"2014","venue":"Structural, Syntactic, and Statistical Pattern Recognition -Joint IAPR International Workshop","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sun et al., 2019] Zhenqiang Sun, Jia Zhang, Liang Dai, Candong Li, Changen Zhou, Jiliang Xin, and Shaozi Li. Mutual information based multi-label feature selection via constrained convex optimization. Neurocomputing, 329:447-456, 2019.","title":"Mutual information based multi-label feature selection via constrained convex optimization","authors":"Zhenqiang Sun, Jia Zhang, Liang Dai, Candong Li, Changen Zhou, Jiliang Xin, Shaozi Li","year":"2019","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Vergara and Estévez, 2014] Jorge R. Vergara and Pablo A. Estévez. A review of feature selection methods based on mutual information. Neural Computing and Applications, 24(1):175-186, 2014.","title":"A review of feature selection methods based on mutual information","authors":"Jorge R Vergara, Pablo A Estévez","year":"2014","venue":"Neural Computing and Applications","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wei and Li, 2019] Tong Wei and Yu-Feng Li. Learning compact model for large-scale multi-label data. In Pro- ceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, Hawaii, pages 5385-5392, 2019.","title":"Learning compact model for large-scale multi-label data","authors":"Tong Wei, Yu-Feng Li","year":"2019","venue":"Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu and Zhou, 2017] Xi-Zhu Wu and Zhi-Hua Zhou. A u- nified view of multi-label performance measures. In Pro- ceedings of the 34th International Conference on Machine Learning, Sydney, Australia, pages 3780-3788, 2017.","title":"A unified view of multi-label performance measures","authors":"Xi-Zhu Wu, Zhi-Hua Zhou","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2014] Baoyuan Wu, Zhilei Liu, Shangfei Wang, Bao-Gang Hu, and Qiang Ji. Multi-label learning with missing labels. In Proceedings of the 22nd International Conference on Pattern Recognition, Stockholm, Sweden, pages 1964-1968, 2014.","title":"Multi-label learning with missing labels","authors":"Wu","year":"2014","venue":"Proceedings of the 22nd International Conference on Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Wu, 2015] Min-Ling Zhang and Lei Wu. LIFT: Multi-label learning with label-specific features. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 37(1):107-120, 2015.","title":"LIFT: Multi-label learning with label-specific features","authors":"Wu ; ] Min-Ling Zhang, Lei Zhang, Wu","year":"2015","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang and Zhou, 2007] Min-Ling Zhang and Zhi-Hua Zhou. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition, 40(7):2038-2048, 2007.","title":"ML-KNN: A lazy learning approach to multi-label learning","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2007","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2019] Jia Zhang, Zhiming Luo, Candong Li, Changen Zhou, and Shaozi Li. Manifold regularized dis- criminative feature selection for multi-label learning. Pat- tern Recognition, 95:136-150, 2019.","title":"Manifold regularized discriminative feature selection for multi-label learning","authors":"Jia Zhang, Zhiming Luo, Candong Li, Changen Zhou, Shaozi Li","year":"2019","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}