{"type":"ScholarlyArticle","title":"Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teachers","authors":[{"name":"Jingwen Ye","slug":"jingwen-ye","given":"Jingwen","family":"Ye","affiliation":"College of Computer Science and Technology , Zhejiang University , Hangzhou , China"},{"name":"Xinchao Wang","slug":"xinchao-wang","given":"Xinchao","family":"Wang","affiliation":"Department of Computer Science , Stevens Institute of Technology , New Jersey , United States"},{"name":"Yixin Ji","slug":"yixin-ji","given":"Yixin","family":"Ji","affiliation":"College of Computer Science and Technology , Zhejiang University , Hangzhou , China"},{"name":"Kairi Ou","slug":"kairi-ou","given":"Kairi","family":"Ou","affiliation":"Alibaba Group , Hangzhou , China"},{"name":"Mingli Song","slug":"mingli-song","given":"Mingli","family":"Song","affiliation":"College of Computer Science and Technology , Zhejiang University , Hangzhou , China"}],"abstract":"Many well-trained Convolutional Neural Network~(CNN) models have now been released online by developers for the sake of effortless reproducing. In this paper, we treat such pre-trained networks as teachers and explore how to learn a target student network for customized tasks, using multiple teachers that handle different tasks. We assume no human-labelled annotations are available, and each teacher model can be either single- or multi-task network, where the former is a degenerated case of the latter. The student model, depending on the customized tasks, learns the related knowledge filtered from the multiple teachers, and eventually masters the complete or a subset of expertise from all teachers. To this end, we adopt a layer-wise training strategy, which entangles the student's network block to be learned with the corresponding teachers. As demonstrated on several benchmarks, the learned student network achieves very promising results, even outperforming the teachers on the customized tasks.","doi":"10.24963/ijcai.2019/573","url":"https://ijcai.org/proceedings/2019/573","pdf":"https://ijcai.org/proceedings/2019/0573.pdf","markdown":"https://ijcai.org/proceedings/2019/573.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/573","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":573,"pages":{"first":4128,"last":4134},"keywords":["Machine Learning: Deep Learning","Machine Learning: Learning Graphical Models","Machine Learning: Developmental Learning","Machine Learning: Classification","Computer Vision: Structural and Model-Based Approaches, Knowledge Representation and Reasoning","Machine Learning: Dimensionality Reduction and Manifold Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2017] Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Manmohan Chandraker. Learning efficient object detection models with knowledge distillation. In NeurIPS, pages 742-751, 2017.","title":"Learning efficient object detection models with knowledge distillation","authors":"Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, Manmohan Chandraker","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"International journal of computer vision","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gao et al., 2017a] Binbin Gao, Chao Xing, Chenwei Xie, Jianxin Wu, and Xin Geng. Deep label distribution learn- ing with label ambiguity. TIP, 26(6):2825-2838, 2017.","title":"Deep label distribution learning with label ambiguity","authors":"Gao","year":"2017","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gao et al., 2017b] Jiyang Gao, Zijian Guo, Zhen Li, and Ram Nevatia. Knowledge concentration: Learning 100k object classifiers in a single cnn. arXiv, 2017.","title":"Knowledge concentration: Learning 100k object classifiers in a single cnn","authors":"Gao","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Han et al., 2014] Lei Han, Yu Zhang, Guojie Song, and Kunqing Xie. Encoding tree sparsity in multi-task learn- ing: A probabilistic framework. AAAI, pages 1854-1860, 2014.","title":"Encoding tree sparsity in multi-task learning: A probabilistic framework","authors":"Lei Han, Yu Zhang, Guojie Song, Kunqing Xie","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. ECCV, pages 630-645, 2016.","title":"Identity mappings in deep residual networks","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hinton et al., 2015] Geoffrey E Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural net- work. NeurIPS, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey E Hinton, Oriol Vinyals, Jeffrey Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. CVPR, pages 2261-2269, 2017.","title":"Densely connected convolutional networks","authors":"Gao Huang, Zhuang Liu, Laurens Van Der Maaten, Kilian Q Weinberger","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2018] Mingkun Huang, Yongbin You, Zhe- huai Chen, Yanmin Qian, and Kai Yu. Knowledge distilla- tion for sequence model. Proc. Interspeech, pages 3703- 3707, 2018.","title":"Knowledge distillation for sequence model","authors":"Huang","year":"2018","venue":"Proc. Interspeech","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin et al., 2014] Tsung Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Com- mon objects in context. ECCV, pages 740-755, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung Yi, Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Misra et al., 2016] Ishan Misra, Abhinav Shrivastava, Abhi- nav Gupta, and Martial Hebert. Cross-stitch networks for multi-task learning. computer vision and pattern recogni- tion, pages 3994-4003, 2016.","title":"Cross-stitch networks for multi-task learning","authors":"Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, Martial Hebert","year":"2016","venue":"computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Shen et al., 2019] Chengchao Shen, Xinchao Wang, Jie Song, Li Sun, and Mingli Song. Amalgamating knowledge towards comprehensive classification. In AAAI, 2019.","title":"Amalgamating knowledge towards comprehensive classification","authors":"Shen","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Wang et al., 2011] Xinchao Wang, Zhu Li, and Dacheng Tao. Subspaces indexing model on Grassmann manifold for image search. TIP, 20(9):2627-2635, 2011.","title":"Subspaces indexing model on Grassmann manifold for image search","authors":"Wang","year":"2011","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Wang et al., 2014] Xinchao Wang, Engin Turetken, Fran- cois Fleuret, and Pascal Fua. Tracking interacting objects optimally using integer programming. In ECCV, pages 17-32, 2014.","title":"Tracking interacting objects optimally using integer programming","authors":"Wang","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wang et al., 2016] Jiang Wang, Yi Yang, Junhua Mao, Zhi- heng Huang, Chang Huang, and Wei Xu. Cnn-rnn: A unified framework for multi-label image classification. In CVPR, pages 2285-2294, 2016.","title":"Cnn-rnn: A unified framework for multi-label image classification","authors":"Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, Wei Xu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2017] Zhouxia Wang, Tianshui Chen, Guanbin Li, Ruijia Xu, and Liang Lin. Multi-label image recogni- tion by recurrently discovering attentional regions. ICCV, pages 464-472, 2017.","title":"Multi-label image recognition by recurrently discovering attentional regions","authors":"Wang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2018] Fangfang Wang, Liming Zhao, Xi Li, Xinchao Wang, and Dacheng Tao. Geometry-aware scene text detection with instance transformation network. In CVPR, pages 1381-1389, 2018.","title":"Geometry-aware scene text detection with instance transformation network","authors":"Fangfang Wang, Liming Zhao, Xi Li, Xinchao Wang, Dacheng Tao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wei et al., 2014] Yunchao Wei, Wei Xia, Junshi Huang, Bingbing ni, Jian Dong, Yao Zhao, and Shuicheng Yan. Cnn: Single-label to multi-label. CoRR, 2014.","title":"Cnn: Single-label to multi-label","authors":"Yunchao Wei, Wei Xia, Junshi Huang, Bingbing Ni, Jian Dong, Yao Zhao, Shuicheng Yan","year":"2014","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wei et al., 2016] Yunchao Wei, Wei Xia, Min Lin, Junshi Huang, Bingbing Ni, Jian Dong, Yao Zhao, and Shuicheng Yan. Hcp: A flexible cnn framework for multi-label image classification. TPAMI, 38(9):1901-1907, 2016.","title":"Hcp: A flexible cnn framework for multi-label image classification","authors":"Yunchao Wei, Wei Xia, Min Lin, Junshi Huang, Bingbing Ni, Jian Dong, Yao Zhao, Shuicheng Yan","year":"2016","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2016] Qingyao Wu, Mingkui Tan, Hengjie Song, Jian Chen, and Michael K Ng. Ml-forest: A multi- label tree ensemble method for multi-label classification. TKDE, 28(10):2665-2680, 2016.","title":"Ml-forest: A multilabel tree ensemble method for multi-label classification","authors":"Wu","year":"2016","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yin et al., 2018] Xiaoqing Yin, Xinchao Wang, Jun Yu, Maojun Zhang, Pascal Fua, and Dacheng Tao. Fisheyerec- net: A multi-context collaborative deep network for fish- eye image rectification. In ECCV, pages 475-490, 2018.","title":"Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification","authors":"Xiaoqing Yin, Xinchao Wang, Jun Yu, Maojun Zhang, Pascal Fua, Dacheng Tao","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yu et al., 2017] Xiyu Yu, Tongliang Liu, Xinchao Wang, and Dacheng Tao. On compressing deep models by low rank and sparse decomposition. In CVPR, pages 7370- 7379., 2017.","title":"On compressing deep models by low rank and sparse decomposition","authors":"Xiyu Yu, Tongliang Liu, Xinchao Wang, Dacheng Tao","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yuying et al., 2018] Xing Yuying, Yu Guoxian, Domeni- coni Carlotta, Wang Jun, and Zhang Zili. Multi-label co- training. In IJCAI, pages 2882-2888, 2018.","title":"Multi-label cotraining","authors":"Xing Yuying, Yu Guoxian, Domeniconi Carlotta, Wang Jun, Zhang Zili","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Zhang, 2014] Cha Zhang and Zhengyou Zhang. Improving multiview face detection with multi-task deep convolutional neural networks. WACV, pages 1036-1041, 2014.","title":"Improving multiview face detection with multi-task deep convolutional neural networks","authors":"Zhang Zhang, Cha Zhang, Zhengyou Zhang","year":"2014","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2014] Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. Facial land- mark detection by deep multi-task learning. ECCV, pages 94-108, 2014.","title":"Facial landmark detection by deep multi-task learning","authors":"Zhang","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2018] Junjie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, and Jianfeng Lu. Multilabel image classifi- cation with regional latent semantic dependencies. TMM, 20(10):2801-2813, 2018.","title":"Multilabel image classification with regional latent semantic dependencies","authors":"Junjie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu","year":"2018","venue":"TMM","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2018] Wei Zhao, Benyou Wang, Jianbo Ye, Min Yang, Zhou Zhao, Ruotian Luo, and Yu Qiao. A multi-task learning approach for image captioning. In IJCAI, pages 1205-1211, 2018.","title":"A multi-task learning approach for image captioning","authors":"Wei Zhao, Benyou Wang, Jianbo Ye, Min Yang, Zhou Zhao, Ruotian Luo, Yu Qiao","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/168"},{"position":28,"raw":"[Zhou et al., 2018] Tao Zhou, Zhixin Li, Canlong Zhang, and Lan Lin. An improved convolutional neural network model with adversarial net for multi-label image classifi- cation. PRICAI, pages 38-46, 2018.","title":"An improved convolutional neural network model with adversarial net for multi-label image classification","authors":"Tao Zhou, Zhixin Li, Canlong Zhang, Lan Lin","year":"2018","venue":"PRICAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}