{"type":"ScholarlyArticle","title":"Rapid Performance Gain through Active Model Reuse","authors":[{"name":"Feng Shi","slug":"feng-shi","given":"Feng","family":"Shi","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"},{"name":"Yu-Feng Li","slug":"yu-feng-li","given":"Yu-Feng","family":"Li","affiliation":"National Key Laboratory for Novel Software Technology Nanjing University , Nanjing 210023 , China"}],"abstract":"Model reuse aims at reducing the need of learning resources for a newly target task. In previous model reuse studies, the target task usually receives labeled data passively, which results in a slow performance improvement. However, learning models for target tasks are often required to achieve good enough performance rapidly for practical usage. In this paper, we propose the AcMR (Active Model Reuse) method for the rapid performance improvement problem. Firstly, we construct queries through pre-trained models to facilitate the active learner when labeled examples are insufficient in the target task. Secondly, we consider that pre-trained models are able to filter out not-very-necessary queries so that AcMR can save considerable queries compared with direct active learning. Theoretical analysis verifies that AcMR requires fewer queries than direct active learning. Experimental results validate the effectiveness of AcMR.","doi":"10.24963/ijcai.2019/472","url":"https://ijcai.org/proceedings/2019/472","pdf":"https://ijcai.org/proceedings/2019/0472.pdf","markdown":"https://ijcai.org/proceedings/2019/472.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/472","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":472,"pages":{"first":3404,"last":3410},"keywords":["Machine Learning: Active Learning","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Albrecht, 2016] Jan Philipp Albrecht. How the gdpr will change the world. European Data Protection Law Review, 2:287, 2016.","title":"How the gdpr will change the world","authors":"Jan Philipp, Albrecht","year":"2016","venue":"European Data Protection Law Review","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chakraborty et al., 2015] Shayok Chakraborty, Vineeth Balasubramanian, Qian Sun, Sethuraman Panchanathan, and Jie-Ping Ye. Active batch selection via convex relaxations with guaranteed solution bounds. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 37(10):1945-1958, 2015.","title":"Active batch selection via convex relaxations with guaranteed solution bounds","authors":"Shayok Chakraborty, Vineeth Balasubramanian, Qian Sun, Sethuraman Panchanathan, Jie-Ping Ye","year":"2015","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dietterich, 2000] Thomas G Dietterich. Ensemble methods in machine learning. In International Workshop on Mul- tiple Classifier Systems, pages 1-15, Günzburg, Germany, 2000.","title":"Ensemble methods in machine learning","authors":"G Thomas, Dietterich","year":"2000","venue":"International Workshop on Multiple Classifier Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Harrell, 2015] Frank E Harrell. Ordinal logistic regression. In Regression modeling strategies, pages 311-325. 2015.","title":"Ordinal logistic regression","authors":"E Frank, Harrell","year":"2015","venue":"Regression modeling strategies","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[He et al., 2018] Kai-Ming He, Ross Girshick, and Piotr Dollár. Rethinking imagenet pre-training. arXiv preprint arXiv:1811.08883, 2018.","title":"Rethinking imagenet pre-training","authors":"Kai-Ming He, Ross Girshick, Piotr Dollár","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Huang et al., 2010] Sheng-Jun Huang, Rong Jin, and Zhi- Hua Zhou. Active learning by querying informative and representative examples. In Advances in Neural Infor- mation Processing Systems, pages 892-900, Vancouver, Canada, 2010.","title":"Active learning by querying informative and representative examples","authors":"Sheng-Jun Huang, Rong Jin, Zhi-Hua Zhou","year":"2010","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lewis and Gale, 1994] David D Lewis and William A Gale. A sequential algorithm for training text classifiers. In Pro- ceedings of the 17th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3-12, 1994.","title":"A sequential algorithm for training text classifiers","authors":"D David, William A Lewis, Gale","year":"1994","venue":"Proceedings of the 17th International ACM SIGIR Conference on Research and Development in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li and Liang, 2019] Yu-Feng Li and De-Ming Liang. Safe semi-supervised learning: a brief introduction. Frontiers of Computer Science, pages 1-8, 2019.","title":"Safe semi-supervised learning: a brief introduction","authors":"Yu-Feng Li, De-Ming Liang","year":"2019","venue":"Frontiers of Computer Science","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2013] Nan Li, Ivor W Tsang, and Zhi-Hua Zhou. Efficient optimization of performance measures by classi- fier adaptation. IEEE transactions on Pattern Analysis and Machine Intelligence, 35(6):1370-1382, 2013.","title":"Efficient optimization of performance measures by classifier adaptation","authors":"Nan Li, Ivor W Tsang, Zhi-Hua Zhou","year":"2013","venue":"IEEE transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2017] Yu-Feng Li, Han-Wen Zha, and Zhi-Hua Zhou. Learning safe prediction for semi-supervised re- gression. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, volume 2017, pages 2217-2223, San Francisco, CA, 2017.","title":"Learning safe prediction for semi-supervised regression","authors":"Yu-Feng Li, Han-Wen Zha, Zhi-Hua Zhou","year":"2017","venue":"Proceedings of the 31st AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2019] Yu-Feng Li, Hai Wang, Tong Wei, and Wei- Wei Tu. Towards automated semi-supervised learning. In Proceedings of the 33rd AAAI conference on Artificial In- telligence, Honolulu, HI, 2019.","title":"Towards automated semi-supervised learning","authors":"Yu-Feng Li, Hai Wang, Tong Wei, Wei-Wei Tu","year":"2019","venue":"Proceedings of the 33rd AAAI conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Long et al., 2017] Ming-Sheng Long, Han Zhu, Jian-Min Wang, and Michael I Jordan. Deep transfer learning with joint adaptation networks. In Proceedings of the 34th International Conference on Machine Learning, pages 2208-2217, Sydney, Australia, 2017.","title":"Deep transfer learning with joint adaptation networks","authors":"Ming-Sheng Long, Han Zhu, Jian-Min Wang, Michael I Jordan","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Murugesan et al., 2016] Keerthiram Murugesan, Han-Xiao Liu, Jaime Carbonell, and Yi-Ming Yang. Adaptive smoothed online multi-task learning. In Advances in Neu- ral Information Processing Systems, pages 4296-4304, Barcelona, Spain, 2016.","title":"Adaptive smoothed online multi-task learning","authors":"Murugesan","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pan and Yang, 2010] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on Knowl- edge and Data Engineering, 22(10):1345-1359, 2010.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2010","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Pan et al., 2011] Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Net- works, 22(2):199-210, 2011.","title":"Domain adaptation via transfer component analysis","authors":"Ivor W Sinno Jialin Pan, James T Tsang, Qiang Kwok, Yang","year":"2011","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Parkhi et al., 2015] Omkar M. Parkhi, Andrea Vedaldi, and Andrew Zisserman. Deep face recognition. In Proceedings of the 26th British Machine Vision Conference, Swansea, UK, 2015.","title":"Deep face recognition","authors":"M Omkar, Andrea Parkhi, Andrew Vedaldi, Zisserman","year":"2015","venue":"Proceedings of the 26th British Machine Vision Conference","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Settles, 2012] Burr Settles. Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 6(1):1-114, 2012.","title":"Burr Settles. Active learning","authors":"Settles","year":"2012","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Seung et al., 1992] H Sebastian Seung, Opper Manfred, and Sompolinsky Haim. Query by committee. In Proceedings of the 5th Annual Workshop on Computational Learning Theory, pages 287-294, Pittsburgh, PA, 1992.","title":"Query by committee","authors":"Sebastian Seung, Manfred Opper, Sompolinsky Haim","year":"1992","venue":"Proceedings of the 5th Annual Workshop on Computational Learning Theory","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi et al., 2008] Xiao-Xiao Shi, Wei Fan, and Jiang-Tao Ren. Actively transfer domain knowledge. In Proceed- ings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 342-357, Antwerp, Belgium, 2008.","title":"Actively transfer domain knowledge","authors":"Xiao-Xiao Shi, Wei Fan, Jiang-Tao Ren","year":"2008","venue":"Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang and Ye, 2015] Zheng Wang and Jie-Ping Ye. Query- ing discriminative and representative samples for batch mode active learning. ACM Transactions on Knowledge Discovery from Data, 9(3):17, 2015.","title":"Querying discriminative and representative samples for batch mode active learning","authors":"Zheng Wang, Jie-Ping Ye","year":"2015","venue":"ACM Transactions on Knowledge Discovery from Data","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2017a] Yang Yang, De-Chuan Zhan, Ying Fan, Yuan Jiang, and Zhi-Hua Zhou. Deep learning for fixed model reuse. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, pages 2831-2837, San Fran- cisco, CA, 2017.","title":"Deep learning for fixed model reuse","authors":"Yang","year":"2017","venue":"Proceedings of the 31st AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2017b] Yang Yang, De-Chuan Zhan, Xiang-Yu Guo, and Yuan Jiang. Modal consistency based pre-trained multi-model reuse. 2017.","title":"Modal consistency based pre-trained multi-model reuse","authors":"Yang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2017/459"},{"position":24,"raw":"[Ye et al., 2018] Han-Jia Ye, De-Chuan Zhan, Yuan Jiang, and Zhi-Hua Zhou. Rectify heterogeneous models with semantic mapping. In Proceedings of the 35th Inter- national Conference on Machine Learning, pages 1904- 1913, Stockholm, Sweden, 2018.","title":"Rectify heterogeneous models with semantic mapping","authors":"Han-Jia Ye, Yuan De-Chuan Zhan, Zhi-Hua Jiang, Zhou","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou, 2012] Zhi-Hua Zhou. Ensemble Methods: founda- tions and algorithms. 2012.","title":"Ensemble Methods: foundations and algorithms","authors":"Zhi-Hua Zhou","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou, 2016] Zhi-Hua Zhou. Learnware: on the future of machine learning. Frontiers of Computer Science, 10(4):589-590, 2016.","title":"Learnware: on the future of machine learning","authors":"Zhi-Hua Zhou","year":"2016","venue":"Frontiers of Computer Science","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}