{"type":"ScholarlyArticle","title":"On Deep Unsupervised Active Learning","authors":[{"name":"Changsheng Li","slug":"changsheng-li","given":"Changsheng","family":"Li","affiliation":"School of Computer Science and Technology , Beijing Institute of Technology , Beijing , China"},{"name":"Handong Ma","slug":"handong-ma","given":"Handong","family":"Ma","affiliation":"SCSE , University of Electronic Science and Technology of China , Chengdu , China"},{"name":"Zhao Kang","slug":"zhao-kang","given":"Zhao","family":"Kang","affiliation":"SCSE , University of Electronic Science and Technology of China , Chengdu , China"},{"name":"Ye Yuan","slug":"ye-yuan","given":"Ye","family":"Yuan","affiliation":"School of Computer Science and Technology , Beijing Institute of Technology , Beijing , China"},{"name":"Xiao-Yu Zhang","slug":"xiao-yu-zhang","given":"Xiao-Yu","family":"Zhang","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences , Beijing , China"},{"name":"Guoren Wang","slug":"guoren-wang","given":"Guoren","family":"Wang","affiliation":"School of Computer Science and Technology , Beijing Institute of Technology , Beijing , China"}],"abstract":"Unsupervised active learning has attracted increasing attention in recent years, where its goal is to select representative samples in an unsupervised setting for human annotating. Most existing works are based on shallow linear models by assuming that each sample can be well approximated by the span (i.e., the set of all linear combinations) of certain selected samples, and then take these selected samples as representative ones to label. However, in practice, the data do not necessarily conform to linear models, and how to model nonlinearity of data often becomes the key point to success. In this paper, we present a novel Deep neural network framework for Unsupervised Active Learning, called DUAL. DUAL can explicitly learn a nonlinear embedding to map each input into a latent space through an encoder-decoder architecture, and introduce a selection block to select representative samples in the the learnt latent space. In the selection block, DUAL considers to simultaneously preserve the whole input patterns as well as the cluster structure of data. Extensive experiments are performed on six publicly available datasets, and experimental results clearly demonstrate the efficacy of our method, compared with state-of-the-arts.","doi":"10.24963/ijcai.2020/364","url":"https://ijcai.org/proceedings/2020/364","pdf":"https://ijcai.org/proceedings/2020/0364.pdf","markdown":"https://ijcai.org/proceedings/2020/364.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/364","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":364,"pages":{"first":2626,"last":2632},"keywords":["Machine Learning: Active Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24756/file_24756.m3u8","length_sec":308,"provider":"hls"}],"references":[{"position":1,"raw":"[Aggarwal et al., 2014] Charu C Aggarwal, Xiangnan Kong, Quanquan Gu, Jiawei Han, and S Yu Philip. Active learn- ing: A survey. In Data Classification, pages 599-634.","title":"Active learning: A survey","authors":"Xiangnan Charu C Aggarwal, Quanquan Kong, Jiawei Gu, S Han, Philip Yu","year":null,"venue":"Data Classification","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Chapman and Hall/CRC, 2014.","title":null,"authors":"Hall / Chapman, Crc","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Baram et al., 2004] Yoram Baram, Ran El Yaniv, and Kobi Luz. Online choice of active learning algorithms. JMLR, 5(3):255-291, 2004.","title":"Online choice of active learning algorithms","authors":"Baram","year":"2004","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai and He, 2011] Deng Cai and Xiaofei He. Mani- fold adaptive experimental design for text categorization. TKDE, 24(4):707-719, 2011.","title":"Manifold adaptive experimental design for text categorization","authors":"Deng Cai, Xiaofei He","year":"2011","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cai et al., 2019] Jia-Jia Cai, Jun Tang, Qing-Guo Chen, Yao Hu, Xiaobo Wang, and Sheng-Jun Huang. Multi-view ac- tive learning for video recommendation. In IJCAI, pages 2053-2059. AAAI Press, 2019.","title":"Multi-view active learning for video recommendation","authors":"Jia-Jia Cai, Jun Tang, Qing-Guo Chen, Yao Hu, Xiaobo Wang, Sheng-Jun Huang","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/284"},{"position":6,"raw":"[Chan, 1987] Tony F Chan. Rank revealing qr factorizations. Linear algebra and its applications, 88:67-82, 1987.","title":"Rank revealing qr factorizations","authors":"Tony F Chan","year":"1987","venue":"Linear algebra and its applications","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Elhamifar et al., 2013] Ehsan Elhamifar, Guillermo Sapiro, Allen Yang, and S. Shankar Sasrty. A convex optimization framework for active learning. In ICCV, pages 209-216, 2013.","title":"A convex optimization framework for active learning","authors":"Ehsan Elhamifar, Guillermo Sapiro, Allen Yang, S Shankar Sasrty","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Freund et al., 1997] Yoav Freund, H. Sebastian Seung, Eli Shamir, and Naftali Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28(2- 3):133-168, 1997.","title":"Selective sampling using the query by committee algorithm","authors":"Yoav Freund, H Sebastian Seung, Eli Shamir, Naftali Tishby","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[FY et al., 2015] Zhu FY, Xingliang Zhu, Xiang SM, Pan CH, et al. 10,000+ times accelerated robust subset selec- tion (arss). In AAAI, pages 3217-3223, 2015.","title":"10,000+ times accelerated robust subset selection (arss)","authors":"F Y Zhu, Xingliang Zhu, S M Xiang, C H Pan","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Haussmann et al., 2019] Manuel Haussmann, Fred Ham- precht, and Melih Kandemir. Deep active learning with adaptive acquisition. In IJCAI, pages 2470-2476, 2019.","title":"Deep active learning with adaptive acquisition","authors":"Manuel Haussmann, Fred Hamprecht, Melih Kandemir","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/343"},{"position":11,"raw":"[Hu et al., 2013] Yao Hu, Debing Zhang, Zhongming Jin, Deng Cai, and Xiaofei He. Active learning via neighbor- hood reconstruction. In IJCAI, pages 1415-1421, 2013.","title":"Active learning via neighborhood reconstruction","authors":"Yao Hu, Debing Zhang, Zhongming Jin, Deng Cai, Xiaofei He","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2010] ShengJun Huang, Rong Jin, and Zhi- Hua Zhou. Active learning by querying informative and representative examples. In NIPS, pages 892-900, 2010.","title":"Active learning by querying informative and representative examples","authors":"Huang","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jain and Kapoor, 2009] Prateek Jain and Ashish Kapoor. Active learning for large multi-class problems. In CVPR, pages 762-769, 2009.","title":"Active learning for large multi-class problems","authors":"Kapoor Jain, Prateek Jain, Ashish Kapoor","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Joshi et al., 2009] Ajay J Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classification. In CVPR, pages 2372-2379, 2009.","title":"Multi-class active learning for image classification","authors":"Ajay J Joshi, Fatih Porikli, Nikolaos Papanikolopoulos","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kapoor et al., 2007] Ashish Kapoor, Eric Horvitz, and Sumit Basu. Selective supervision: Guiding supervised learning with decision-theoretic active learning. In IJCAI, volume 7, pages 877-882, 2007.","title":"Selective supervision: Guiding supervised learning with decision-theoretic active learning","authors":"Ashish Kapoor, Eric Horvitz, Sumit Basu","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/141"},{"position":16,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2017] Qin Li, Xiaoshuang Shi, Linfei Zhou, Zhifeng Bao, and Zhenhua Guo. Active learning via lo- cal structure reconstruction. PRL, 92:81-88, 2017.","title":"Active learning via local structure reconstruction","authors":"Qin Li, Xiaoshuang Shi, Linfei Zhou, Zhifeng Bao, Zhenhua Guo","year":"2017","venue":"PRL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2019] Changsheng Li, Xiangfeng Wang, Weishan Dong, Junchi Yan, Qingshan Liu, and Hongyuan Zha. Joint active learning with feature selection via cur matrix decomposition. TPAMI, 41(6):1382-1396, 2019.","title":"Joint active learning with feature selection via cur matrix decomposition","authors":"Li","year":"2019","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using tsne. JMLR, 9(11):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mahoney and Drineas, 2009] Michael W Mahoney and Pet- ros Drineas. Cur matrix decompositions for improved data analysis. PNAS, 106(3):697-702, 2009.","title":"Cur matrix decompositions for improved data analysis","authors":"W Michael, Petros Mahoney, Drineas","year":"2009","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nie et al., 2013] Feiping Nie, Wang Hua, Heng Huang, and Chris Ding. Early active learning via robust representation and structured sparsity. In IJCAI, pages 1572-1578, 2013.","title":"Early active learning via robust representation and structured sparsity","authors":"Nie","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/234"},{"position":22,"raw":"[Papailiopoulos et al., 2014] Dimitris Papailiopoulos, Anas- tasios Kyrillidis, and Christos Boutsidis. Provable deter- ministic leverage score sampling. In KDD, pages 997- 1006, 2014.","title":"Provable deterministic leverage score sampling","authors":"Dimitris Papailiopoulos, Anastasios Kyrillidis, Christos Boutsidis","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Roweis and Saul, 2000] Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally lin- ear embedding. science, 290(5500):2323-2326, 2000.","title":"Nonlinear dimensionality reduction by locally linear embedding","authors":"T Sam, Lawrence K Roweis, Saul","year":"2000","venue":"science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Settles, 2009] Burr Settles. Active learning literature sur- vey. Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009.","title":"Burr Settles. Active learning literature survey","authors":"Settles","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shi and Shen, 2016] Lei Shi and Yi-Dong Shen. Diversi- fying convex transductive experimental design for active learning. In IJCAI, pages 1997-2003. AAAI Press, 2016.","title":"Diversifying convex transductive experimental design for active learning","authors":"Shen ; ] Lei Shi, Yi-Dong Shi, Shen","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/285"},{"position":26,"raw":"[Vijayanarasimhan and Grauman, 2014] Sudheendra Vi- jayanarasimhan and Kristen Grauman. Large-scale live active learning: Training object detectors with crawled data and crowds. IJCV, 108(1-2):97-114, 2014.","title":"Large-scale live active learning: Training object detectors with crawled data and crowds","authors":"Sudheendra Vijayanarasimhan, Kristen Grauman","year":"2014","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2006] Kai Yu, Jinbo Bi, and Volker Tresp. Active learning via transductive experimental design. In ICML, pages 1081-1088, 2006.","title":"Active learning via transductive experimental design","authors":"Kai Yu, Jinbo Bi, Volker Tresp","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2011] Lijun Zhang, Chun Chen, Jiajun Bu, Deng Cai, Xiaofei He, and Thomas S Huang. Active learning based on locally linear reconstruction. TPAMI, 33(10):2026-2038, 2011.","title":"Active learning based on locally linear reconstruction","authors":"Lijun Zhang, Chun Chen, Jiajun Bu, Deng Cai, Xiaofei He, Thomas S Huang","year":"2011","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2017] XiaoYu Zhang, Shupeng Wang, and X- iaochun Yun. Bidirectional active learning: A two-way exploration into unlabeled and labeled data set. TNNLS, 26(12):3034-3044, 2017.","title":"Bidirectional active learning: A two-way exploration into unlabeled and labeled data set","authors":"Zhang","year":"2017","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zheng and Ye, 2015] Wang Zheng and Jieping Ye. Query- ing discriminative and representative samples for batch mode active learning. ACM Transactions on Knowledge Discovery from Data, 9(3):1-23, 2015.","title":"Querying discriminative and representative samples for batch mode active learning","authors":"Ye ; ] Zheng, Zheng Wang, Jieping Ye","year":"2015","venue":"ACM Transactions on Knowledge Discovery from Data","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}