{"type":"ScholarlyArticle","title":"Deep Active Learning with Adaptive Acquisition","authors":[{"name":"Manuel Haussmann","slug":"manuel-haussmann","given":"Manuel","family":"Haussmann","affiliation":"HCI/IWR , Heidelberg University , Germany"},{"name":"Fred Hamprecht","slug":"fred-hamprecht","given":"Fred","family":"Hamprecht","affiliation":"HCI/IWR , Heidelberg University , Germany"},{"name":"Melih Kandemir","slug":"melih-kandemir","given":"Melih","family":"Kandemir","affiliation":"Bosch Center for Artificial Intelligence , Renningen , Germany"}],"abstract":"Model selection is treated as a standard performance boosting step in many machine learning applications. Once all other properties of a learning problem are fixed, the model is selected by grid search on a held-out validation set. This is strictly inapplicable to active learning. Within the standardized workflow, the acquisition function is chosen among available heuristics a priori, and its success is observed only after the labeling budget is already exhausted. More importantly, none of the earlier studies report a unique consistently successful acquisition heuristic to the extent to stand out as the unique best choice. We present a method to break this vicious circle by defining the acquisition function as a learning predictor and training it by reinforcement feedback collected from each labeling round. As active learning is a scarce data regime, we bootstrap from a well-known heuristic that filters the bulk of data points on which all heuristics would agree, and learn a policy to warp the top portion of this ranking in the most beneficial way for the character of a specific data distribution. Our system consists of a Bayesian neural net, the predictor, a bootstrap acquisition function, a probabilistic state definition, and another Bayesian policy network that can effectively incorporate this input distribution. We observe on three benchmark data sets that our method always manages to either invent a new superior acquisition function or to adapt itself to the a priori unknown best performing heuristic for each specific data set.","doi":"10.24963/ijcai.2019/343","url":"https://ijcai.org/proceedings/2019/343","pdf":"https://ijcai.org/proceedings/2019/0343.pdf","markdown":"https://ijcai.org/proceedings/2019/343.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/343","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":343,"pages":{"first":2470,"last":2476},"keywords":["Machine Learning: Deep Learning","Uncertainty in AI: Approximate Probabilistic Inference","Humans and AI: Human-AI Collaboration","Machine Learning: Probabilistic Machine Learning","Machine Learning: Active Learning"],"videos":[],"references":[{"position":1,"raw":"[Baram et al., 2004] Y. Baram, R. Yaniv, and K. Luz. Online choice of active learning algorithms. JMLR, 2004.","title":"Online choice of active learning algorithms","authors":"Y Baram, R Yaniv, K Luz","year":"2004","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Blundell et al., 2015] C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra. Weight uncertainty in neural network. In ICML, 2015.","title":"Weight uncertainty in neural network","authors":"C Blundell, J Cornebise, K Kavukcuoglu, D Wierstra","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Callaway et al., 2018] F. Callaway, S. Gul, P.M. Krueger, T.L. Griffiths, and F. Lieder. Learning to select compu- tations. 2018.","title":"Learning to select computations","authors":"F Callaway, S Gul, P M Krueger, T L Griffiths, F Lieder","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chu and Lin, 2016] H. Chu and H. Lin. Can active learning experience be transferred? In ICDM, 2016.","title":"Can active learning experience be transferred?","authors":"H Chu, H Lin","year":"2016","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deisenroth and Rasmussen, 2011] M. Deisenroth and C. E. Rasmussen. Pilco: A model-based and data-efficient ap- proach to policy search. In ICML, 2011.","title":"Pilco: A model-based and data-efficient approach to policy search","authors":"M Deisenroth, C E Rasmussen","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Depeweg et al., 2018] S. Depeweg, J. Hernandez-Lobato, F. Doshi-Velez, and S. Udluft. Decomposition of uncer- tainty in Bayesian deep learning for efficient and risk- sensitive learning. In ICML, 2018.","title":"Decomposition of uncertainty in Bayesian deep learning for efficient and risksensitive learning","authors":"S Depeweg, J Hernandez-Lobato, F Doshi-Velez, S Udluft","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ebert et al., 2012] S. Ebert, M. Fritz, and B. Schiele. Ralf: A reinforced active learning formulation for object class recognition. In CVPR, 2012.","title":"Ralf: A reinforced active learning formulation for object class recognition","authors":"S Ebert, M Fritz, B Schiele","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2017] M. Fang, Y. Li, and T. Cohn. Learning how to active learn: A deep reinforcement learning ap- proach. In EMNLP, 2017.","title":"Learning how to active learn: A deep reinforcement learning approach","authors":"M Fang, Y Li, T Cohn","year":"2017","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Frey and Hinton, 1999] B. J. Frey and G. E. Hinton. Varia- tional learning in nonlinear gaussian belief networks. Neu- ral Computation, 1999.","title":"Variational learning in nonlinear gaussian belief networks","authors":"B J Frey, G E Hinton","year":"1999","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gal and Ghahramani, 2016] Y. Gal and Z. Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In ICML, 2016.","title":"Dropout as a bayesian approximation: Representing model uncertainty in deep learning","authors":"Y Gal, Z Ghahramani","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gal et al., 2017] Y. Gal, R. Islam, and Z. Ghahramani. Deep Bayesian active learning with image data. In ICML, 2017.","title":"Deep Bayesian active learning with image data","authors":"Y Gal, R Islam, Z Ghahramani","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gast and Roth, 2018] J. Gast and S. Roth. Lightweight probabilistic deep networks. In CVPR, 2018.","title":"Lightweight probabilistic deep networks","authors":"J Gast, S Roth","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Haussmann et al., 2019] M. Haussmann, F.A. Hamprecht, and M. Kandemir. Sampling-free variational inference for bayesian neural networks by variance backpropaga- tion. UAI, 2019.","title":"Sampling-free variational inference for bayesian neural networks by variance backpropagation","authors":"M Haussmann, F A Hamprecht, M Kandemir","year":"2019","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hernández-Lobato and Adams, 2015] J.M. Hernández- Lobato and R. Adams. Probabilistic backpropagation for scalable learning of bayesian neural networks. In ICML, 2015.","title":"Probabilistic backpropagation for scalable learning of bayesian neural networks","authors":"Hernández-Lobato And Adams, ; ] J M Hernández-Lobato, R Adams","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Houlsby et al., 2012] N. Houlsby, F. Huszar, Z. Ghahra- mani, and J.M Hernández-Lobato. Collaborative gaussian processes for preference learning. In NIPS, 2012.","title":"Collaborative gaussian processes for preference learning","authors":"N Houlsby, F Huszar, Z Ghahramani, J M Hernández-Lobato","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hsu and Lin, 2015] W. Hsu and H. Lin. Active learning by learning. In AAAI, 2015.","title":"Active learning by learning","authors":"W Hsu, H Lin","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kingma and Ba, 2015] D.P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"D P Kingma, J Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kingma and Welling, 2014] D.P. Kingma and M. Welling. Auto-encoding variational bayes. In ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"D P Kingma, M Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kingma et al., 2015] D.P. Kingma, T. Salimans, and M. Welling. Variational dropout and the local reparame- terization trick. In NIPS, 2015.","title":"Variational dropout and the local reparameterization trick","authors":"D P Kingma, T Salimans, M Welling","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Konyushkova et al., 2017] K. Konyushkova, R. Sznitman, and P. Fua. Learning active learning from data. In NIPS. 2017.","title":"Learning active learning from data","authors":"K Konyushkova, R Sznitman, P Fua","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Louizos et al., 2017] C. Louizos, K. Ullrich, and M. Welling. Bayesian compression for deep learn- ing. In NIPS, 2017.","title":"Bayesian compression for deep learning","authors":"C Louizos, K Ullrich, M Welling","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Molchanov et al., 2017] D. Molchanov, A. Ashukha, and D. Vetrov. Variational dropout sparsifies deep neural net- works. In ICML, 2017.","title":"Variational dropout sparsifies deep neural networks","authors":"D Molchanov, A Ashukha, D Vetrov","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pang et al., 2018] K. Pang, M. Dong, Y. Wu, and T. Hospedales. Meta-learning transferable active learning policies by deep reinforcement learning. arXiv preprint, 2018.","title":"Meta-learning transferable active learning policies by deep reinforcement learning","authors":"K Pang, M Dong, Y Wu, T Hospedales","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Qiu et al., 2017] Z. Qiu, D.J. Miller, and G. Kesidis. A max- imum entropy framework for semisupervised and active learning with unknown and label-scarce classes. IEEE transactions on neural networks and learning systems, 2017.","title":"A maximum entropy framework for semisupervised and active learning with unknown and label-scarce classes","authors":"Z Qiu, D J Miller, G Kesidis","year":"2017","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Schulman et al., 2015] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz. Trust region policy optimization. In ICML, 2015.","title":"Trust region policy optimization","authors":"J Schulman, S Levine, P Abbeel, M Jordan, P Moritz","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulman et al., 2017] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint, 2017.","title":"Proximal policy optimization algorithms","authors":"J Schulman, F Wolski, P Dhariwal, A Radford, O Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sener and Savarese, 2018] O. Sener and S. Savarese. Ac- tive learning for convolutional neural networks: Acore-set approach. In CVPR, 2018.","title":"Active learning for convolutional neural networks: Acore-set approach","authors":"O Sener, S Savarese","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Settles, 2012] B. Settles. Active learning. Synthesis Lec- tures on Artificial Intelligence and Machine Learning, 2012.","title":"Active learning","authors":"B Settles","year":"2012","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Srinivas et al., 2012] N. Srinivas, A. Krause, S.M. Kakade, and M.W. Seeger. Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory, 2012.","title":"Information-theoretic regret bounds for gaussian process optimization in the bandit setting","authors":"N Srinivas, A Krause, S M Kakade, M W Seeger","year":"2012","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang and Manning, 2013] S. Wang and C. Manning. Fast dropout training. In ICML, 2013.","title":"Fast dropout training","authors":"S Wang, C Manning","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2017] K. Wang, D. Zhang, Y. Li, R. Zhang, and L. Lin. Cost-effective active learning for deep image clas- sification. IEEE Transactions on Circuits and Systems for Video Technology, 2017.","title":"Cost-effective active learning for deep image classification","authors":"K Wang, D Zhang, Y Li, R Zhang, L Lin","year":"2017","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Williams, 1992] R.J. Williams. Simple statistical gradient- following algorithms for connectionist reinforcement learning. Machine learning, 1992.","title":"Simple statistical gradientfollowing algorithms for connectionist reinforcement learning","authors":"R J Williams","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wu et al., 2019] A. Wu, S. Nowozin, E. Meeds, R. E. Turner, J.M. Hernández-Lobato, and A.L. Gaunt. Deter- ministic variational inference for robust bayesian neural networks. ICLR, 2019.","title":"Deterministic variational inference for robust bayesian neural networks","authors":"A Wu, S Nowozin, E Meeds, R E Turner, J M Hernández-Lobato, A L Gaunt","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}