{"type":"ScholarlyArticle","title":"Energy-efficient Amortized Inference with Cascaded Deep Classifiers","authors":[{"name":"Jiaqi Guan","slug":"jiaqi-guan","given":"Jiaqi","family":"Guan","affiliation":"Tsinghua University; University of Illinois at Urbana-Champaign"},{"name":"Yang Liu","slug":"yang-liu","given":"Yang","family":"Liu","affiliation":"University of Illinois at Urbana-Champaign"},{"name":"Qiang Liu","slug":"qiang-liu","given":"Qiang","family":"Liu","affiliation":"University of Texas at Austin"},{"name":"Jian Peng","slug":"jian-peng","given":"Jian","family":"Peng","affiliation":"University of Illinois at Urbana-Champaign"}],"abstract":"Deep neural networks have been remarkable successful in various AI tasks but often cast high computation and energy cost for energy-constrained applications such as mobile sensing.  We address this problem by proposing a novel framework that optimizes the prediction accuracy and energy cost simultaneously, thus enabling effective cost-accuracy trade-off at test time. In our framework, each data instance is pushed into a cascade of deep neural networks with increasing sizes, and a selection module is used to sequentially determine when a sufficiently accurate classifier can be used for this data instance. The cascade of neural networks and the selection module are jointly trained in an end-to-end fashion by the REINFORCE algorithm to optimize a trade-off between the computational cost and the predictive accuracy. Our method is able to simultaneously improve the accuracy and efficiency by learning to assign easy instances to fast yet sufficiently accurate classifiers to save computation and energy cost, while assigning harder instances to deeper and more powerful classifiers to ensure satisfiable accuracy. Moreover, we demonstrate our method's effectiveness with extensive experiments on CIFAR-10/100, ImageNet32x32 and original ImageNet dataset.","doi":"10.24963/ijcai.2018/302","url":"https://ijcai.org/proceedings/2018/302","pdf":"https://ijcai.org/proceedings/2018/0302.pdf","markdown":"https://ijcai.org/proceedings/2018/302.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/302","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":302,"pages":{"first":2184,"last":2190},"keywords":["Machine Learning: Classification","Computer Vision: Computer Vision","Machine Learning: Reinforcement Learning"],"videos":[],"references":[{"position":1,"raw":"[Almahairi et al., 2016] Amjad Almahairi, Nicolas Ballas, Tim Cooijmans, Yin Zheng, Hugo Larochelle, and Aaron Courville. Dynamic capacity networks. In Interna- tional Conference on Machine Learning, pages 2549- 2558, 2016.","title":"Dynamic capacity networks","authors":"Amjad Almahairi, Nicolas Ballas, Tim Cooijmans, Yin Zheng, Hugo Larochelle, Aaron Courville","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2015] Emmanuel Bengio, Pierre Luc Bacon, Joelle Pineau, and Doina Precup. Conditional computation in neural networks for faster models. Computer Science, 164(1):34-44, 2015.","title":"Conditional computation in neural networks for faster models","authors":"Emmanuel Bengio, Pierre Luc Bacon, Joelle Pineau, Doina Precup","year":"2015","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bolukbasi et al., 2017] Tolga Bolukbasi, Joseph Wang, Ofer Dekel, and Venkatesh Saligrama. Adaptive neural networks for efficient inference. 2017. [Chrabaszcz et al., 2017]","title":"Adaptive neural networks for efficient inference","authors":"Tolga Bolukbasi, Joseph Wang, Ofer Dekel, Venkatesh Saligrama","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter. A downsampled vari- ant of imagenet as an alternative to the cifar datasets. arXiv preprint arXiv:1707.08819, 2017.","title":"A downsampled variant of imagenet as an alternative to the cifar datasets","authors":"Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Figurnov et al., 2017] Michael Figurnov, Maxwell D. Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, and Ruslan Salakhutdinov. Spatially adaptive computation time for residual networks. In IEEE Confer- ence on Computer Vision and Pattern Recognition, pages 1790-1799, 2017.","title":"Spatially adaptive computation time for residual networks","authors":"Michael Figurnov, Maxwell D Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, Ruslan Salakhutdinov","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Graves, 2016] Alex Graves. Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983, 2016.","title":"Adaptive computation time for recurrent neural networks","authors":"Alex Graves","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Han et al., 2015] Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural net- works with pruning, trained quantization and huffman cod- ing. Fiber, 56(4):3-7, 2015.","title":"Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding","authors":"Han","year":"2015","venue":"Fiber","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016a] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"He","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016b] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, pages 630-645. Springer, 2016.","title":"Identity mappings in deep residual networks","authors":"He","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Howard et al., 2017] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mo- bilenets: Efficient convolutional neural networks for mo- bile vision applications. arXiv preprint arXiv:1704.04861, 2017.","title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications","authors":"Menglong Andrew G Howard, Bo Zhu, Dmitry Chen, Weijun Kalenichenko, Tobias Wang, Marco Weyand, Hartwig Andreetto, Adam","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2017] Hanzhang Hu, Debadeepta Dey, J Andrew Bagnell, and Martial Hebert. Anytime neural networks via joint optimization of auxiliary losses. arXiv preprint arXiv:1708.06832, 2017.","title":"Anytime neural networks via joint optimization of auxiliary losses","authors":"Hu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Iandola et al., 2016] Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size. arXiv preprint arXiv:1602.07360, 2016.","title":"Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size","authors":"Song Forrest N Iandola, Matthew W Han, Khalid Moskewicz, William J Ashraf, Kurt Dally, Keutzer","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Interna- tional Conference on Machine Learning, pages 448-456, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krizhevsky and Hinton, 2009] Alex Krizhevsky and Geof- frey Hinton. Learning multiple layers of features from tiny images. 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural in- formation processing systems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lee et al., 2015] Chen-Yu Lee, Saining Xie, Patrick Gal- lagher, Zhengyou Zhang, and Zhuowen Tu. Deeply- supervised nets. In Artificial Intelligence and Statistics, pages 562-570, 2015.","title":"Deeplysupervised nets","authors":"Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, Zhuowen Tu","year":"2015","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2015] Haoxiang Li, Zhe Lin, Xiaohui Shen, Jonathan Brandt, and Gang Hua. A convolutional neural network cascade for face detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 5325-5334, 2015.","title":"A convolutional neural network cascade for face detection","authors":"Haoxiang Li, Zhe Lin, Xiaohui Shen, Jonathan Brandt, Gang Hua","year":"2015","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, and Michael Bernstein. Imagenet large scale visual recogni- tion challenge. International Journal of Computer Vision, 115(3):211-252, 2015.","title":"Imagenet large scale visual recognition challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein","year":"2015","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. Computer Science, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":"Computer Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Szegedy et al., 2014] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. pages 1-9, 2014.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Szegedy et al., 2017] Christian Szegedy, Sergey Ioffe, Vin- cent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI, pages 4278-4284, 2017.","title":"Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning","authors":"Szegedy","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Williams, 1992] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine learning, 8(3-4):229-256, 1992.","title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","authors":"Williams Ronald","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang et al., 2016] Fan Yang, Wongun Choi, and Yuanqing Lin. Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejec- tion classifiers. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2129- 2137, 2016.","title":"Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers","authors":"Fan Yang, Wongun Choi, Yuanqing Lin","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zagoruyko and Komodakis, 2016] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.","title":"Wide residual networks","authors":"Sergey Zagoruyko, Nikos Komodakis","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2017] Xiangyu Zhang, Xinyu Zhou, Mengx- iao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. arXiv preprint arXiv:1707.01083, 2017.","title":"Shufflenet: An extremely efficient convolutional neural network for mobile devices","authors":"Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, Jian Sun","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}