{"type":"ScholarlyArticle","title":"Towards Real-Time DNN Inference on Mobile Platforms with Model Pruning and Compiler Optimization","authors":[{"name":"Wei Niu","slug":"wei-niu","given":"Wei","family":"Niu","affiliation":"College of William and"},{"name":"Pu Zhao","slug":"pu-zhao","given":"Pu","family":"Zhao","affiliation":"Northeastern University"},{"name":"Zheng Zhan","slug":"zheng-zhan","given":"Zheng","family":"Zhan","affiliation":"Northeastern University"},{"name":"Xue Lin","slug":"xue-lin","given":"Xue","family":"Lin","affiliation":"Northeastern University"},{"name":"Yanzhi Wang","slug":"yanzhi-wang","given":"Yanzhi","family":"Wang","affiliation":"Northeastern University"},{"name":"Bin Ren","slug":"bin-ren","given":"Bin","family":"Ren","affiliation":"College of William and"}],"abstract":"High-end mobile platforms rapidly serve as primary computing devices for a wide range of Deep Neural Network (DNN) applications. However, the constrained computation and storage resources on these devices still pose significant challenges for real-time DNN inference executions. To address this problem, we propose a set of hardware-friendly structured model pruning and compiler optimization techniques to accelerate DNN executions on mobile devices. This demo shows that these optimizations can enable real-time mobile execution of multiple DNN applications, including style transfer, DNN coloring and super resolution.","doi":"10.24963/ijcai.2020/778","url":"https://ijcai.org/proceedings/2020/778","pdf":"https://ijcai.org/proceedings/2020/0778.pdf","markdown":"https://ijcai.org/proceedings/2020/778.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/778","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Demos","track":"Demos","number":778,"pages":{"first":5306,"last":5308},"keywords":["Machine Learning: general","Computer Vision: general"],"videos":[],"references":[{"position":1,"raw":"[Boyd et al., 2011] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed op- timization and statistical learning via the alternating direc- tion method of multipliers. Foundations and Trends R in Machine learning, 3(1):1-122, 2011.","title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","authors":"Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein","year":"2011","venue":"Foundations and Trends R in Machine learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2018] Tianqi Chen, Thierry Moreau, Ziheng Jiang, et al. {TVM}: An automated end-to-end optimiz- ing compiler for deep learning. In 13th USENIX, pages 578-594, 2018.","title":"{TVM}: An automated end-to-end optimizing compiler for deep learning","authors":"Tianqi Chen, Thierry Moreau, Ziheng Jiang","year":"2018","venue":"13th USENIX","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dong et al., 2014] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Learning a deep convolutional net- work for image super-resolution. In ECCV, 2014.","title":"Learning a deep convolutional network for image super-resolution","authors":"Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gatys et al., 2016] Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In CVPR, pages 2414-2423, 2016.","title":"Image style transfer using convolutional neural networks","authors":"Leon A Gatys, Alexander S Ecker, Matthias Bethge","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.","title":"Deep Learning","authors":"Ian Goodfellow, Yoshua Bengio, Aaron Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guo et al., 2016] Yiwen Guo, Anbang Yao, and Yurong Chen. Dynamic network surgery for efficient dnns. In NeurIPS, pages 1379-1387, 2016.","title":"Dynamic network surgery for efficient dnns","authors":"Yiwen Guo, Anbang Yao, Yurong Chen","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Howard et al., 2017] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mo- bilenets: Efficient convolutional neural networks for mo- bile vision applications. arXiv:1704.04861, 2017.","title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications","authors":"Menglong Andrew G Howard, Bo Zhu, Dmitry Chen, Weijun Kalenichenko, Tobias Wang, Marco Weyand, Hartwig Andreetto, Adam","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Iizuka et al., 2016] Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Let there be color! joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph., 35(4), July 2016.","title":"Let there be color! joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification","authors":"Satoshi Iizuka, Edgar Simo-Serra, Hiroshi Ishikawa","year":"2016","venue":"ACM Trans. Graph","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lane et al., 2015] Nicholas D Lane, Sourav Bhattacharya, Petko Georgiev, Claudio Forlivesi, and Fahim Kawsar. An early resource characterization of deep learning on wear- ables, smartphones and internet-of-things devices. In IoT- App, 2015.","title":"An early resource characterization of deep learning on wearables, smartphones and internet-of-things devices","authors":"Sourav Nicholas D Lane, Petko Bhattacharya, Claudio Georgiev, Fahim Forlivesi, Kawsar","year":"2015","venue":"IoT-App","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, et al. Microsoft coco: Common objects in context. In ECCV, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2019] Ning Liu, Xiaolong Ma, et al. Autoslim: An automatic dnn structured pruning framework for ultra- high compression rates. arXiv:1907.03141, 2019.","title":"Autoslim: An automatic dnn structured pruning framework for ultrahigh compression rates","authors":"Ning Liu, Xiaolong Ma","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Luo and Wu, 2017] Jian-Hao Luo and Jianxin Wu. An entropy-based pruning method for cnn compression. arXiv:1706.05791, 2017.","title":"An entropy-based pruning method for cnn compression","authors":"Jian-Hao Luo, Jianxin Wu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ma et al., 2019] Xiaolong Ma, Fu-Ming Guo, Wei Niu, Xue Lin, Jian Tang, Kaisheng Ma, Bin Ren, and Yanzhi Wang. Pconv: The missing but desirable sparsity in dnn weight pruning for real-time execution on mobile devices. arXiv:1909.05073, 2019.","title":"Pconv: The missing but desirable sparsity in dnn weight pruning for real-time execution on mobile devices","authors":"Xiaolong Ma, Fu-Ming Guo, Wei Niu, Xue Lin, Jian Tang, Kaisheng Ma, Bin Ren, Yanzhi Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mao et al., 2017] Huizi Mao, Song Han, Jeff Pool, Wen- shuo Li, et al. Exploring the regularity of sparse struc- ture in convolutional neural networks. arXiv:1705.08922, 2017.","title":"Exploring the regularity of sparse structure in convolutional neural networks","authors":"Mao","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mobile Neural Network, 2019] Mobile Neural Network. Mobile neural network. https://github.com/alibaba/MNN, 2019.","title":"Mobile Neural Network. Mobile neural network","authors":null,"year":"2019","venue":null,"doi":null,"url":"https://github.com/alibaba/MNN,2019","internal_url":null},{"position":17,"raw":"[Philipp et al., 2011] Damian Philipp, Frank Durr, and Kurt Rothermel. A sensor network abstraction for flexible pub- lic sensing systems. In MASS, pages 460-469. IEEE, 2011.","title":"A sensor network abstraction for flexible public sensing systems","authors":"Damian Philipp, Frank Durr, Kurt Rothermel","year":"2011","venue":"MASS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tensorflow Lite, 2017] Tensorflow Lite. tensorflow lite. https://www.tensorflow.org/mobile/tflite/, 2017.","title":"Tensorflow Lite. tensorflow lite","authors":"Tensorflow Lite","year":"2017","venue":null,"doi":null,"url":"https://www.tensorflow.org/mobile/tflite/,2017","internal_url":null},{"position":20,"raw":"[Timofte et al., 2017] Radu Timofte, Eirikur Agustsson, Luc Van Gool, et al. Ntire 2017 challenge on single image super-resolution: Methods and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 114-125, 2017.","title":"Ntire 2017 challenge on single image super-resolution: Methods and results","authors":"Radu Timofte, Eirikur Agustsson, Luc Van Gool","year":"2017","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition workshops","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wen et al., 2016] Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Learning structured sparsity in deep neural networks. In NeurIPS, 2016.","title":"Learning structured sparsity in deep neural networks","authors":"Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, Hai Li","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Weng et al., 2020] Tsui-Wei Weng, Pu Zhao, Sijia Liu, Pin- Yu Chen, et al. Towards certificated model robustness against weight perturbations. In AAAI, 2020.","title":"Towards certificated model robustness against weight perturbations","authors":"Pu Tsui-Wei Weng, Sijia Zhao, Pin-Yu Liu, Chen","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2018] Jiahui Yu, Yuchen Fan, Jianchao Yang, et al. Wide activation for efficient and accurate image super- resolution. arXiv:1808.08718, 2018.","title":"Wide activation for efficient and accurate image superresolution","authors":"Jiahui Yu, Yuchen Fan, Jianchao Yang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Dana, 2017] Hang Zhang and Kristin Dana. Multi-style generative network for real-time transfer. arXiv:1703.06953, 2017.","title":"Multi-style generative network for real-time transfer","authors":"Dana ; Zhang, Hang Zhang, Kristin Dana","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2018] Pu Zhao, Sijia Liu, Yanzhi Wang, and Xue Lin. An admm-based universal framework for ad- versarial attacks on deep neural networks. In ACM Multi- media 2018, 2018.","title":"An admm-based universal framework for adversarial attacks on deep neural networks","authors":"Pu Zhao, Sijia Liu, Yanzhi Wang, Xue Lin","year":"2018","venue":"ACM Multimedia 2018","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhao et al., 2019a] Pu Zhao, Sijia Liu, Pin-Yu Chen, Nghia Hoang, et al. On the design of black-box adversarial exam- ples by leveraging gradient-free optimization and operator splitting method. In ICCV 2019, October 2019.","title":"On the design of black-box adversarial examples by leveraging gradient-free optimization and operator splitting method","authors":"Zhao","year":"2019","venue":"ICCV 2019","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2019b] Pu Zhao, Siyue Wang, Cheng Gongye, et al. Fault sneaking attack: a stealthy framework for mis- leading deep neural networks. In ICCAD, 2019.","title":"Fault sneaking attack: a stealthy framework for misleading deep neural networks","authors":"Zhao","year":"2019","venue":"ICCAD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2014] Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, et al. Learning deep features for scene recognition using places database. In NeurIPS, pages 487-495, 2014.","title":"Learning deep features for scene recognition using places database","authors":"Zhou","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}