{"type":"ScholarlyArticle","title":"Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems","authors":[{"name":"Seyed Iman Mirzadeh","slug":"seyed-iman-mirzadeh","given":"Seyed Iman","family":"Mirzadeh","affiliation":"Washington State University , USA"},{"name":"Hassan Ghasemzadeh","slug":"hassan-ghasemzadeh","given":"Hassan","family":"Ghasemzadeh","affiliation":"Washington State University , USA"}],"abstract":"With the recent advances in both machine learning and embedded systems research, the demand to deploy computational models for real-time execution on edge devices has increased substantially. Without deploying computational models on edge devices, the frequent transmission of sensor data to the cloud results in rapid battery draining due to the energy consumption of wireless data transmission. This rapid power dissipation leads to a considerable reduction in the battery lifetime of the system, therefore jeopardizing the real-world utility of smart devices. It is well-established that for difficult machine learning tasks, models with higher performance often require more computation power and thus are not power-efficient choices for deployment on edge devices. However, the trade-offs between performance and power consumption are not well studied. While numerous methods (e.g., model compression) have been developed to obtain an optimal model, these methods focus on improving the efficiency of a \"single\" model. In an entirely new direction, we introduce an effective method to find a combination of \"multiple\" models that are optimal in terms of power-efficiency and performance by solving an optimization problem in which both performance and power consumption are taken into account. Experimental results demonstrate that on the ImageNet dataset, we can achieve a 20% energy reduction with only 0.3% accuracy drop compared to Squeeze-and-Excitation Networks. Compared to a pruned convolutional neural network for human activity recognition, while consuming 1.7% less energy, our proposed policy achieves 1.3% higher accuracy.","doi":"10.24963/ijcai.2020/473","url":"https://ijcai.org/proceedings/2020/473","pdf":"https://ijcai.org/proceedings/2020/0473.pdf","markdown":"https://ijcai.org/proceedings/2020/473.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/473","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":473,"pages":{"first":3422,"last":3429},"keywords":["Multidisciplinary Topics and Applications: Ubiquitous Computing Systems","Multidisciplinary Topics and Applications: Real-Time Systems","Machine Learning: Deep Learning","Machine Learning: Ensemble Methods"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25336/file_25336.m3u8","length_sec":297,"provider":"hls"}],"references":[{"position":1,"raw":"[Akshay Agrawal and Boyd, 2018] Steven Diamond Ak- shay Agrawal, Robin Verschueren and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1), 2018.","title":"A rewriting system for convex optimization problems","authors":"Akshay Agrawal, Boyd ;, Steven Diamond Akshay Agrawal, Robin Verschueren, Stephen Boyd","year":"2018","venue":"Journal of Control and Decision","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alinia et al., 2020] Parastoo Alinia, Seyed-Iman Mirzadeh, and Hassan Ghasemzadeh. Actilabel: A combinato- rial transfer learning framework for activity recognition. ArXiv, abs/2003.07415, 2020.","title":"Actilabel: A combinatorial transfer learning framework for activity recognition","authors":"Parastoo Alinia, Seyed-Iman, Hassan Mirzadeh, Ghasemzadeh","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Anguita et al., 2013] Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz. A public domain dataset for human activity recognition using smartphones. In ESANN, 2013.","title":"A public domain dataset for human activity recognition using smartphones","authors":"Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, Jorge Luis, Reyes-Ortiz","year":"2013","venue":"ESANN","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Caprara et al., 2000] Alberto Caprara, Hans Kellerer, Ul- rich Pferschy, and David Pisinger. Approximation algo- rithms for knapsack problems with cardinality constraints. European Journal of Operational Research, 123, 2000.","title":"Approximation algorithms for knapsack problems with cardinality constraints","authors":"Alberto Caprara, Hans Kellerer, Ulrich Pferschy, David Pisinger","year":"2000","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Center, 2019] Erlangen Regional Computing Center. Lik- wid: Performance monitoring and benchmarking suite. https://github.com/RRZE-HPC/likwid, 2019. [Online], Last accessed on 05/2020.","title":"Likwid: Performance monitoring and benchmarking suite","authors":null,"year":"2019","venue":null,"doi":null,"url":"https://github.com/RRZE-HPC/likwid,2019","internal_url":null},{"position":6,"raw":"[Courbariaux and Bengio, 2016] Matthieu Courbariaux and Yoshua Bengio. Binarynet: Training deep neural net- works with weights and activations constrained to +1 or -1. ArXiv, abs/1602.02830, 2016.","title":"Binarynet: Training deep neural networks with weights and activations constrained to +1 or -1","authors":"Bengio Courbariaux, Matthieu Courbariaux, Yoshua Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Daquila and Shirer, 2019] Marianne Daquila and Michael Shirer. Worldwide Spending on Artificial Intelli- gence Systems Will Be Nearly 98 Billion Dollars in 2023. https://www.idc.com/getdoc.jsp?containerId= prUS45481219, 2019. [Online], Last accessed on 052020.","title":"Worldwide Spending on Artificial Intelligence Systems Will Be Nearly 98 Billion Dollars in 2023","authors":"Marianne Daquila, Michael Shirer","year":null,"venue":null,"doi":null,"url":"https://www.idc.com/getdoc.jsp?containerId=prUS45481219,2019","internal_url":null},{"position":8,"raw":"[Denton et al., 2014] Emily L. Denton, Wojciech Zaremba, Joan Bruna, and et. al. Exploiting linear structure within convolutional networks for efficient evaluation. In NIPS, 2014.","title":"Exploiting linear structure within convolutional networks for efficient evaluation","authors":"Emily L Denton, Wojciech Zaremba, Joan Bruna","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Diamond and Boyd, 2016] Steven Diamond and Stephen Boyd. CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Re- search, 17(83), 2016.","title":"CVXPY: A Python-embedded modeling language for convex optimization","authors":"Boyd ; Diamond, Steven Diamond, Stephen Boyd","year":"2016","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Domahidi et al., 2013] Alexander Domahidi, Eric Chu, and Stephen P. Boyd. ECOS: an SOCP solver for embedded systems. In European Control Conference, ECC, 2013.","title":"ECOS: an SOCP solver for embedded systems","authors":"Alexander Domahidi, Eric Chu, Stephen P Boyd","year":"2013","venue":"European Control Conference","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Franklin, 2019] Dustin Franklin. Jetson Nano Brings AI Computing to Everyone. https://devblogs.nvidia.com/ jetson-nano-ai-computing/, 2019. [Online], Last accessed on 05/2020.","title":"Jetson Nano Brings AI Computing to Everyone","authors":"Dustin Franklin","year":"2020","venue":null,"doi":null,"url":"https://devblogs.nvidia.com/jetson-nano-ai-computing/,2019","internal_url":null},{"position":12,"raw":"[Han et al., 2015] Song Han, Jeff Pool, John Tran, and William J. Dally. Learning both weights and connections for efficient neural networks. In NIPS, 2015.","title":"Learning both weights and connections for efficient neural networks","authors":"Han","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hassibi et al., 1993] Babak Hassibi, David G. Stork, and Gregory J. Wolff. Optimal brain surgeon and general net- work pruning. IEEE International Conference on Neural Networks, 1993.","title":"Optimal brain surgeon and general network pruning","authors":"Babak Hassibi, David G Stork, Gregory J Wolff","year":"1993","venue":"IEEE International Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hinton et al., 2015] Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural net- work. ArXiv, abs/1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey E Hinton, Oriol Vinyals, Jeffrey Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hu et al., 2018] Jie Hu, Li Shen, and Gang Sun. Squeeze- and-excitation networks. 2018.","title":"Squeezeand-excitation networks","authors":"Jie Hu, Li Shen, Gang Sun","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Intel, 2019] Intel. Intel Nervana Neural Network Proces- sors. https://www.intel.ai/nervana-nnp, 2019. [Online], Last accessed on 05/2020.","title":"Intel. Intel Nervana Neural Network Processors","authors":"Intel","year":"2019","venue":null,"doi":null,"url":"https://www.intel.ai/nervana-nnp,2019","internal_url":null},{"position":18,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. ArXiv, abs/1502.03167, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jacob et al., 2017] Benoit Jacob, Skirmantas Kligys, Bo Chen, and et. al. Quantization and training of neural networks for efficient integer-arithmetic-only inference. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017.","title":"Quantization and training of neural networks for efficient integer-arithmetic-only inference","authors":"Benoit Jacob, Skirmantas Kligys, Bo Chen","year":"2017","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Jayakodi et al., 2020] Nitthilan Kanappan Jayakodi, Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Design and optimization of energy-accuracy tradeoff net- works for mobile platforms via pretrained deep mod- els. ACM Transactions on Embedded Computing Systems (TECS), 19, 2020.","title":"Nitthilan Kanappan Jayakodi, Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Design and optimization of energy-accuracy tradeoff networks for mobile platforms via pretrained deep models","authors":"Jayakodi","year":"2020","venue":"ACM Transactions on Embedded Computing Systems (TECS)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Jouppi et al., 2017] Norman P. Jouppi, Cliff Young, Nishant Patil, and et. al. In-datacenter performance analysis of a tensor processing unit. 2017 ACM/IEEE 44th Annual In- ternational Symposium on Computer Architecture (ISCA), 2017.","title":"In-datacenter performance analysis of a tensor processing unit","authors":"Norman P Jouppi, Cliff Young, Nishant Patil","year":"2017","venue":"ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Keerthi et al., 2006] S. Sathiya Keerthi, Olivier Chapelle, and Dennis DeCoste. Building support vector machines with reduced classifier complexity. J. Mach. Learn. Res., 7, 2006.","title":"Building support vector machines with reduced classifier complexity","authors":"S Sathiya Keerthi, Olivier Chapelle, Dennis Decoste","year":"2006","venue":"J. Mach. Learn. Res","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kellerer et al., 2004] Hans Kellerer, Ulrich Pferschy, and David Pisinger. Knapsack problems. Springer, 2004.","title":"Knapsack problems","authors":"Kellerer","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kingma and Ba, 2014] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li et al., 2019] Wenxin Li, Joohyun Lee, and Ness B. Shroff. A faster fptas for knapsack problem with cardi- nality constraint. ArXiv, abs/1902.00919, 2019.","title":"A faster fptas for knapsack problem with cardinality constraint","authors":"Wenxin Li, Joohyun Lee, Ness B Shroff","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ma et al., 2018] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet V2: practical guidelines for efficient CNN architecture design. In Computer Vision -ECCV, 2018.","title":"Shufflenet V2: practical guidelines for efficient CNN architecture design","authors":"Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, Jian Sun","year":"2018","venue":"Computer Vision -ECCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Mirzadeh et al., 2019] Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant: Bridging the gap between student and teacher. ArXiv, abs/1902.03393, 2019.","title":"Improved knowledge distillation via teacher assistant: Bridging the gap between student and teacher","authors":"Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Hassan Ghasemzadeh","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Soumith Chintala, and et. al. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Soumith Chintala","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Pedregosa et al., 2011] F. Pedregosa, G. Varoquaux, Gram- fort, and et. al. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2011.","title":"Scikit-learn: Machine learning in python","authors":"F Pedregosa, G Varoquaux, Gramfort","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Pham et al., 2018] Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, and Jeff Dean. Efficient neural architec- ture search via parameter sharing. ArXiv, abs/1802.03268, 2018.","title":"Efficient neural architecture search via parameter sharing","authors":"Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, Jeff Dean","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Russakovsky et al., 2014] Olga Russakovsky, Jun Deng, Hao Su, Jonathan Krause, and et. al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115, 2014.","title":"Imagenet large scale visual recognition challenge","authors":"Olga Russakovsky, Jun Deng, Hao Su, Jonathan Krause","year":"2014","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sandler et al., 2018] Mark Sandler, Andrew G. Howard, Menglong Zhu, and et. al. Mobilenetv2: Inverted resid- uals and linear bottlenecks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2018.","title":"Mobilenetv2: Inverted residuals and linear bottlenecks","authors":"Mark Sandler, Andrew G Howard, Menglong Zhu","year":"2018","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Shirer and MacGillivray, 2019] Michael Shirer and Carrie MacGillivray. The Growth in Connected IoT Devices Is Expected to Generate 79.4ZB of Data in 2025, Accord- ing to a New IDC Forecast. https://www.idc.com/getdoc. jsp?containerId=prUS45213219, 2019. [Online], Last ac- cessed on 05/2020.","title":"The Growth in Connected IoT Devices Is Expected to Generate 79.4ZB of Data in 2025, According to a New IDC Forecast","authors":"Macgillivray Shirer, Michael Shirer, Carrie Macgillivray","year":"2019","venue":null,"doi":null,"url":"https://www.idc.com/getdoc.jsp?containerId=prUS45213219,2019","internal_url":null},{"position":34,"raw":"[Shirer and Palma, 2019] Michael Shirer and Carrie Michael Palma. Annual Edge AI Processor Shipments Fore- cast to Reach 1.5 Billion Units by 2023, According to IDC. https://www.idc.com/getdoc.jsp?containerId= prUS45124019, 2019. [Online], Last accessed on 05/2020.","title":"Annual Edge AI Processor Shipments Forecast to Reach 1.5 Billion Units by 2023","authors":"Michael Shirer, Carrie Michael, Palma","year":"2020","venue":null,"doi":null,"url":"https://www.idc.com/getdoc.jsp?containerId=prUS45124019,2019","internal_url":null},{"position":35,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In 3rd International Con- ference on Learning Representations, ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"3rd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Weaver et al., 2012] V. M. Weaver, M. Johnson, K. Ka- sichayanula, J. Ralph, and et. al. Measuring energy and power with papi. In 2012 41st International Conference on Parallel Processing Workshops, Sep. 2012.","title":"Measuring energy and power with papi","authors":"V M Weaver, M Johnson, K Kasichayanula, J Ralph","year":"2012","venue":"41st International Conference on Parallel Processing Workshops","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhu and Gupta, 2017] Michael Zhu and Suyog Gupta. To prune, or not to prune: Exploring the efficacy of pruning for model compression. ArXiv, abs/1710.01878, 2017.","title":"To prune, or not to prune: Exploring the efficacy of pruning for model compression","authors":"Michael Zhu, Suyog Gupta","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}