{"type":"ScholarlyArticle","title":"CDC: Classification Driven Compression for Bandwidth Efficient Edge-Cloud Collaborative Deep Learning","authors":[{"name":"Yuanrui Dong","slug":"yuanrui-dong","given":"Yuanrui","family":"Dong","affiliation":"National Engineering Laboratory for Big Data Analytics , Xi'an Jiaotong University , China"},{"name":"Peng Zhao","slug":"peng-zhao","given":"Peng","family":"Zhao","affiliation":"National Engineering Laboratory for Big Data Analytics , Xi'an Jiaotong University , China"},{"name":"Hanqiao Yu","slug":"hanqiao-yu","given":"Hanqiao","family":"Yu","affiliation":"National Engineering Laboratory for Big Data Analytics , Xi'an Jiaotong University , China"},{"name":"Cong Zhao","slug":"cong-zhao","given":"Cong","family":"Zhao","affiliation":"Department of Computing , Imperial College London , UK"},{"name":"Shusen Yang","slug":"shusen-yang","given":"Shusen","family":"Yang","affiliation":"National Engineering Laboratory for Big Data Analytics , Xi'an Jiaotong University , China"}],"abstract":"The emerging edge-cloud collaborative Deep Learning (DL) paradigm aims at improving the performance of practical DL implementations in terms of cloud bandwidth consumption, response latency, and data privacy preservation. Focusing on bandwidth efficient edge-cloud collaborative training of DNN-based classifiers, we present CDC, a Classification Driven Compression framework that reduces bandwidth consumption while preserving classification accuracy of edge-cloud collaborative DL. Specifically, to reduce bandwidth consumption, for resource-limited edge servers, we develop a lightweight autoencoder with a classification guidance for compression with classification driven feature preservation, which allows edges to only upload the latent code of raw data for accurate global training on the Cloud. Additionally, we design an adjustable quantization scheme adaptively pursuing the tradeoff between bandwidth consumption and classification accuracy under different network conditions, where only fine-tuning is required for rapid compression ratio adjustment. Results of extensive experiments demonstrate that, compared with DNN training with raw data, CDC consumes 14.9× less bandwidth with an accuracy loss no more than 1.06%, and compared with DNN training with data compressed by AE without guidance, CDC introduces at least 100% lower accuracy loss.","doi":"10.24963/ijcai.2020/467","url":"https://ijcai.org/proceedings/2020/467","pdf":"https://ijcai.org/proceedings/2020/0467.pdf","markdown":"https://ijcai.org/proceedings/2020/467.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/467","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning Applications","track":"Main track","number":467,"pages":{"first":3378,"last":3384},"keywords":["Agent-based and Multi-agent Systems: Coordination and Cooperation","Machine Learning Applications: Applications of Supervised Learning","Machine Learning: Federated Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25589/file_25589.m3u8","length_sec":255,"provider":"hls"}],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016.","title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems","authors":"Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen and Ran, 2019] Jiasi Chen and Xukan Ran. Deep learning with edge computing: A review. Proc. IEEE, 107(8):1655-1674, 2019.","title":"Deep learning with edge computing: A review","authors":"Chen, Ran ;, Jiasi Chen, Xukan Ran","year":"2019","venue":"Proc. IEEE","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cipolla et al., 2018] Roberto Cipolla, Yarin Gal, and Alex Kendall. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proc. of IEEE CVPR, pages 7482-7491, 2018.","title":"Multi-task learning using uncertainty to weigh losses for scene geometry and semantics","authors":"Roberto Cipolla, Yarin Gal, Alex Kendall","year":"2018","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cisco, 2016] Cisco. Cisco global cloud index: Forecast and methodology, 2016-2021.","title":"Cisco. Cisco global cloud index: Forecast and methodology","authors":null,"year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Cisco White paper, 2016.","title":"Cisco White paper","authors":null,"year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proc. IEEE CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"Proc. IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hardy et al., 2017] Corentin Hardy, Erwan Le Merrer, and Bruno Sericola. Distributed deep learning on edge- devices: feasibility via adaptive compression. In Proc. IEEE NCA, pages 1-8. IEEE, 2017.","title":"Distributed deep learning on edgedevices: feasibility via adaptive compression","authors":"Corentin Hardy, Erwan Le Merrer, Bruno Sericola","year":"2017","venue":"Proc. IEEE NCA","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jackson et al., 2018] Zohar Jackson, Csar Souza, Jason Flaks, and H Nicolas. Jakobovski/free-spoken-digit- dataset v1. 0.7, 2018.","title":"Zohar Jackson","authors":"Jackson","year":"2018","venue":"Jakobovski/free-spoken-digitdataset","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kairouz et al., 2019] Peter Kairouz, H Brendan McMa- han, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Ar- jun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Gra- ham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977, 2019.","title":"Advances and open problems in federated learning","authors":"Peter Kairouz, Brendan Mcmahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Nitin Arjun, Keith Bhagoji, Zachary Bonawitz, Graham Charles, Rachel Cormode, Cummings","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Le et al., 2018] Lei Le, Andrew Patterson, and Martha White. Supervised autoencoders: Improving generaliza- tion performance with unsupervised regularizers. In Proc. NeurIPS, pages 107-117, 2018.","title":"Supervised autoencoders: Improving generalization performance with unsupervised regularizers","authors":"Lei Le, Andrew Patterson, Martha White","year":"2018","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2014] Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, et al. Scaling distributed machine learning with the parameter server. In Proc. OSDI, pages 583-598, 2014.","title":"Scaling distributed machine learning with the parameter server","authors":"Mu Li, Jun Woo David G Andersen, Alexander J Park, Smola","year":"2014","venue":"Proc. OSDI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018a] En Li, Zhi Zhou, and Xu Chen. Edge in- telligence: On-demand deep learning model co-inference with device-edge synergy. In Proc. Workshop on Mobile Edge Communications, pages 31-36. ACM, 2018.","title":"Edge intelligence: On-demand deep learning model co-inference with device-edge synergy","authors":"Li","year":"2018","venue":"Proc. Workshop on Mobile Edge Communications","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018b] He Li, Kaoru Ota, and Mianxiong Dong. Learning iot in edge: deep learning for the internet of things with edge computing. IEEE Network, 32(1):96- 101, 2018.","title":"Learning iot in edge: deep learning for the internet of things with edge computing","authors":"Li","year":"2018","venue":"IEEE Network","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2018c] Mu Li, Wangmeng Zuo, Shuhang Gu, De- bin Zhao, and David Zhang. Learning convolutional net- works for content-weighted image compression. In Proc. of IEEE CVPR, pages 3214-3223, 2018.","title":"Learning convolutional networks for content-weighted image compression","authors":"Li","year":"2018","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2016] Tongliang Liu, Dacheng Tao, Mingli Song, and Stephen J Maybank. Algorithm-dependent gener- alization bounds for multi-task learning. IEEE PAMI, 39(2):227-241, 2016.","title":"Algorithm-dependent generalization bounds for multi-task learning","authors":"Liu","year":"2016","venue":"IEEE PAMI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mell et al., 2011] Peter Mell, Tim Grance, et al. The nist definition of cloud computing. 2011.","title":"The nist definition of cloud computing","authors":"Mell","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Salamon et al., 2014] Justin Salamon, Christopher Jacoby, and Juan Pablo Bello. A dataset and taxonomy for urban sound research. In Proc. ACM ICME, pages 1041-1044, 2014.","title":"A dataset and taxonomy for urban sound research","authors":"Justin Salamon, Christopher Jacoby, Juan Pablo Bello","year":"2014","venue":"Proc. ACM ICME","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Satyanarayan, 2017] Mahadev Satyanarayan. The emer- gence of edge computing. Computer, 50(1):30-39, 2017.","title":"The emergence of edge computing","authors":"Mahadev Satyanarayan","year":"2017","venue":"Computer","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shaham and Michaeli, 2018] Tamar Rott Shaham and Tomer Michaeli. Deformation aware image compression. In Proc. of IEEE CVPR, 2018.","title":"Deformation aware image compression","authors":"Michaeli Shaham, Tamar Rott Shaham, Tomer Michaeli","year":"2018","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi and Dustdar, 2016] Weisong Shi and Schahram Dust- dar. The promise of edge computing. Computer, 49(5):78- 81, 2016.","title":"Weisong Shi and Schahram Dustdar. The promise of edge computing","authors":"Dustdar Shi","year":"2016","venue":"Computer","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Smith et al., 2017] Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi- task learning. In Proc. of NeurIPS, pages 4424-4434, 2017.","title":"Federated multitask learning","authors":"Smith","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tang et al., 2018] Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, and Ji Liu. Communication compression for decentralized training. In Proc. NeurIPS, pages 7652- 7662, 2018.","title":"Communication compression for decentralized training","authors":"Hanlin Tang, Shaoduo Gan, Ce Zhang, Tong Zhang, Ji Liu","year":"2018","venue":"Proc. NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Teerapittayanon et al., 2017] Surat Teerapittayanon, Bradley McDanel, and HT Kung. Distributed deep neural networks over the cloud, the edge and end devices. In Proc. IEEE ICDCS, pages 328-339, 2017.","title":"Distributed deep neural networks over the cloud, the edge and end devices","authors":"Teerapittayanon","year":"2017","venue":"Proc. IEEE ICDCS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Theis et al., 2017] Lucas Theis, Wenzhe Shi, Andrew Cun- ningham, and Ferenc Huszár. Lossy image compression with compressive autoencoders. ICLR, 2017.","title":"Lossy image compression with compressive autoencoders","authors":"Lucas Theis, Wenzhe Shi, Andrew Cunningham, Ferenc Huszár","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie et al., 2017] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In Proc. of IEEE CVPR, pages 1492-1500, 2017.","title":"Aggregated residual transformations for deep neural networks","authors":"Xie","year":"2017","venue":"Proc. of IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yu et al., 2015] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, et al. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.","title":"Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop","authors":"Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhou et al., 2018] Lei Zhou, Chunlei Cai, Yue Gao, San- bao Su, and Junmin Wu. Variational autoencoder for low bit-rate image compression. In Proc. IEEE CVPR, pages 2617-2620, 2018.","title":"Variational autoencoder for low bit-rate image compression","authors":"Lei Zhou, Chunlei Cai, Yue Gao, Sanbao Su, Junmin Wu","year":"2018","venue":"Proc. IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2019] Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, and Junshan Zhang. Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proc. IEEE, 107(8):1738-1762, 2019.","title":"Edge intelligence: Paving the last mile of artificial intelligence with edge computing","authors":"Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, Junshan Zhang","year":"2019","venue":"Proc. IEEE","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}