{"type":"ScholarlyArticle","title":"Split to Be Slim: An Overlooked Redundancy in Vanilla Convolution","authors":[{"name":"Qiulin Zhang","slug":"qiulin-zhang","given":"Qiulin","family":"Zhang","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Zhuqing Jiang","slug":"zhuqing-jiang","given":"Zhuqing","family":"Jiang","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Qishuo Lu","slug":"qishuo-lu","given":"Qishuo","family":"Lu","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Jia'nan Han","slug":"jia-nan-han","given":"Jia'nan","family":"Han","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Zhengxin Zeng","slug":"zhengxin-zeng","given":"Zhengxin","family":"Zeng","affiliation":"Beijing University of Posts and Telecommunications"},{"name":"Shanghua Gao","slug":"shanghua-gao","given":"Shanghua","family":"Gao","affiliation":"Nankai University"},{"name":"Aidong Men","slug":"aidong-men","given":"Aidong","family":"Men","affiliation":"Beijing University of Posts and Telecommunications"}],"abstract":"Many effective solutions have been proposed to reduce the redundancy of models for inference acceleration. Nevertheless, common approaches mostly focus on eliminating less important filters or constructing efficient operations, while ignoring the pattern redundancy in feature maps. We reveal that many feature maps within a layer share similar but not identical patterns. However, it is difficult to identify if features with similar patterns are redundant or contain essential details. Therefore, instead of directly removing uncertain redundant features, we propose a split based convolutional operation, namely SPConv, to tolerate features with similar patterns but require less computation. Specifically, we split input feature maps into the representative part and the uncertain redundant part, where intrinsic information is extracted from the representative part through relatively heavy computation while tiny hidden details in the uncertain redundant part are processed with some light-weight operation. To recalibrate and fuse these two groups of processed features, we propose a parameters-free feature fusion module. Moreover, our SPConv is formulated to replace the vanilla convolution in a plug-and-play way. Without any bells and whistles, experimental results on benchmarks demonstrate SPConv-equipped networks consistently outperform state-of-the-art baselines in both accuracy and inference time on GPU, with FLOPs and parameters dropped sharply.","doi":"10.24963/ijcai.2020/442","url":"https://ijcai.org/proceedings/2020/442","pdf":"https://ijcai.org/proceedings/2020/0442.pdf","markdown":"https://ijcai.org/proceedings/2020/442.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/442","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":442,"pages":{"first":3195,"last":3201},"keywords":["Machine Learning: Feature Selection; Learning Sparse Models","Machine Learning: Deep Learning: Convolutional networks"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24728/file_24728.m3u8","length_sec":287,"provider":"hls"}],"references":[{"position":1,"raw":"[Chen et al., 2019a] Kai Chen, Jiaqi Wang, et al. Mmde- tection: Open mmlab detection toolbox and benchmark. CoRR, abs/1906.07155, 2019.","title":"Mmdetection: Open mmlab detection toolbox and benchmark","authors":"Chen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2019b] Yunpeng Chen, Haoqi Fan, et al. Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution. In ICCV, 2019.","title":"Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution","authors":"Chen","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chollet, 2017] Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions. In CVPR, 2017.","title":"Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions","authors":"Chollet","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Fei-Fei Li","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT, 2019.","title":"BERT: pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gao et al., 2019] Xitong Gao, Yiren Zhao, Lukasz Dudziak, Robert D. Mullins, and Cheng-Zhong Xu. Dynamic chan- nel pruning: Feature boosting and suppression. In ICLR, 2019.","title":"Dynamic channel pruning: Feature boosting and suppression","authors":"Gao","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao et al., 2020] Shang-Hua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, and Philip Torr. Res2net: A new multi-scale backbone architecture. IEEE TPAMI, 2020.","title":"Res2net: A new multi-scale backbone architecture","authors":"Shang-Hua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, Philip Torr","year":"2020","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Han et al., 2020] Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More fea- tures from cheap operations. In CVPR, 2020.","title":"Ghostnet: More features from cheap operations","authors":"Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, Chang Xu","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. Mask R-CNN. In ICCV, 2017.","title":"Mask R-CNN","authors":"He","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Howard et al., 2017] Andrew G Howard, Menglong Zhu, et al. Mobilenets: Efficient convolutional neural net- works for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.","title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications","authors":"Menglong Andrew G Howard, Zhu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.","title":"Densely connected convolutional networks","authors":"Gao Huang, Zhuang Liu, Laurens Van Der Maaten, Kilian Q Weinberger","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In ICML, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny im- ages. Technical report, Citeseer, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[LeCun et al., 1990] Yann LeCun, John S Denker, and Sara A Solla. Optimal brain damage. In NeurIPS, 1990.","title":"Optimal brain damage","authors":"Yann Lecun, John S Denker, Sara A Solla","year":"1990","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2019] Xiang Li, Wenhai Wang, Xiaolin Hu, and Jian Yang. Selective kernel networks. In (CVPR), 2019.","title":"Selective kernel networks","authors":"Xiang Li, Wenhai Wang, Xiaolin Hu, Jian Yang","year":"2019","venue":"CVPR)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lin et al., 2014a] Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. In ICLR, 2014.","title":"Network in network","authors":"Lin","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2014b] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Lin","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin et al., 2017a] Tsung-Yi Lin, Piotr Dollár, Ross Gir- shick, Kaiming He, Bharath Hariharan, and Serge Be- longie. Feature pyramid networks for object detection. In CVPR, 2017.","title":"Feature pyramid networks for object detection","authors":"Lin","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lin et al., 2017b] Tsung-Yi Lin, Priya Goyal, Ross Gir- shick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, 2017.","title":"Focal loss for dense object detection","authors":"Lin","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Micikevicius et al., 2018] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory F. Diamos, Erich Elsen, David García, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. Mixed precision training. In ICLR, 2018.","title":"Mixed precision training","authors":"Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory F Diamos, Erich Elsen, David García, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, Hao Wu","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Singh et al., 2019] Pravendra Singh, Vinay Kumar Verma, Piyush Rai, and Vinay P Namboodiri. Hetconv: Heteroge- neous kernel-based convolutions for deep cnns. In CVPR, 2019.","title":"Hetconv: Heterogeneous kernel-based convolutions for deep cnns","authors":"Pravendra Singh, Vinay Kumar Verma, Piyush Rai, P Vinay, Namboodiri","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabi- novich. Going deeper with convolutions. In CVPR, 2015.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vanhoucke, 2014] Vincent Vanhoucke. Learning visual rep- resentations at scale. ICLR invited talk, 1, 2014.","title":"Learning visual representations at scale","authors":"Vincent Vanhoucke","year":"2014","venue":"ICLR invited talk","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xie et al., 2017] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In CVPR, 2017.","title":"Aggregated residual transformations for deep neural networks","authors":"Xie","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2017] Ting Zhang, Guo-Jun Qi, Bin Xiao, and Jingdong Wang. Interleaved group convolutions. In ICCV, 2017.","title":"Interleaved group convolutions","authors":"Ting Zhang, Guo-Jun Qi, Bin Xiao, Jingdong Wang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2018] Xiangyu Zhang, Xinyu Zhou, Mengx- iao Lin, and Jian Sun. Shufflenet: An extremely effi- cient convolutional neural network for mobile devices. In CVPR, 2018.","title":"Shufflenet: An extremely efficient convolutional neural network for mobile devices","authors":"Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, Jian Sun","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}