{"type":"ScholarlyArticle","title":"Group Reconstruction and Max-Pooling Residual Capsule Network","authors":[{"name":"Xinpeng Ding","slug":"xinpeng-ding","given":"Xinpeng","family":"Ding","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Electronic Engineering , Xidian University , Xi'an , China"},{"name":"Nannan Wang","slug":"nannan-wang","given":"Nannan","family":"Wang","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Telecommunications Engineering , Xidian University , Xi'an , China"},{"name":"Xinbo Gao","slug":"xinbo-gao","given":"Xinbo","family":"Gao","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Electronic Engineering , Xidian University , Xi'an , China"},{"name":"Jie Li","slug":"jie-li","given":"Jie","family":"Li","affiliation":"State Key Laboratory of Integrated Services Networks , Xidian University , Xi'an , China; School of Electronic Engineering , Xidian University , Xi'an , China"},{"name":"Xiaoyu Wang","slug":"xiaoyu-wang","given":"Xiaoyu","family":"Wang","affiliation":"IntelliFusion , Shenzhen , China"}],"abstract":"In capsule networks, the mapping of low-level capsules to high-level capsules is achieved by a routing-by-agreement algorithm. Since the capsule is made up of collections of neurons and the routing mechanism involves all the capsules instead of simply discarding some of the neurons like Max-Pooling, the capsule network has stronger representation ability than the traditional neural network. However, considering too much low-level capsules' information will cause its corresponding upper layer capsules to be interfered by other irrelevant information or noise capsules. Therefore, the original capsule network does not perform well on complex data structure. What's worse, computational complexity becomes a bottleneck in dealing with large data networks. In order to solve these shortcomings, this paper proposes a group reconstruction and max-pooling residual capsule network (GRMR-CapsNet). We build a block in which all capsules are divided into different groups and perform group reconstruction routing algorithm to obtain the corresponding high-level capsules. Between the lower and higher layers, Capsule Max-Pooling is adopted to prevent overfitting. We conduct experiments on CIFAR-10/100 and SVHN datasets and the results show that our method can perform better against state-of-the-arts.","doi":"10.24963/ijcai.2019/310","url":"https://ijcai.org/proceedings/2019/310","pdf":"https://ijcai.org/proceedings/2019/0310.pdf","markdown":"https://ijcai.org/proceedings/2019/310.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/310","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":310,"pages":{"first":2237,"last":2243},"keywords":["Computer Vision: Computer Vision","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Ben-Israel and Greville, 2003] Adi Ben-Israel and Thomas NE Greville. Generalized inverses: theory and applications, volume 15. Springer Science & Business Media, 2003.","title":"Generalized inverses: theory and applications","authors":"Adi Ben, - Israel, Thomas Ne Greville","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang and Chen, 2015] Jia-Ren Chang and Yong-Sheng Chen. Batch-normalized maxout network in network. arXiv preprint arXiv:1511.02583, 2015.","title":"Batch-normalized maxout network in network","authors":"Jia-Ren Chang, Yong-Sheng Chen","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Crandall, 2018] Zhenhua Chen and David Cran- dall. Generalized capsule networks with trainable routing procedure. arXiv preprint arXiv:1808.08692, 2018.","title":"Generalized capsule networks with trainable routing procedure","authors":"Zhenhua Chen, David Crandall","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cohen and Welling, 2016] Taco Cohen and Max Welling. Group equivariant convolutional networks. In Interna- tional conference on machine learning, pages 2990-2999, 2016.","title":"Group equivariant convolutional networks","authors":"Welling Cohen, Taco Cohen, Max Welling","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cohen et al., 2018] Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. arXiv preprint arXiv:1801.10130, 2018.","title":"Spherical cnns","authors":"Mario Taco S Cohen, Jonas Geiger, Max Köhler, Welling","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2013] Ian J Goodfellow, David Warde- Farley, Mehdi Mirza, Aaron Courville, and Yoshua Ben- gio. Maxout networks. arXiv preprint arXiv:1302.4389, 2013.","title":"Maxout networks","authors":"Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, Yoshua Bengio","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hinton et al., 2011] Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang. Transforming auto-encoders. In Inter- national Conference on Artificial Neural Networks, pages 44-51. Springer, 2011.","title":"Transforming auto-encoders","authors":"Geoffrey E Hinton, Alex Krizhevsky, Sida D Wang","year":"2011","venue":"International Conference on Artificial Neural Networks","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hinton et al., 2018] Geoffrey E Hinton, Sara Sabour, and Nicholas Frosst. Matrix capsules with em routing. 2018.","title":"Matrix capsules with em routing","authors":"Geoffrey E Hinton, Sara Sabour, Nicholas Frosst","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krizhevsky and Hinton, 2009] Alex Krizhevsky and Geof- frey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Ad- vances in Neural Information Processing Systems 25, pages 1097-1105.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Curran Associates, Inc., 2012.","title":null,"authors":"Curran Associates, Inc","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[LeCun, 1998] Yann LeCun. The mnist database of hand- written digits. http://yann. lecun. com/exdb/mnist/, 1998.","title":"The mnist database of handwritten digits","authors":"Yann Lecun","year":null,"venue":null,"doi":null,"url":"http://yann.lecun.com/exdb/mnist/,1998","internal_url":null},{"position":17,"raw":"[Lenssen et al., 2018] Jan Eric Lenssen, Matthias Fey, and Pascal Libuschewski. Group equivariant capsule networks. CoRR, abs/1806.05086, 2018.","title":"Group equivariant capsule networks","authors":"Lenssen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2018] Hongyang Li, Xiaoyang Guo, Bo Dai, Wanli Ouyang, and Xiaogang Wang. Neural network encapsula- tion. In European Conference on Computer Vision, pages 266-282. Springer, 2018.","title":"Neural network encapsulation","authors":"Hongyang Li, Xiaoyang Guo, Bo Dai, Wanli Ouyang, Xiaogang Wang","year":"2018","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mishkin and Matas, 2015] Dmytro Mishkin and Jiri Matas. All you need is a good init. CoRR, abs/1511.06422, 2015.","title":"All you need is a good init","authors":"Matas Mishkin, Dmytro Mishkin, Jiri Matas","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nair and Hinton, 2010] Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann ma- chines. In Proceedings of the 27th international con- ference on machine learning (ICML-10), pages 807-814, 2010.","title":"Rectified linear units improve restricted boltzmann machines","authors":"Vinod Nair, Geoffrey E Hinton","year":"2010","venue":"Proceedings of the 27th international conference on machine learning (ICML-10)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learn- ing. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, page 5, 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":"NIPS workshop on deep learning and unsupervised feature learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sabour et al., 2017] Sara Sabour, Nicholas Frosst, and Ge- offrey E Hinton. Dynamic routing between capsules. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fer- gus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 3856- 3866.","title":"Dynamic routing between capsules","authors":"Sara Sabour, Nicholas Frosst, Geoffrey E Hinton","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xi et al., 2017] Edgar Xi, Selina Bing, and Yang Jin. Cap- sule network performance on complex data. arXiv preprint arXiv:1712.03480, 2017.","title":"Capsule network performance on complex data","authors":"Edgar Xi, Selina Bing, Yang Jin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2018a] Liheng Zhang, Marzieh Edraki, and Guo-Jun Qi. Cappronet: Deep feature learning via orthog- onal projections onto capsule subspaces. arXiv preprint arXiv:1805.07621, 2018.","title":"Cappronet: Deep feature learning via orthogonal projections onto capsule subspaces","authors":"Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2018b] Xinsong Zhang, Pengshuai Li, Wei- jia Jia, and Hai Zhao. Multi-labeled relation extrac- tion with attentive capsule network. arXiv preprint arXiv:1811.04354, 2018.","title":"Multi-labeled relation extraction with attentive capsule network","authors":"Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}