{"type":"ScholarlyArticle","title":"Ensemble Soft-Margin Softmax Loss for Image Classification","authors":[{"name":"Xiaobo Wang","slug":"xiaobo-wang","given":"Xiaobo","family":"Wang","affiliation":"CBSR&NLPR , Institute of Automation , Chinese Academy of Sciences , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Shifeng Zhang","slug":"shifeng-zhang","given":"Shifeng","family":"Zhang","affiliation":"CBSR&NLPR , Institute of Automation , Chinese Academy of Sciences , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Zhen Lei","slug":"zhen-lei","given":"Zhen","family":"Lei","affiliation":"CBSR&NLPR , Institute of Automation , Chinese Academy of Sciences , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"},{"name":"Si Liu","slug":"si-liu","given":"Si","family":"Liu","affiliation":"School of Computer Science and Engineering , Beihang University , Beijing , China"},{"name":"Xiaojie Guo","slug":"xiaojie-guo","given":"Xiaojie","family":"Guo","affiliation":"School of Computer Software , Tianjin University , Tianjin , China"},{"name":"Stan Z. Li","slug":"stan-z-li","given":"Stan Z.","family":"Li","affiliation":"Faculty of Information Technology , Macau University of Science and Technology , Macau , China; CBSR&NLPR , Institute of Automation , Chinese Academy of Sciences , Beijing , China; University of Chinese Academy of Sciences , Beijing , China"}],"abstract":"Softmax loss is arguably one of the most popular losses to train CNN models for image classification. However, recent works have exposed its limitation on feature discriminability. This paper casts a new viewpoint on the weakness of softmax loss. On the one hand, the CNN features learned using the softmax loss are often inadequately discriminative. We hence introduce a soft-margin softmax function to explicitly encourage the discrmination between different classes. On the other hand, the learned classifier of softmax loss is weak. We propose to assemble multiple these weak classifiers to a strong one, inspired by the recognition that the diversity among weak classifiers is critical to a good ensemble. To achieve the diversity, we adopt the Hilbert-Schmidt Independence Criterion (HSIC). Considering these two aspects in one framework, we design a novel loss, named as Ensemble Soft-Margin Softmax (EM-Softmax). Extensive experiments on benchmark datasets are conducted to show the superiority of our design over the baseline softmax loss and several state-of-the-art alternatives.","doi":"10.24963/ijcai.2018/138","url":"https://ijcai.org/proceedings/2018/138","pdf":"https://ijcai.org/proceedings/2018/0138.pdf","markdown":"https://ijcai.org/proceedings/2018/138.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/138","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":138,"pages":{"first":992,"last":998},"keywords":["Machine Learning: Deep Learning","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Bell and Bala., 2015] S. Bell and K. Bala. Learning vi- sual similarity for product design with convolutional neu- ral networks. In TOG, 2015.","title":"Learning visual similarity for product design with convolutional neural networks","authors":"S Bell, K Bala","year":"2015","venue":"TOG","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chrabaszcz et al., 2017] P. Chrabaszcz, L. Ilya, and H. Frank. A downsampled variant of imagenet as an alternative to the cifar datasets. arXiv preprint arXiv:1707.08819, 2017.","title":"A downsampled variant of imagenet as an alternative to the cifar datasets","authors":"P Chrabaszcz, L Ilya, H Frank","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fredricks and Nelsen., 2007] G. Fredricks and R. Nelsen. On the relationship between spearman's rho and kendall's tau for pairs of continuous random variables. Journal of statistical planning and inference., 2007.","title":"On the relationship between spearman's rho and kendall's tau for pairs of continuous random variables","authors":"G Fredricks, R Nelsen","year":"2007","venue":"Journal of statistical planning and inference","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2013] I.","title":null,"authors":"Goodfellow","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Goodfellow, D. Warde-Farley, and M. Mirza. Maxout networks. In ICML, 2013.","title":"Maxout networks","authors":"D Goodfellow, M Warde-Farley, Mirza","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gretton et al., 2005] A. Gretton, O. Bousquet, and A. Smola. Measuring statistical dependence with hilbert-schmidt norms. In ICAL, 2005.","title":"Measuring statistical dependence with hilbert-schmidt norms","authors":"A Gretton, O Bousquet, A Smola","year":"2005","venue":"ICAL","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guo et al., 2017] X. Guo, X. Wang, and H. Ling. Exclusiv- ity regularized machine. In IJCAI, 2017.","title":"Exclusivity regularized machine","authors":"Guo","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2016] K. He, X. Zhang, and S. Ren. Deep residual learning for image recognition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"K He, X Zhang, S Ren","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hinton et al., 2012] G. Hinton, N. Srivastava, and A. Krizhevsky. Improving neural networks by pre- venting co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580., 2012.","title":"Improving neural networks by preventing co-adaptation of feature detectors","authors":"G Hinton, N Srivastava, A Krizhevsky","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia et al., 2014] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Y Jia, E Shelhamer, J Donahue, S Karayev, J Long, R Girshick, S Guadarrama, T Darrell","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krizhevsky and Hinton, 2009] A. Krizhevsky and G. Hin- ton. Learning multiple layers of features from tiny images. Technical report, 2009.","title":"Learning multiple layers of features from tiny images","authors":"A Krizhevsky, G Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krizhevsky et al., 2012] A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolu- tional neural networks. In NIPS, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"A Krizhevsky, I Sutskever, G Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[LeCun et al., 1998a] Y. LeCun, L. Bottou, and Y. Bengio. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Lecun","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[LeCun et al., 1998b] Y. LeCun, C. Cortes, and C. Burges. The mnist database of handwritten digits. 1998.","title":"The mnist database of handwritten digits","authors":"Lecun","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lee et al., 2015] Y. Lee, S. Xie, and P. Gallagher. Deeply- supervised nets. In AISTATS, 2015.","title":"Deeplysupervised nets","authors":"Y Lee, S Xie, P Gallagher","year":"2015","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2012] N. Li, Y. Yu, and Z. Zhou. Diversity regu- larized ensemble pruning. In ECML, 2012.","title":"Diversity regularized ensemble pruning","authors":"N Li, Y Yu, Z Zhou","year":"2012","venue":"ECML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2016] Y. Li, K. He, and J. Sun. R-fcn: Object de- tection via region-based fully convolutional networks. In NIPS, 2016.","title":"R-fcn: Object detection via region-based fully convolutional networks","authors":"Y Li, K He, J Sun","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liang et al., 2017] X. Liang, X. Wang, Z. Lei, S. Liao, and Stan. Li. Soft-margin softmax for deep classification. In ICONIP, 2017.","title":"Soft-margin softmax for deep classification","authors":"Liang","year":"2017","venue":"ICONIP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2013] M. Lin, Q. Chen, and Y. Yan. Network in network. In ICLR, 2013.","title":"Network in network","authors":"M Lin, Q Chen, Y Yan","year":"2013","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2016] W. Liu, Y. Wen, and Z. Yu. Large-margin softmax loss for convolutional neural networks. In ICML, 2016.","title":"Large-margin softmax loss for convolutional neural networks","authors":"W Liu, Y Wen, Z Yu","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2017a] W. Liu, Y. Wen, Z. Yu, M. Li, and L. Song. Sphereface: Deep hypersphere embedding for face recognition. In CVPR, 2017.","title":"Sphereface: Deep hypersphere embedding for face recognition","authors":"Liu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2017b] Y. Liu, H. Li, and X. Wang. Learn- ing deep features via congenerous cosine loss for person recognition. arXiv preprint arXiv:1702.06890., 2017.","title":"Learning deep features via congenerous cosine loss for person recognition","authors":"Liu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Martins and Astudillo., 2016] A. Martins and R. Astudillo. From softmax to sparsemax: A sparse model of attention and multi-label classification. In ICML, 2016.","title":"From softmax to sparsemax: A sparse model of attention and multi-label classification","authors":"A Martins, R Astudillo","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Schroff et al., 2015] F. Schroff, D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recog- nition and clustering. In CVPR, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"F Schroff, D Kalenichenko, J Philbin","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Simonyan and Andrew, 2014] K. Simonyan and Z. Andrew. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"K Simonyan, Z Andrew","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun et al., 2014] Y. Sun, Y. Chen, and X. Wang. Deep learn- ing face representation by joint identification-verification. In NIPS, 2014.","title":"Deep learning face representation by joint identification-verification","authors":"Y Sun, Y Chen, X Wang","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tang, 2013] Y. Tang. Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239, 2013.","title":"Deep learning using linear support vector machines","authors":"Y Tang","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2015] X. Wang, X. Guo, and S. Li. Adap- tively unified semi-supervised dictionary learning with ac- tive points. In CVPR, 2015.","title":"Adaptively unified semi-supervised dictionary learning with active points","authors":"Wang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2017a] F. Wang, X. Xiang, J. Chen, and A. Yuille. Normface: l 2 hypersphere embedding for face verification.. In ACM MM, 2017.","title":"Normface: l 2 hypersphere embedding for face verification","authors":"Wang","year":"2017","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2017b] X. Wang, X. Guo, Z. Lei, C. Zhang, and S. Li. Exclusivity-consistency regularized multi-view sub- space clustering. In CVPR, 2017.","title":"Exclusivity-consistency regularized multi-view subspace clustering","authors":"Wang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2018] X. Wang, Z. Lei, S. Liao, X. Guo, Y. Yang, and S. Li. Dependence-aware feature coding for person re-identification. Signal Processing Letters., 2018.","title":"Dependence-aware feature coding for person re-identification","authors":"Wang","year":"2018","venue":"Signal Processing Letters","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wen et al., 2016] Y. Wen, K. Zhang, and Z. Li. A discrimi- native feature learning approach for deep face recognition. In ECCV, 2016.","title":"A discriminative feature learning approach for deep face recognition","authors":"Y Wen, K Zhang, Z Li","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2016] X. Zhang, Z. Fang, and Y. Wen. Range loss for deep face recognition with long-tail. arXiv preprint arXiv:1611.08976., 2016.","title":"Range loss for deep face recognition with long-tail","authors":"Zhang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhou et al., 2016] B. Zhou, A. Khosla, and A. Lapedriza. Learning deep features for discriminative localization. In CVPR, 2016.","title":"Learning deep features for discriminative localization","authors":"B Zhou, A Khosla, A Lapedriza","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}