{"type":"ScholarlyArticle","title":"Deep Ordinal Regression Based on Data Relationship for Small Datasets","authors":[{"name":"Yanzhu Liu","slug":"yanzhu-liu","given":"Yanzhu","family":"Liu","affiliation":"Rolls-Royce Advanced Technology Centre , Singapore"},{"name":"Adams Wai Kin Kong","slug":"adams-wai-kin-kong","given":"Adams Wai Kin","family":"Kong","affiliation":"Rolls-Royce Advanced Technology Centre , Singapore"},{"name":"Chi Keong Goh","slug":"chi-keong-goh","given":"Chi Keong","family":"Goh","affiliation":"Nanyang Technological University , Singapore"}],"abstract":"Ordinal regression aims to classify instances into ordinal categories. As with other supervised learning problems, learning an effective deep ordinal model from a small dataset is challenging. This paper proposes a new approach which transforms the ordinal regression problem to binary classification problems and uses triplets with instances from different categories to train deep neural networks such that high-level features describing their ordinal relationship can be extracted automatically. In the testing phase, triplets are formed by a testing instance and other instances with known ranks. A decoder is designed to estimate the rank of the testing instance based on the outputs of the network. Because of the data argumentation by permutation, deep learning can work for ordinal regression even on small datasets. Experimental results on the historical color image benchmark and MSRA image search datasets demonstrate that the proposed algorithm outperforms the traditional deep learning approach and is comparable with other state-of-the-art methods, which are highly based on prior knowledge to design effective features.","doi":"10.24963/ijcai.2017/330","url":"https://ijcai.org/proceedings/2017/330","pdf":"https://ijcai.org/proceedings/2017/0330.pdf","markdown":"https://ijcai.org/proceedings/2017/330.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/330","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":330,"pages":{"first":2372,"last":2378},"keywords":["Machine Learning: Machine Learning","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Cheng et al., 2008] Jianlin Cheng, Zheng Wang, and Gian- luca Pollastri. A neural network approach to ordinal re- gression. In Neural Networks, 2008. IJCNN 2008.(IEEE World Congress on Computational Intelligence). IEEE In- ternational Joint Conference on, pages 1279-1284. IEEE, 2008.","title":"A neural network approach to ordinal regression","authors":"Jianlin Cheng, Zheng Wang, Gianluca Pollastri","year":"2008","venue":"IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chu and Ghahramani, 2005] Wei Chu and Zoubin Ghahra- mani. Gaussian processes for ordinal regression. In Journal of Machine Learning Research, pages 1019-1041, 2005.","title":"Gaussian processes for ordinal regression","authors":"Chu, Ghahramani, Wei Chu, Zoubin Ghahramani","year":"2005","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chu and Keerthi, 2007] Wei Chu and S Sathiya Keerthi. Support vector ordinal regression. Neural computation, 19(3):792-815, 2007.","title":"Support vector ordinal regression","authors":"Wei Chu, S Sathiya, Keerthi","year":"2007","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Frank and Hall, 2001] Eibe Frank and Mark Hall. A simple approach to ordinal classification. Springer, 2001.","title":"A simple approach to ordinal classification","authors":"Hall ; Frank, Eibe Frank, Mark Hall","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gutierrez et al., 2016] Pedro Antonio Gutierrez, Maria Perez-Ortiz, Javier Sanchez-Monedero, Francisco Fernandez-Navarro, and Cesar Hervas-Martinez. Ordinal regression methods: survey and experimental study. Knowledge and Data Engineering, IEEE Transactions on, 28(1):127-146, 2016.","title":"Ordinal regression methods: survey and experimental study","authors":"Pedro Antonio Gutierrez, Maria Perez-Ortiz, Javier Sanchez-Monedero, Francisco Fernandez-Navarro, Cesar Hervas-Martinez","year":"2016","venue":"Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Herbrich et al., 1999] Ralf Herbrich, Thore Graepel, and Klaus Obermayer. Large margin rank boundaries for ordi- nal regression. Advances in neural information processing systems, pages 115-132, 1999.","title":"Large margin rank boundaries for ordinal regression","authors":"Ralf Herbrich, Thore Graepel, Klaus Obermayer","year":"1999","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural in- formation processing systems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lin and Li, 2012] Hsuan-Tien Lin and Ling Li. Reduc- tion from cost-sensitive ordinal ranking to weighted bi- nary classification. Neural Computation, 24(5):1329- 1367, 2012.","title":"Reduction from cost-sensitive ordinal ranking to weighted binary classification","authors":"Lin, Li ; ] Hsuan-Tien Lin, Ling Li","year":"2012","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2011] Yang Liu, Yan Liu, and Keith CC Chan. Ordinal regression via manifold learning. In Twenty-fifth AAAI conference on artificial intelligence, 2011.","title":"Ordinal regression via manifold learning","authors":"Yang Liu, Yan Liu, Keith Cc Chan","year":"2011","venue":"Twenty-fifth AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Martin et al., 2014] Paul Martin, Antoine Doucet, and Frédéric Jurie. Dating color images with ordinal classi- fication. In Proceedings of International Conference on Multimedia Retrieval, page 447. ACM, 2014.","title":"Dating color images with ordinal classification","authors":"Paul Martin, Antoine Doucet, Frédéric Jurie","year":"2014","venue":"Proceedings of International Conference on Multimedia Retrieval","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[McCullagh and Nelder, 1989] Peter McCullagh and John A Nelder. Generalized linear models, volume 37. CRC press, 1989.","title":"Generalized linear models","authors":"Nelder Mccullagh, Peter Mccullagh, John A Nelder","year":"1989","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Niu et al., 2016] Zhenxing Niu, Mo Zhou, Le Wang, Xinbo Gao, and Gang Hua. Ordinal regression with multiple out- put cnn for age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4920-4928, 2016.","title":"Ordinal regression with multiple output cnn for age estimation","authors":"Zhenxing Niu, Mo Zhou, Le Wang, Xinbo Gao, Gang Hua","year":"2016","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[O'Connell, 2006] Ann A O'Connell. Logistic regression models for ordinal response variables. Number 146. Sage, 2006.","title":"Logistic regression models for ordinal response variables","authors":"A O' O'connell ; Ann, Connell","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Palermo et al., 2012] Frank Palermo, James Hays, and Alexei A. Efros. Dating historical color images. In Andrew Fitzgibbon, Svetlana Lazebnik, Yoichi Sato, and Cordelia Schmid, editors, ECCV (6), volume 7577 of Lec- ture Notes in Computer Science, pages 499-512. Springer, 2012.","title":"Dating historical color images","authors":"Frank Palermo, James Hays, Alexei A Efros","year":"2012","venue":"ECCV (6)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shashua and Levin, 2002] Amnon Shashua and Anat Levin. Taxonomy of large margin principle algorithms for ordinal regression problems. Advances in neural information pro- cessing systems, 15:937-944, 2002.","title":"Taxonomy of large margin principle algorithms for ordinal regression problems","authors":"Levin ; Shashua, Amnon Shashua, Anat Levin","year":"2002","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tian and Chen, 2015] Qing Tian and Songcan Chen. A novel ordinal learning strategy: Ordinal nearest-centroid projection. Knowledge-Based Systems, 88:144-153, 2015.","title":"A novel ordinal learning strategy: Ordinal nearest-centroid projection","authors":"Chen ; ] Qing Tian, Songcan Tian, Chen","year":"2015","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2009] Meng Wang, Linjun Yang, and Xian- Sheng Hua. Msra-mm: Bridging research and industrial societies for multimedia information retrieval. Microsoft Research Asia, Tech. Rep, 2009.","title":"Msra-mm: Bridging research and industrial societies for multimedia information retrieval","authors":"Meng Wang, Linjun Yang, Xian-Sheng Hua","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}