{"type":"ScholarlyArticle","title":"Self-paced Mixture of Regressions","authors":[{"name":"Longfei Han","slug":"longfei-han","given":"Longfei","family":"Han","affiliation":"School of Information and Electronics , Beijing Institute of Technology"},{"name":"Dingwen Zhang","slug":"dingwen-zhang","given":"Dingwen","family":"Zhang","affiliation":"School of Automation , Northwestern Polytechnical University"},{"name":"Dong Huang","slug":"dong-huang","given":"Dong","family":"Huang","affiliation":"School of Computer Science , Carnegie Mellon University"},{"name":"Xiaojun Chang","slug":"xiaojun-chang","given":"Xiaojun","family":"Chang","affiliation":"School of Computer Science , Carnegie Mellon University"},{"name":"Jun Ren","slug":"jun-ren","given":"Jun","family":"Ren","affiliation":"Beijing Electro-Mechanical Engineering Institute"},{"name":"Senlin Luo","slug":"senlin-luo","given":"Senlin","family":"Luo","affiliation":"School of Information and Electronics , Beijing Institute of Technology"},{"name":"Junwei Han","slug":"junwei-han","given":"Junwei","family":"Han","affiliation":"School of Automation , Northwestern Polytechnical University"}],"abstract":"Mixture of regressions (MoR) is the well-established and effective approach to model discontinuous and heterogeneous data in regression problems. Existing MoR approaches assume smooth joint distribution for its good anlaytic properties. However, such assumption makes existing MoR very sensitive to intra-component outliers (the noisy training data residing in certain components) and the inter-component imbalance (the different amounts of training data in different components). In this paper, we make the earliest effort on Self-paced Learning (SPL) in MoR, i.e., Self-paced mixture of regressions (SPMoR) model. We propose a novel self-paced regularizer based on the Exclusive LASSO, which improves inter-component balance of training data. As a robust learning regime, SPL pursues confidence sample reasoning. To demonstrate the effectiveness of SPMoR, we conducted experiments on both the sythetic examples and real-world applications to age estimation and glucose estimation. The results show that SPMoR outperforms the state-of-the-arts methods.","doi":"10.24963/ijcai.2017/252","url":"https://ijcai.org/proceedings/2017/252","pdf":"https://ijcai.org/proceedings/2017/0252.pdf","markdown":"https://ijcai.org/proceedings/2017/252.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/252","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":252,"pages":{"first":1816,"last":1822},"keywords":["Machine Learning: Data Mining","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Basso et al., 2010] Rodrigo M Basso, Víctor H Lachos, Celso Rômulo Barbosa Cabral, and Pulak Ghosh. Robust mixture modeling based on scale mixtures of skew-normal distributions. CSDA, 54(12):2926-2941, 2010.","title":"Robust mixture modeling based on scale mixtures of skew-normal distributions","authors":"Rodrigo M Basso, Celso Víctor H Lachos, Rômulo Barbosa, Pulak Cabral, Ghosh","year":"2010","venue":"CSDA","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bennett et al., 2007] CM Bennett, M Guo, and SC Dhar- mage. Hba1c as a screening tool for detection of type 2 di- abetes: a systematic review. Diabetic Medicine, 24(4):333- 343, 2007.","title":"Hba1c as a screening tool for detection of type 2 diabetes: a systematic review","authors":"M Bennett, Guo, Dharmage","year":"2007","venue":"Diabetic Medicine","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Campbell and Allen, 2015] Frederick Campbell and Genev- era I Allen. Within group variable selection through the exclusive lasso. arXiv preprint arXiv:1505.07517, 2015.","title":"Frederick Campbell and Genevera I Allen. Within group variable selection through the exclusive lasso","authors":"Allen Campbell","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chamroukhi, 2016] Faicel Chamroukhi. Robust mixture of experts modeling using the t distribution. Neural Networks, 79:20-36, 2016.","title":"Faicel Chamroukhi. Robust mixture of experts modeling using the t distribution","authors":"Chamroukhi","year":"2016","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chang et al., 2011] Kuang-Yu Chang, Chu-Song Chen, and Yi-Ping Hung. Ordinal hyperplanes ranker with cost sensi- tivities for age estimation. In CVPR, 2011.","title":"Ordinal hyperplanes ranker with cost sensitivities for age estimation","authors":"Kuang-Yu Chang, Chu-Song Chen, Yi-Ping Hung","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cheng et al., 2016] Gong Cheng, Peicheng Zhou, and Jun- wei Han. Rifd-cnn: Rotation-invariant and fisher discrimi- native convolutional neural networks for object detection. In CVPR, 2016.","title":"Rifd-cnn: Rotation-invariant and fisher discriminative convolutional neural networks for object detection","authors":"Cheng","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Geng et al., 2013] Xin Geng, Chao Yin, and Zhi-Hua Zhou. Facial age estimation by learning from label distributions. TPAMI, 35(10):2401-2412, 2013.","title":"Facial age estimation by learning from label distributions","authors":"Xin Geng, Zhi-Hua Chao Yin, Zhou","year":"2013","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo and Mu, 2011] Guodong Guo and Guowang Mu. Simul- taneous dimensionality reduction and human age estimation via kernel partial least squares regression. In CVPR, 2011.","title":"Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression","authors":"Mu Guo, Guodong Guo, Guowang Mu","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo and Mu, 2013] Guodong Guo and Guowang Mu. Joint estimation of age, gender and ethnicity: Cca vs. pls. In AFGR, 2013.","title":"Joint estimation of age, gender and ethnicity: Cca vs. pls","authors":"Mu Guo, Guodong Guo, Guowang Mu","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2009] Guodong Guo, Guowang Mu, Yun Fu, and Thomas S Huang. Human age estimation using bio-inspired features. In CVPR, 2009.","title":"Human age estimation using bio-inspired features","authors":"Guodong Guo, Guowang Mu, Yun Fu, Thomas S Huang","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Han et al., 2015] Hu Han, Charles Otto, Xiaoming Liu, and Anil K Jain. Demographic estimation from face images: Human vs. machine performance. TPAMI, 37(6):1148- 1161, 2015.","title":"Demographic estimation from face images: Human vs. machine performance","authors":"Charles Hu Han, Xiaoming Otto, Anil K Liu, Jain","year":"2015","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hara and Chellappa, 2014] Kota Hara and Rama Chellappa. Growing regression forests by classification: Applications to object pose estimation. In ECCV, 2014.","title":"Growing regression forests by classification: Applications to object pose estimation","authors":"Chellappa Hara, Kota Hara, Rama Chellappa","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang and Yao, 2012] Mian Huang and Weixin Yao. Mix- ture of regression models with varying mixing proportions: a semiparametric approach. JASA, 107(498):711-724, 2012.","title":"Mixture of regression models with varying mixing proportions: a semiparametric approach","authors":"Mian Huang, Weixin Yao","year":"2012","venue":"JASA","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jacobs et al., 1991] Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mix- tures of local experts. Neural computation, 3(1):79-87, 1991.","title":"Adaptive mixtures of local experts","authors":"Robert A Jacobs, Michael I Jordan, Steven J Nowlan, Geoffrey E Hinton","year":"1991","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jiang et al., 2014a] Lu Jiang, Deyu Meng, Teruko Mitamura, and Alexander G Hauptmann. Easy samples first: Self- paced reranking for zero-example multimedia search. In ACM MM, 2014. Jiang et al., 2014b] Lu Jiang, Deyu Meng, Shoou-I Yu, Zhenzhong Lan, Shiguang Shan, and Alexander Haupt- mann. Self-paced learning with diversity. In NIPS, 2014.","title":"Easy samples first: Selfpaced reranking for zero-example multimedia search","authors":"Jiang","year":"2014","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jordan and Jacobs, 1994] Michael I Jordan and Robert A Ja- cobs. Hierarchical mixtures of experts and the em algorithm. Neural computation, 6(2):181-214, 1994.","title":"Hierarchical mixtures of experts and the em algorithm","authors":"I Michael, Robert A Jordan, Jacobs","year":"1994","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jordan and Xu, 1995] Michael I Jordan and Lei Xu. Conver- gence results for the em approach to mixtures of experts architectures. Neural networks, 8(9):1409-1431, 1995.","title":"Convergence results for the em approach to mixtures of experts architectures","authors":"Jordan, Xu ; Michael, I Jordan, Lei Xu","year":"1995","venue":"Neural networks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kong et al., 2014] Deguang Kong, Ryohei Fujimaki, Ji Liu, Feiping Nie, and Chris Ding. Exclusive feature learning on arbitrary structures via ℓ 1,2 -norm. In NIPS, 2014.","title":"Exclusive feature learning on arbitrary structures via ℓ 1,2 -norm","authors":"Kong","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kumar et al., 2010] M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable mod- els. In NIPS, 2010.","title":"Self-paced learning for latent variable models","authors":"Benjamin M Pawan Kumar, Daphne Packer, Koller","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin, 2010] Tsung-I Lin. Robust mixture modeling using multivariate skew t distributions. SC, 20(3):343-356, 2010.","title":"Robust mixture modeling using multivariate skew t distributions","authors":"Tsung-I Lin","year":"2010","venue":"SC","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nguyen and McLachlan, 2016] Hien D Nguyen and Geof- frey J McLachlan. Laplace mixture of linear experts. CSDA, 93:177-191, 2016.","title":"Laplace mixture of linear experts","authors":"D Hien, Geoffrey J Nguyen, Mclachlan","year":"2016","venue":"CSDA","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ricanek and Tesafaye, 2006] Karl Ricanek and Tamirat Tesafaye. Morph: A longitudinal image database of normal adult age-progression. In AFGR, 2006.","title":"Karl Ricanek and Tamirat Tesafaye. Morph: A longitudinal image database of normal adult age-progression","authors":"Tesafaye Ricanek","year":"2006","venue":"AFGR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Song et al., 2014] Weixing Song, Weixin Yao, and Yanru Xing. Robust mixture regression model fitting by laplace distribution. CSDA, 71:128-137, 2014.","title":"Robust mixture regression model fitting by laplace distribution","authors":"Weixing Song, Weixin Yao, Yanru Xing","year":"2014","venue":"CSDA","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Supancic and Ramanan, 2013] James S Supancic and Deva Ramanan. Self-paced learning for long-term tracking. In CVPR, 2013.","title":"Self-paced learning for long-term tracking","authors":"S James, Deva Supancic, Ramanan","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Vijayakumar et al., 2017] Pavithra Vijayakumar, Robert G Nelson, Robert L Hanson, William C Knowler, and Mad- humita Sinha. Hba1c and the prediction of type 2 diabetes in children and adults. Diabetes Care, 40(1):16-21, 2017.","title":"Hba1c and the prediction of type 2 diabetes in children and adults","authors":"Pavithra Vijayakumar, Robert L Robert G Nelson, William C Hanson, Madhumita Knowler, Sinha","year":"2017","venue":"Diabetes Care","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Young and Hunter, 2010] Derek S Young and David R Hunter. Mixtures of regressions with predictor-dependent mixing proportions. CSDA, 54(10):2253-2266, 2010.","title":"Mixtures of regressions with predictor-dependent mixing proportions","authors":"S Derek, David R Young, Hunter","year":"2010","venue":"CSDA","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yuan and Lin, 2006] Ming Yuan and Yi Lin. Model selec- tion and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B, 68(1):49- 67, 2006.","title":"Model selection and estimation in regression with grouped variables","authors":"Lin ; Yuan, Ming Yuan, Yi Lin","year":"2006","venue":"Journal of the Royal Statistical Society: Series B","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2016] Dingwen Zhang, Junwei Han, Chao Li, Jingdong Wang, and Xuelong Li. Detection of co-salient objects by looking deep and wide. IJCV, 120(2):215-232, 2016.","title":"Detection of co-salient objects by looking deep and wide","authors":"Zhang","year":"2016","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2017] Dingwen Zhang, Deyu Meng, and Jun- wei Han. Co-saliency detection via a self-paced multiple- instance learning framework. TPAMI, 39(5):865-878, 2017.","title":"Co-saliency detection via a self-paced multipleinstance learning framework","authors":"Zhang","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zipf et al., 2013] George Zipf, Michele Chiappa, Kathryn S Porter, Yechiam Ostchega, Brenda G Lewis, and Jennifer Dostal. National health and nutrition examination survey: plan and operations, 1999-2010. Vital Health Stat 1, (56):1- 37, 2013.","title":"National health and nutrition examination survey: plan and operations, 1999-2010","authors":"George Zipf, Michele Chiappa, Kathryn S Porter, Yechiam Ostchega, Brenda G Lewis, Jennifer Dostal","year":"2013","venue":"Vital Health Stat","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}