{"type":"ScholarlyArticle","title":"Student-t Process Regression with Student-t Likelihood","authors":[{"name":"Qingtao Tang","slug":"qingtao-tang","given":"Qingtao","family":"Tang","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Li Niu","slug":"li-niu","given":"Li","family":"Niu","affiliation":"‡ Department of Computer Science and Technology , Tsinghua University , China"},{"name":"Yisen Wang","slug":"yisen-wang","given":"Yisen","family":"Wang","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Tao Dai","slug":"tao-dai","given":"Tao","family":"Dai","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Wangpeng An","slug":"wangpeng-an","given":"Wangpeng","family":"An","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Jianfei Cai","slug":"jianfei-cai","given":"Jianfei","family":"Cai","affiliation":"‡ Department of Computer Science and Technology , Tsinghua University , China"},{"name":"Shu-Tao Xia","slug":"shu-tao-xia","given":"Shu-Tao","family":"Xia","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"}],"abstract":"Gaussian Process Regression (GPR) is a powerful Bayesian method. However, the performance of GPR can be significantly degraded when the training data are contaminated by outliers, including target outliers and input outliers. Although there are some variants of GPR (e.g., GPR with Student-t likelihood (GPRT)) aiming to handle outliers, most of the variants focus on handling the target outliers while little effort has been done to deal with the input outliers. In contrast, in this work, we aim to handle both the target outliers and the input outliers at the same time. Specifically, we replace the Gaussian noise in GPR with independent Student-t noise to cope with the target outliers. Moreover, to enhance the robustness w.r.t. the input outliers, we use a Student-t Process prior instead of the common Gaussian Process prior, leading to Student-t Process Regression with Student-t Likelihood (TPRT). We theoretically show that TPRT is more robust to both input and target outliers than GPR and GPRT, and prove that both GPR and GPRT are special cases of TPRT. Various experiments demonstrate that TPRT outperforms GPR and its variants on both synthetic and real datasets.","doi":"10.24963/ijcai.2017/393","url":"https://ijcai.org/proceedings/2017/393","pdf":"https://ijcai.org/proceedings/2017/0393.pdf","markdown":"https://ijcai.org/proceedings/2017/393.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/393","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":393,"pages":{"first":2822,"last":2828},"keywords":["Machine Learning: Machine Learning","Machine Learning: Kernel Methods"],"videos":[],"references":[{"position":1,"raw":"[Alcalá et al., 2010] J Alcalá, A Fernández, J Luengo, J Der- rac, S García, L Sánchez, and F Herrera. Keel data-mining software tool: Data set repository, integration of algo- rithms and experimental analysis framework. Journal of Multiple-Valued Logic and Soft Computing, 17(2-3):255- 287, 2010.","title":"Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework","authors":"Alcalá, J Fernández, J Luengo, S Derrac, L García, Sánchez, Herrera","year":"2010","venue":"Journal of Multiple-Valued Logic and Soft Computing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bendre et al., 1994] SM Bendre, V Barnett, and T Lewis. Outliers in statistical data, 1994.","title":"Outliers in statistical data","authors":"Bendre","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Box and Cox, 1964] George EP Box and David R Cox. An analysis of transformations. Journal of the Royal Statis- tical Society. Series B (Methodological), pages 211-252, 1964.","title":"An analysis of transformations","authors":"E P George, David R Box, Cox","year":"1964","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Box and Tiao, 1962] G. E. P. Box and G. C. Tiao. A fur- ther look at robustness via bayes's theorem. Biometrika, 49(3/4):419-432, 1962.","title":"A further look at robustness via bayes's theorem","authors":"G E P Box, G C Tiao","year":"1962","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2012] Yang Chen, Feng Chen, Jing Dai, T Charles Clancy, and Yao-Jan Wu. Student-t based robust spatio-temporal prediction. In ICDM, pages 151-160. IEEE, 2012.","title":"Student-t based robust spatio-temporal prediction","authors":"Yang Chen, Feng Chen, Jing Dai, Charles Clancy, Yao-Jan Wu","year":"2012","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deisenroth and Ng, 2015] Marc Deisenroth and Jun Wei Ng. Distributed gaussian processes. In ICML, pages 1481- 1490, 2015.","title":"Distributed gaussian processes","authors":"Ng Deisenroth, Marc Deisenroth, Jun Wei Ng","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jylänki et al., 2011] Pasi Jylänki, Jarno Vanhatalo, and Aki Vehtari. Robust gaussian process regression with a student-t likelihood. JMLR, 12(Nov):3227-3257, 2011.","title":"Pasi Jylänki, Jarno Vanhatalo, and Aki Vehtari. Robust gaussian process regression with a student-t likelihood","authors":"Jylänki","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kuss and Rasmussen, 2005] Malte Kuss and Carl Edward Rasmussen. Assessing approximate inference for binary gaussian process classification. JMLR, 6(Oct):1679-1704, 2005.","title":"Assessing approximate inference for binary gaussian process classification","authors":"Malte Kuss, Carl Edward Rasmussen","year":"2005","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kuss, 2006] Malte Kuss. Gaussian process models for ro- bust regression, classification, and reinforcement learn- ing. PhD thesis, Darmstadt University of Technology, Germany, 2006.","title":"Gaussian process models for robust regression, classification, and reinforcement learning","authors":"Malte Kuss","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lichman, 2013] M. Lichman. UCI machine learning repos- itory, 2013.","title":"UCI machine learning repository","authors":"M Lichman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu and Vasconcelos, 2015] Bo Liu and Nuno Vasconcelos. Bayesian model adaptation for crowd counts. In CVPR, pages 4175-4183, 2015.","title":"Bayesian model adaptation for crowd counts","authors":"Bo Liu, Nuno Vasconcelos","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[McNeil, 2006] Alexander J McNeil. Multivariate t distri- butions and their applications. Journal of the American Statistical Association, 101(473):390-391, 2006.","title":"Multivariate t distributions and their applications","authors":"J Alexander, Mcneil","year":"2006","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Nair et al., 2013] Jayakrishnan Nair, Adam Wierman, and Bert Zwart. The fundamentals of heavy-tails: Properties, emergence, and identification. SIGMETRICS Perform. Eval. Rev., 41(1):387-388, June 2013.","title":"The fundamentals of heavy-tails: Properties, emergence, and identification","authors":"Jayakrishnan Nair, Adam Wierman, Bert Zwart","year":"2013","venue":"SIGMETRICS Perform. Eval. Rev","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Naish-Guzman and Holden, 2008] Andrew Naish-Guzman and Sean Holden. Robust regression with twinned gaus- sian processes. In NIPS, pages 1065-1072, 2008.","title":"Robust regression with twinned gaussian processes","authors":"Holden Naish-Guzman, Andrew Naish-Guzman, Sean Holden","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Neal, 1997] Radford M Neal. Monte carlo implementation of gaussian process models for bayesian regression and classification. Technical report, Dept. of statistics and Dept. of Computer Science, University of Toronto, 1997.","title":"Monte carlo implementation of gaussian process models for bayesian regression and classification","authors":"M Radford, Neal","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Niu et al., 2015] Li Niu, Wen Li, and Dong Xu. Visual recognition by learning from web data: A weakly super- vised domain generalization approach. In CVPR, June 2015.","title":"Visual recognition by learning from web data: A weakly supervised domain generalization approach","authors":"Li Niu, Wen Li, Dong Xu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Niu et al., 2016] L. Niu, X. Xu, L. Chen, L. Duan, and D. Xu. Action and event recognition in videos by learning from heterogeneous web sources. TNNLS, PP(99):1-15, 2016.","title":"Action and event recognition in videos by learning from heterogeneous web sources","authors":"L Niu, X Xu, L Chen, L Duan, D Xu","year":"2016","venue":"TNNLS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[O'Hagan, 1979] Anthony O'Hagan. On outlier rejection phenomena in bayes inference. Journal of the Royal Statistical Society. Series B (Methodological), pages 358- 367, 1979.","title":"On outlier rejection phenomena in bayes inference","authors":"O' Anthony, Hagan","year":"1979","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rasmussen and Nickisch, 2010] Carl Edward Rasmussen and Hannes Nickisch. Gaussian processes for machine learning toolbox. JMLR, 11(Nov):3011-3015, 2010.","title":"Gaussian processes for machine learning toolbox","authors":"Nickisch Rasmussen, Carl Edward Rasmussen, Hannes Nickisch","year":"2010","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rasmussen et al., 2003] Carl Edward Rasmussen, Malte Kuss, et al. Gaussian processes in reinforcement learning. In NIPS, volume 4, page 1, 2003.","title":"Gaussian processes in reinforcement learning","authors":"Carl Edward Rasmussen, Malte Kuss","year":"2003","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rasmussen, 2006] Carl Edward Rasmussen. Gaussian pro- cesses for machine learning. 2006.","title":"Gaussian processes for machine learning","authors":"Carl Edward, Rasmussen","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rue et al., 2009] Håvard Rue, Sara Martino, and Nicolas Chopin. Approximate bayesian inference for latent gaus- sian models by using integrated nested laplace approxi- mations. Journal of the royal statistical society: Series b (statistical methodology), 71(2):319-392, 2009.","title":"Approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations","authors":"Håvard Rue, Sara Martino, Nicolas Chopin","year":"2009","venue":"Journal of the royal statistical society: Series b (statistical methodology)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Senanayake et al., 2016] Ransalu Senanayake, Simon OCallaghan, and Fabio Ramos. Predicting spatio- temporal propagation of seasonal influenza using variational gaussian process regression. In AAAI, 2016.","title":"Predicting spatiotemporal propagation of seasonal influenza using variational gaussian process regression","authors":"Senanayake","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shah et al., 2014] Amar Shah, Andrew Wilson, and Zoubin Ghahramani. Student-t processes as alternatives to gaus- sian processes. In AISTAT, pages 877-885, 2014.","title":"Student-t processes as alternatives to gaussian processes","authors":"Amar Shah, Andrew Wilson, Zoubin Ghahramani","year":"2014","venue":"AISTAT","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Silverman, 1986] Bernard W Silverman. Density estimation for statistics and data analysis, volume 26. CRC press, 1986.","title":"Density estimation for statistics and data analysis","authors":"Bernard W Silverman","year":"1986","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Solin and Särkkä, 2015] Arno Solin and Simo Särkkä. State space methods for efficient inference in student-t process regression. In AISTAT, pages 885-893, 2015.","title":"State space methods for efficient inference in student-t process regression","authors":"Särkkä Solin, Arno Solin, Simo Särkkä","year":"2015","venue":"AISTAT","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tang et al., 2016] Qingtao Tang, Yisen Wang, and Shu-Tao Xia. Student-t process regression with dependent student-t noise. In ECAI, pages 82-89, 2016.","title":"Student-t process regression with dependent student-t noise","authors":"Tang","year":"2016","venue":"ECAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vanhatalo et al., 2009] Jarno Vanhatalo, Pasi Jylänki, and Aki Vehtari. Gaussian process regression with student-t likelihood. In NIPS, pages 1910-1918, 2009.","title":"Jarno Vanhatalo, Pasi Jylänki, and Aki Vehtari. Gaussian process regression with student-t likelihood","authors":"Vanhatalo","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[West, 1984] Mike West. Outlier models and prior distribu- tions in bayesian linear regression. Journal of the Royal Statistical Society. Series B (Methodological), pages 431- 439, 1984.","title":"Outlier models and prior distributions in bayesian linear regression","authors":"Mike West","year":"1984","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Williams and Seeger, 2000] Christopher KI Williams and Matthias Seeger. Using the nyström method to speed up kernel machines. In NIPS, pages 661-667, 2000.","title":"Using the nyström method to speed up kernel machines","authors":"K I Christopher, Matthias Williams, Seeger","year":"2000","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wilson and Nickisch, 2015] Andrew Wilson and Hannes Nickisch. Kernel interpolation for scalable structured gaussian processes. In ICML, pages 1775-1784, 2015.","title":"Kernel interpolation for scalable structured gaussian processes","authors":"Wilson, Nickisch, Andrew Wilson, Hannes Nickisch","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}