{"type":"ScholarlyArticle","title":"Robust Survey Aggregation with Student-t Distribution and Sparse Representation","authors":[{"name":"Qingtao Tang","slug":"qingtao-tang","given":"Qingtao","family":"Tang","affiliation":"Department of Computer Science and Technology , Tsinghua University , China"},{"name":"Tao Dai","slug":"tao-dai","given":"Tao","family":"Dai","affiliation":"Department of Computer Science and Technology , Tsinghua University , China"},{"name":"Li Niu","slug":"li-niu","given":"Li","family":"Niu","affiliation":"‡ School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Yisen Wang","slug":"yisen-wang","given":"Yisen","family":"Wang","affiliation":"Department of Computer Science and Technology , Tsinghua University , China"},{"name":"Shu-Tao Xia","slug":"shu-tao-xia","given":"Shu-Tao","family":"Xia","affiliation":"Department of Computer Science and Technology , Tsinghua University , China"},{"name":"Jianfei Cai","slug":"jianfei-cai","given":"Jianfei","family":"Cai","affiliation":"‡ School of Computer Science and Engineering , Nanyang Technological University , Singapore"}],"abstract":"Most existing survey aggregation methods assume that the sample data follow Gaussian distribution. However, these methods are sensitive to outliers, due to the thin-tailed property of the Gaussian distribution. To address this issue, we propose a robust survey aggregation method based on Student-t distribution and sparse representation. Specifically, we assume that the samples follow Student-$t$ distribution, instead of the common Gaussian distribution. Due to the Student-t distribution, our method is robust to outliers, which can be explained from both Bayesian point of view and non-Bayesian point of view. In addition, inspired by James-Stain estimator (JS) and Compressive Averaging (CAvg), we propose to sparsely represent the global mean vector by an adaptive basis comprising both data-specific basis and combined generic bases. Theoretically, we prove that JS and CAvg are special cases of our method. Extensive experiments demonstrate that our proposed method achieves significant improvement over the state-of-the-art methods on both synthetic and real datasets.","doi":"10.24963/ijcai.2017/394","url":"https://ijcai.org/proceedings/2017/394","pdf":"https://ijcai.org/proceedings/2017/0394.pdf","markdown":"https://ijcai.org/proceedings/2017/394.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/394","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":394,"pages":{"first":2829,"last":2835},"keywords":["Machine Learning: Machine Learning","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Aharon et al., 2006] Michal Aharon, Michael Elad, and Al- fred Bruckstein. K-svd: An algorithm for designing over- complete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54(11):4311-4322, Nov 2006.","title":"K-svd: An algorithm for designing overcomplete dictionaries for sparse representation","authors":"Michal Aharon, Michael Elad, Alfred Bruckstein","year":"2006","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arnot et al., 2006] Chris Arnot, Peter C Boxall, and Sean B Cash. Do ethical consumers care about price? a re- vealed preference analysis of fair trade coffee purchases. Canadian Journal of Agricultural Economics/Revue cana- dienne d'agroeconomie, 54(4):555-565, 2006.","title":"Do ethical consumers care about price? a revealed preference analysis of fair trade coffee purchases","authors":"Chris Arnot, Peter C Boxall, Sean B Cash","year":"2006","venue":"Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bamnett and Lewis, 1994] V Bamnett and T Lewis. Out- liers in statistical data. 1994.","title":"Outliers in statistical data","authors":"V Bamnett, Lewis","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Benton et al., 2016] Adrian Benton, Michael J Paul, Braden Hancock, and Mark Dredze. Collective supervision of topic models for predicting surveys with social media. In AAAI, pages 2892-2898. AAAI Press, 2016.","title":"Collective supervision of topic models for predicting surveys with social media","authors":"Adrian Benton, J Michael, Braden Paul, Mark Hancock, Dredze","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bock, 1975] Mary Ellen Bock. Minimax estimators of the mean of a multivariate normal distribution. The Annals of Statistics, pages 209-218, 1975.","title":"Minimax estimators of the mean of a multivariate normal distribution","authors":"Mary Ellen Bock","year":"1975","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bruckstein et al., 2009] Alfred M Bruckstein, David L Donoho, and Michael Elad. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM review, 51(1):34-81, 2009.","title":"From sparse solutions of systems of equations to sparse modeling of signals and images","authors":"David L Alfred M Bruckstein, Michael Donoho, Elad","year":"2009","venue":"SIAM review","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Candès and Wakin, 2008] Emmanuel J Candès and Michael B Wakin. An introduction to compressive sampling. IEEE signal processing magazine, 25(2):21- 30, 2008.","title":"An introduction to compressive sampling","authors":"J Emmanuel, Candès, Michael B Wakin","year":"2008","venue":"IEEE signal processing magazine","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Casella, 1985] George Casella. An introduction to empirical bayes data analysis. The American Statistician, 39(2):83- 87, 1985.","title":"An introduction to empirical bayes data analysis","authors":"George Casella","year":"1985","venue":"The American Statistician","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen and Huang, 2006] Peter Y Chen and Yueng-Hsiang Huang. Conducting telephone surveys. The Pschology Research Handbook: A Guide for Graduate Students and Research Assistants. London: Sage Publications, pages 210-226, 2006.","title":"Conducting telephone surveys","authors":"Chen, Huang ; Peter, Y Chen, Yueng-Hsiang Huang","year":"2006","venue":"The Pschology Research Handbook: A Guide for Graduate Students and Research Assistants","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dai et al., 2015] Tao Dai, Chao-Bing Song, Ji-Ping Zhang, and Shu-Tao Xia. Pmpa: A patch-based multiscale prod- ucts algorithm for image denoising. In ICIP, pages 4406- 4410. IEEE, 2015.","title":"Pmpa: A patch-based multiscale products algorithm for image denoising","authors":"Tao Dai, Chao-Bing Song, Ji-Ping Zhang, Shu-Tao Xia","year":"2015","venue":"ICIP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ding and Vishwanathan, 2010] Nan Ding and SVN Vish- wanathan. t-logistic regression. In NIPS, pages 514-522, 2010.","title":"Nan Ding and SVN Vishwanathan. t-logistic regression","authors":"Vishwanathan Ding","year":"2010","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Efron and Morris, 1972] Bradley Efron and Carl Morris. Limiting the risk of bayes and empirical bayes estima- torspart ii: The empirical bayes case. Journal of the Amer- ican Statistical Association, 67(337):130-139, 1972.","title":"Limiting the risk of bayes and empirical bayes estimatorspart ii: The empirical bayes case","authors":"Morris ; Efron, Bradley Efron, Carl Morris","year":"1972","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Feldman et al., 2012] Sergey Feldman, Maya Gupta, and Bela Frigyik. Multi-task averaging. In NIPS, pages 1169- 1177, 2012.","title":"Multi-task averaging","authors":"Sergey Feldman, Maya Gupta, Bela Frigyik","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[James and Stein, 1961] William James and Charles Stein. Estimation with quadratic loss. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1, pages 361-379, 1961.","title":"Estimation with quadratic loss","authors":"James, Stein, William James, Charles Stein","year":"1961","venue":"Proceedings of the fourth Berkeley symposium on mathematical statistics and probability","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jylänki et al., 2011] Pasi Jylänki, Jarno Vanhatalo, and Aki Vehtari. Robust gaussian process regression with a student-t likelihood. Journal of Machine Learning Re- search, 12(Nov):3227-3257, 2011.","title":"Pasi Jylänki, Jarno Vanhatalo, and Aki Vehtari. Robust gaussian process regression with a student-t likelihood","authors":"Jylänki","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Konno and Kuno, 1995] Hiroshi Konno and Takahito Kuno. Multiplicative programming problems. In Handbook of global optimization, pages 369-405. Springer, 1995.","title":"Multiplicative programming problems","authors":"Kuno Konno, Hiroshi Konno, Takahito Kuno","year":"1995","venue":"Handbook of global optimization","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kuno et al., 1993] Takahito Kuno, Yasutoshi Yajima, and Hiroshi Konno. An outer approximation method for mini- mizing the product of several convex functions on a convex set. Journal of Global optimization, 3(3):325-335, 1993.","title":"An outer approximation method for minimizing the product of several convex functions on a convex set","authors":"Takahito Kuno, Yasutoshi Yajima, Hiroshi Konno","year":"1993","venue":"Journal of Global optimization","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lesage et al., 2005] Sylvain Lesage, Rémi Gribonval, Frédéric Bimbot, and Laurent Benaroya. Learning unions of orthonormal bases with thresholded singular value decomposition. In ICASSP, volume 5, pages v-293. IEEE, 2005.","title":"Learning unions of orthonormal bases with thresholded singular value decomposition","authors":"Rémi Sylvain Lesage, Frédéric Gribonval, Laurent Bimbot, Benaroya","year":"2005","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nair et al., 2013] Jayakrishnan Nair, Adam Wierman, and Bert Zwart. The fundamentals of heavy-tails: Properties, emergence, and identification. SIGMETRICS Perform. Eval. Rev., 41(1):387-388, June 2013.","title":"The fundamentals of heavy-tails: Properties, emergence, and identification","authors":"Jayakrishnan Nair, Adam Wierman, Bert Zwart","year":"2013","venue":"SIGMETRICS Perform. Eval. Rev","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[O'Hagan, 1979] Anthony O'Hagan. On outlier rejection phenomena in bayes inference. Journal of the Royal Statis- tical Society. Series B (Methodological), pages 358-367, 1979.","title":"On outlier rejection phenomena in bayes inference","authors":"O' Anthony, Hagan","year":"1979","venue":"Journal of the Royal Statistical Society. Series B (Methodological)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shah et al., 2014] Amar Shah, Andrew Gordon Wilson, and Zoubin Ghahramani. Student-t processes as alternatives to gaussian processes. In AISTATS, 2014.","title":"Student-t processes as alternatives to gaussian processes","authors":"Amar Shah, Andrew Gordon Wilson, Zoubin Ghahramani","year":"2014","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shi et al., 2016] Tianlin Shi, Forest Agostinelli, Matthew Staib, David Wipf, and Thomas Moscibroda. Improving survey aggregation with sparsely represented signals. In KDD, pages 1845-1854, NY, USA, 2016. ACM.","title":"Improving survey aggregation with sparsely represented signals","authors":"Shi","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Silverman, 1986] Bernard W Silverman. Density estimation for statistics and data analysis, volume 26. CRC press, 1986.","title":"Density estimation for statistics and data analysis","authors":"Bernard W Silverman","year":"1986","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Smith et al., 2015] TW Smith, PV Marsden, M Hout, and J Kim. General social surveys, 1972-2014: cumulative codebook/principal investigator, tom w, 2015.","title":"General social surveys, 1972-2014: cumulative codebook/principal investigator, tom w","authors":"Smith, M Marsden, Hout, Kim","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Stein, 1956] Charles Stein. Inadmissibility of the usual es- timator for the mean of a multivariate normal distribution. In Proceedings of the Third Berkeley Symposium on Math- ematical Statistics and Probability, Volume 1: Contribu- tions to the Theory of Statistics, pages 197-206, Berkeley, Calif., 1956. University of California Press.","title":"Inadmissibility of the usual estimator for the mean of a multivariate normal distribution","authors":"Charles Stein, Stein","year":"1956","venue":"Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tang et al., 2016] Qingtao Tang, Yisen Wang, and Shu-Tao Xia. Student-t process regression with dependent student- t noise. In ECAI 2016, The Hague, The Netherlands, volume 285, page 82. IOS Press, 2016.","title":"Student-t process regression with dependent studentt noise","authors":"Tang","year":"2016","venue":"ECAI 2016","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tropp and Gilbert, 2007] Joel A Tropp and Anna C Gilbert. Signal recovery from random measurements via orthogo- nal matching pursuit. IEEE Transactions on information theory, 53(12):4655-4666, 2007.","title":"Signal recovery from random measurements via orthogonal matching pursuit","authors":"Gilbert ; Tropp, Joel A Tropp, Anna C Gilbert","year":"2007","venue":"IEEE Transactions on information theory","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}