{"type":"ScholarlyArticle","title":"Robust Regression via Heuristic Hard Thresholding","authors":[{"name":"Xuchao Zhang","slug":"xuchao-zhang","given":"Xuchao","family":"Zhang","affiliation":"† Virginia Tech , Falls Church , VA , USA"},{"name":"Liang Zhao","slug":"liang-zhao","given":"Liang","family":"Zhao","affiliation":"‡ George Mason University , Fairfax , VA , USA"},{"name":"Arnold P. Boedihardjo","slug":"arnold-p-boedihardjo","given":"Arnold P.","family":"Boedihardjo","affiliation":"§ U. S. Army Corps of Engineers , Alexandria , VA , USA"},{"name":"Chang-Tien Lu","slug":"chang-tien-lu","given":"Chang-Tien","family":"Lu","affiliation":"† Virginia Tech , Falls Church , VA , USA"}],"abstract":"The presence of data noise and corruptions recently invokes increasing attention on Robust Least Squares Regression (RLSR), which addresses the fundamental problem that learns reliable regression coefficients when response variables can be arbitrarily corrupted. Until now, several important challenges still cannot be handled concurrently: 1) exact recovery guarantee of regression coefficients 2) difficulty in estimating the corruption ratio parameter; and 3) scalability to massive dataset. This paper proposes a novel Robust Least squares regression algorithm via Heuristic Hard thresholding (RLHH), that concurrently addresses all the above challenges. Specifically, the algorithm alternately  optimizes the regression coefficients and estimates the optimal uncorrupted set via heuristic hard thresholding without corruption ratio parameter until it converges. We also prove that our algorithm benefits from strong guarantees analogous to those of state-of-the-art methods in terms of convergence rates and recovery guarantees. We provide empirical evidence to demonstrate that the effectiveness of our new method is superior to that of existing methods in the recovery of both regression coefficients and uncorrupted sets, with very competitive efficiency.","doi":"10.24963/ijcai.2017/480","url":"https://ijcai.org/proceedings/2017/480","pdf":"https://ijcai.org/proceedings/2017/0480.pdf","markdown":"https://ijcai.org/proceedings/2017/480.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/480","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":480,"pages":{"first":3434,"last":3440},"keywords":["Constraints and Satisfiability: Constraint Optimisation","Combinatorial & Heuristic Search: Combinatorial search/optimisation","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Bhatia et al., 2015] Kush Bhatia, Prateek Jain, and Pu- rushottam Kar. Robust regression via hard thresholding. In Advances in Neural Information Processing Systems, pages 721-729, 2015.","title":"Robust regression via hard thresholding","authors":"Bhatia","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2013] Yudong Chen, Constantine Caramanis, and Shie Mannor. Robust sparse regression under ad- versarial corruption. In Sanjoy Dasgupta and David Mcallester, editors, Proceedings of the 30th International Conference on Machine Learning (ICML-13), volume 28, pages 774-782. JMLR Workshop and Conference Pro- ceedings, May 2013.","title":"Robust sparse regression under adversarial corruption","authors":"Yudong Chen, Constantine Caramanis, Shie Mannor","year":"2013","venue":"Proceedings of the 30th International Conference on Machine Learning (ICML-13)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Huber and Ronchetti, 2009] Peter J. Huber and Elvezio M. Ronchetti. The Basic Types of Estimates, pages 45-70.","title":"The Basic Types of Estimates","authors":"J Peter, Elvezio M Huber, Ronchetti","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Jung et al., 2016] Yoonsuh Jung, Seung Pil Lee, and Jian- hua Hu. Robust regression for highly corrupted response by shifting outliers. Statistical Modelling, 16(1):1-23, 2016.","title":"Robust regression for highly corrupted response by shifting outliers","authors":"Jung","year":"2016","venue":"Statistical Modelling","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Lourenco et al., 2011] VM Lourenco, Ana M Pires, and M Kirst. Robust linear regression methods in association studies. Bioinformatics, 27(6):815-821, 2011.","title":"Robust linear regression methods in association studies","authors":"Ana M Vm Lourenco, M Pires, Kirst","year":"2011","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Maronna et al., 2006] RARD Maronna, R Douglas Martin, and Victor Yohai. Robust statistics. John Wiley & Sons, Chichester. ISBN, 2006.","title":"Robust statistics","authors":"Maronna","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Mcwilliams et al., 2014] Brian Mcwilliams, Gabriel Krum- menacher, Mario Lucic, and Joachim M. Buhmann. Fast and robust least squares estimation in corrupted linear models. In Z. Ghahramani, M. Welling, C. Cortes, N.d. Lawrence, and K.q. Weinberger, editors, Advances in Neu- ral Information Processing Systems 27, pages 415-423.","title":"Fast and robust least squares estimation in corrupted linear models","authors":"Brian Mcwilliams, Gabriel Krummenacher, Mario Lucic, Joachim M Buhmann","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"Curran Associates, Inc., 2014.","title":null,"authors":"Curran Associates, Inc","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Naseem et al., 2012] Imran Naseem, Roberto Togneri, and Mohammed Bennamoun. Robust regression for face recognition. Pattern Recognition, 45(1):104-118, 2012.","title":"Robust regression for face recognition","authors":"Imran Naseem, Roberto Togneri, Mohammed Bennamoun","year":"2012","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Nguyen and Tran, 2013] Nam H Nguyen and Trac D Tran. Exact recoverability from dense corrupted observations via l1-minimization. IEEE transactions on information theory, 59(4):2017-2035, 2013.","title":"Exact recoverability from dense corrupted observations via l1-minimization","authors":"Tran ; ] Nguyen, H Nam, Trac D Nguyen, Tran","year":"2013","venue":"IEEE transactions on information theory","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Rousseeuw and Leroy, 2005] Peter J Rousseeuw and An- nick M Leroy. Robust regression and outlier detection, volume 589. John wiley & sons, 2005.","title":"Robust regression and outlier detection","authors":"Leroy ; Rousseeuw, J Peter, Annick M Rousseeuw, Leroy","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[She and Owen, 2011] Yiyuan She and Art B. Owen. Out- lier detection using nonconvex penalized regression. Jour- nal of the American Statistical Association, 106(494):626- 639, 2011.","title":"Yiyuan She and Art B. Owen. Outlier detection using nonconvex penalized regression","authors":"Owen She","year":"2011","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Smolic and Ohm, 2000] Aljoscha Smolic and Jens-Rainer Ohm. Robust global motion estimation using a simpli- fied m-estimator approach. In Image Processing, 2000. Proceedings. 2000 International Conference on, volume 1, pages 868-871. IEEE, 2000.","title":"Robust global motion estimation using a simplified m-estimator approach","authors":"Ohm Smolic, Aljoscha Smolic, Jens-Rainer Ohm","year":"2000","venue":"Proceedings. 2000 International Conference on","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Studer et al., 2012] Christoph Studer, Patrick Kuppinger, Graeme Pope, and Helmut Bolcskei. Recovery of sparsely corrupted signals. IEEE Transactions on Information The- ory, 58(5):3115-3130, 2012.","title":"Recovery of sparsely corrupted signals","authors":"Christoph Studer, Patrick Kuppinger, Graeme Pope, Helmut Bolcskei","year":"2012","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wright and Ma, 2010] John Wright and Yi Ma. Dense error correction via l1-minimization. IEEE Trans. Inf. Theor., 56(7):3540-3560, July 2010.","title":"Dense error correction via l1-minimization","authors":"Ma ; Wright, John Wright, Yi Ma","year":"2010","venue":"IEEE Trans. Inf. Theor","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wright et al., 2009] John Wright, Allen Y Yang, Arvind Ganesh, S Shankar Sastry, and Yi Ma. Robust face recog- nition via sparse representation. IEEE transactions on pattern analysis and machine intelligence, 31(2):210-227, 2009.","title":"Robust face recognition via sparse representation","authors":"John Wright, Allen Y Yang, Arvind Ganesh, S Shankar Sastry, Yi Ma","year":"2009","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Yang et al., 2010] Allen Yang, Arvind Ganesh, Shankar Sastry, and Yi Ma. Fast l1-minimization algorithms and an application in robust face recognition: A review. Techni- cal Report UCB/EECS-2010-13, EECS Department, Uni- versity of California, Berkeley, Feb 2010.","title":"Fast l1-minimization algorithms and an application in robust face recognition: A review","authors":"Allen Yang, Arvind Ganesh, Shankar Sastry, Yi Ma","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zoubir et al., 2012] Abdelhak M Zoubir, Visa Koivunen, Yacine Chakhchoukh, and Michael Muma. Robust esti- mation in signal processing: A tutorial-style treatment of fundamental concepts. IEEE Signal Processing Magazine, 29(4):61-80, 2012.","title":"Robust estimation in signal processing: A tutorial-style treatment of fundamental concepts","authors":"Visa Abdelhak M Zoubir, Yacine Koivunen, Michael Chakhchoukh, Muma","year":"2012","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}