{"type":"ScholarlyArticle","title":"Sinkhorn Regression","authors":[{"name":"Lei Luo","slug":"lei-luo","given":"Lei","family":"Luo","affiliation":"JD Finance America Corporation , Mountain View , CA , USA; Department of Electrical and Computer Engineering , University of Pittsburgh , PA , USA"},{"name":"Jian Pei","slug":"jian-pei","given":"Jian","family":"Pei","affiliation":"Department of Computing Science , Simon Fraser University , Canada"},{"name":"Heng Huang","slug":"heng-huang","given":"Heng","family":"Huang","affiliation":"JD Finance America Corporation , Mountain View , CA , USA; Department of Electrical and Computer Engineering , University of Pittsburgh , PA , USA"}],"abstract":"This paper introduces a novel Robust Regression\r\n(RR) model, named Sinkhorn regression, which\r\nimposes Sinkhorn distances on both loss function\r\nand regularization. Traditional RR methods target\r\nat searching for an element-wise loss function\r\n(e.g., Lp-norm) to characterize the errors such that\r\noutlying data have a relatively smaller influence on\r\nthe regression estimator. Due to the neglect of the\r\ngeometric information, they often lead to the suboptimal\r\nresults in the practical applications. To\r\naddress this problem, we use a cross-bin distance\r\nfunction, i.e., Sinkhorn distances, to capture the geometric\r\nknowledge of real data. Sinkhorn distances\r\nis invariant in movement, rotation and zoom. Thus,\r\nour method is more robust to variations of data\r\nthan traditional regression models. Meanwhile, we\r\nleverage Kullback-Leibler divergence to relax the\r\nproposed model with marginal constraints into its\r\nunbalanced formulation to adapt more types of features.\r\nIn addition, we propose an efficient algorithm\r\nto solve the relaxed model and establish its\r\ncomplete statistical guarantees under mild conditions.\r\nExperiments on the five publicly available\r\nmicroarray data sets and one mass spectrometry\r\ndata set demonstrate the effectiveness and robustness\r\nof our method.","doi":"10.24963/ijcai.2020/360","url":"https://ijcai.org/proceedings/2020/360","pdf":"https://ijcai.org/proceedings/2020/0360.pdf","markdown":"https://ijcai.org/proceedings/2020/360.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/360","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":360,"pages":{"first":2598,"last":2604},"keywords":["Machine Learning: Classification","Machine Learning Applications: Other"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25331/file_25331.m3u8","length_sec":298,"provider":"hls"}],"references":[{"position":1,"raw":"[Arras et al., 2017] Benjamin Arras, Ehsan Azmoodeh, Guillaume Poly, and Yvik Swan. A bound on the 2-wasserstein distance between linear combinations of independent random variables. arXiv preprint arXiv:1704.01376, 2017.","title":"A bound on the 2-wasserstein distance between linear combinations of independent random variables","authors":"Benjamin Arras, Ehsan Azmoodeh, Guillaume Poly, Yvik Swan","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bartlett and Mendelson, 2002] Peter L Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. JMLR, 3(Nov):463-482, 2002.","title":"Rademacher and gaussian complexities: Risk bounds and structural results","authors":"L Peter, Shahar Bartlett, Mendelson","year":"2002","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bhattacharjee et al., 2001] Arindam Bhattacharjee, William G Richards, Jane Staunton, et al. Classification of human lung car- cinomas by mrna expression profiling reveals distinct adenocar- cinoma subclasses. PNAS, 98(24):13790-13795, 2001.","title":"Classification of human lung carcinomas by mrna expression profiling reveals distinct adenocarcinoma subclasses","authors":"Arindam Bhattacharjee, William G Richards, Jane Staunton","year":"2001","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cuturi, 2013] Marco Cuturi. Sinkhorn distances: Lightspeed com- putation of optimal transport. In NeurIPS, pages 2292-2300, 2013.","title":"Sinkhorn distances: Lightspeed computation of optimal transport","authors":"Marco Cuturi","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ding and Peng, 2005] Chris Ding and Hanchuan Peng. Minimum redundancy feature selection from microarray gene expression data. JBCB, 3(02):185-205, 2005.","title":"Minimum redundancy feature selection from microarray gene expression data","authors":"Chris Ding, Hanchuan Peng","year":"2005","venue":"JBCB","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Flamary et al., 2016] Rémi Flamary, Marco Cuturi, Nicolas Courty, and Alain Rakotomamonjy. Wasserstein discriminant analysis. Machine Learning, pages 1-23, 2016.","title":"Wasserstein discriminant analysis","authors":"Rémi Flamary, Marco Cuturi, Nicolas Courty, Alain Rakotomamonjy","year":"2016","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Stephen PA Fodor. Massively parallel genomics. Sci- ence, 277(5324):393, 1997.","title":"Massively parallel genomics","authors":"P A Stephen, Fodor","year":"1997","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Frogner et al., 2015] Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso A Poggio. Learning with a wasserstein loss. In NeurIPS, pages 2053-2061, 2015.","title":"Learning with a wasserstein loss","authors":"Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, Tomaso A Poggio","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2011] Ran He, Wei-Shi Zheng, and Bao-Gang Hu. Max- imum correntropy criterion for robust face recognition. IEEE TPAMI, 33(8):1561-1576, 2011.","title":"Maximum correntropy criterion for robust face recognition","authors":"Ran He, Wei-Shi Zheng, Bao-Gang Hu","year":"2011","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Janati et al., 2018] Hicham Janati, Marco Cuturi, and Alexandre Gramfort. Wasserstein regularization for sparse multi-task re- gression. arXiv preprint arXiv:1805.07833, 2018.","title":"Wasserstein regularization for sparse multi-task regression","authors":"Hicham Janati, Marco Cuturi, Alexandre Gramfort","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jia et al., 2012] Kui Jia, Tsung-Han Chan, and Yi Ma. Robust and practical face recognition via structured sparsity. In ECCV, pages 331-344. Springer, 2012.","title":"Robust and practical face recognition via structured sparsity","authors":"Kui Jia, Tsung-Han Chan, Yi Ma","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kira and Rendell, 1992] Kenji Kira and Larry A Rendell. A practi- cal approach to feature selection. In Machine Learning Proceed- ings 1992, pages 249-256. Elsevier, 1992.","title":"A practical approach to feature selection","authors":"Kenji Kira, Larry A Rendell","year":"1992","venue":"Machine Learning Proceedings","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kolouri and Rohde, 2015] Soheil Kolouri and Gustavo K Rohde. Transport-based single frame super resolution of very low reso- lution face images. In CVPR, pages 4876-4884, 2015.","title":"Transport-based single frame super resolution of very low resolution face images","authors":"Soheil Kolouri, Gustavo K Rohde","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kusner et al., 2015] Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. From word embeddings to document dis- tances. In ICML, pages 957-966, 2015.","title":"From word embeddings to document distances","authors":"Matt Kusner, Yu Sun, Nicholas Kolkin, Kilian Weinberger","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2013] Xiao-Xin Li, Dao-Qing Dai, Xiao-Fei Zhang, and Chuan-Xian Ren. Structured sparse error coding for face recog- nition with occlusion. IEEE TIP, 22(5):1889-1900, 2013.","title":"Structured sparse error coding for face recognition with occlusion","authors":"Xiao-Xin Li, Dao-Qing Dai, Xiao-Fei Zhang, Chuan-Xian Ren","year":"2013","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2014] Zechao Li, Jing Liu, Yi Yang, Xiaofang Zhou, et al. Clustering-guided sparse structural learning for unsuper- vised feature selection. IEEE TKDE, 26(9):2138-2150, 2014.","title":"Clustering-guided sparse structural learning for unsupervised feature selection","authors":"Zechao Li, Jing Liu, Yi Yang, Xiaofang Zhou","year":"2014","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2010] Guangcan Liu, Zhouchen Lin, and Yong Yu. Ro- bust subspace segmentation by low-rank representation. In ICML, pages 663-670, 2010.","title":"Robust subspace segmentation by low-rank representation","authors":"Guangcan Liu, Zhouchen Lin, Yong Yu","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lu et al., 2012] Can-Yi Lu, Hai Min, Zhong-Qiu Zhao, Lin Zhu, et al. Robust and efficient subspace segmentation via least squares regression. In ECCV, pages 347-360. Springer, 2012.","title":"Robust and efficient subspace segmentation via least squares regression","authors":"Lu","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Naseem et al., 2010] Imran Naseem, Roberto Togneri, and Mo- hammed Bennamoun. Linear regression for face recognition. IEEE TPAMI, 32(11):2106-2112, 2010.","title":"Linear regression for face recognition","authors":"Imran Naseem, Roberto Togneri, Mohammed Bennamoun","year":"2010","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. Efficient and robust feature selection via joint l2, 1-norms minimization. In NeurIPS, pages 1813-1821, 2010.","title":"Efficient and robust feature selection via joint l2, 1-norms minimization","authors":"Feiping Nie, Heng Huang, Xiao Cai, Chris H Ding","year":"2010","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nutt et al., 2003] Catherine L Nutt, DR Mani, Rebecca A Beten- sky, et al. Gene expression-based classification of malignant gliomas correlates better with survival than histological classi- fication. Cancer research, 63(7):1602-1607, 2003.","title":"Gene expression-based classification of malignant gliomas correlates better with survival than histological classification","authors":"Catherine L Nutt, D R Mani, Rebecca A Betensky","year":"2003","venue":"Cancer research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ozolek et al., 2014] John A Ozolek, Akif Burak Tosun, Wei Wang, Cheng Chen, et al. Accurate diagnosis of thyroid follicular le- sions from nuclear morphology using supervised learning. Med- ical image analysis, 18(5):772-780, 2014.","title":"Accurate diagnosis of thyroid follicular lesions from nuclear morphology using supervised learning","authors":"Akif John A Ozolek, Wei Burak Tosun, Cheng Wang, Chen","year":"2014","venue":"Medical image analysis","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Petricoin III et al., 2002] Emanuel F Petricoin III, David K Orn- stein, Cloud P Paweletz, Ali Ardekani, Paul S Hackett, Ben A Hitt, Alfredo Velassco, Christian Trucco, Laura Wiegand, Kamil- lah Wood, et al. Serum proteomic patterns for detection of prostate cancer. JNCI, 94(20):1576-1578, 2002.","title":"Serum proteomic patterns for detection of prostate cancer","authors":"Iii Emanuel F Petricoin, David K Ornstein, P Cloud, Ali Paweletz, Paul S Ardekani, Ben A Hackett, Alfredo Hitt, Christian Velassco, Laura Trucco, Kamillah Wiegand, Wood","year":"2002","venue":"JNCI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Raileanu and Stoffel, 2004] Laura Elena Raileanu and Kilian Stof- fel. Theoretical comparison between the gini index and informa- tion gain criteria. AMAI, 41(1):77-93, 2004.","title":"Theoretical comparison between the gini index and information gain criteria","authors":"Laura Elena, Raileanu, Kilian Stoffel","year":"2004","venue":"AMAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rolet et al., 2016] Antoine Rolet, Marco Cuturi, and Gabriel Peyré. Fast dictionary learning with a smoothed wasserstein loss. In Artificial Intelligence and Statistics, pages 630-638, 2016.","title":"Fast dictionary learning with a smoothed wasserstein loss","authors":"Rolet","year":"2016","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sandler and Lindenbaum, 2011] Roman Sandler and Michael Lin- denbaum. Nonnegative matrix factorization with earth mover's distance metric for image analysis. IEEE TPAMI, 33(8):1590- 1602, 2011.","title":"Roman Sandler and Michael Lindenbaum. Nonnegative matrix factorization with earth mover's distance metric for image analysis","authors":"Lindenbaum Sandler","year":"2011","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Singh et al., 2002] Dinesh Singh, Phillip G Febbo, Kenneth Ross, Donald G Jackson, Judith Manola, Christine Ladd, Pablo Tamayo, et al. Gene expression correlates of clinical prostate cancer behavior. Cancer cell, 1(2):203-209, 2002.","title":"Gene expression correlates of clinical prostate cancer behavior","authors":"Dinesh Singh, Phillip G Febbo, Kenneth Ross, Donald G Jackson, Judith Manola, Christine Ladd, Pablo Tamayo","year":"2002","venue":"Cancer cell","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sommerfeld et al., 2018] Max Sommerfeld, Jörn Schrieber, and Axel Munk. Optimal transport: Fast probabilistic approximation with exact solvers. arXiv preprint arXiv:1802.05570, 2018.","title":"Optimal transport: Fast probabilistic approximation with exact solvers","authors":"Sommerfeld","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sulaiman and Labadin, 2015] Muhammad Aliyu Sulaiman and Jane Labadin. Feature selection based on mutual information. In CITA, pages 1-6. IEEE, 2015.","title":"Muhammad Aliyu Sulaiman and Jane Labadin. Feature selection based on mutual information","authors":"Labadin Sulaiman","year":"2015","venue":"CITA","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Yang et al., 2006] Kun Yang, Zhipeng Cai, Jianzhong Li, and Guo- hui Lin. A stable gene selection in microarray data analysis. BMC bioinformatics, 7(1):228, 2006.","title":"A stable gene selection in microarray data analysis","authors":"Kun Yang, Zhipeng Cai, Jianzhong Li, Guohui Lin","year":"2006","venue":"BMC bioinformatics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2011] Meng Yang, Lei Zhang, Jian Yang, and David Zhang. Robust sparse coding for face recognition. In CVPR 2011, pages 625-632. IEEE, 2011.","title":"Robust sparse coding for face recognition","authors":"Meng Yang, Lei Zhang, Jian Yang, David Zhang","year":"2011","venue":"CVPR 2011","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yang et al., 2012] Jian Yang, Lei Zhang, Yong Xu, and Jing-yu Yang. Beyond sparsity: The role of l1-optimizer in pattern clas- sification. Pattern Recognition, 45(3):1104-1118, 2012.","title":"Beyond sparsity: The role of l1-optimizer in pattern classification","authors":"Jian Yang, Lei Zhang, Yong Xu, Jing-Yu Yang","year":"2012","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yang et al., 2013] Jian Yang, Delin Chu, Lei Zhang, Yong Xu, and Jingyu Yang. Sparse representation classifier steered discrim- inative projection with applications to face recognition. IEEE TNNLS, 24(7):1023-1035, 2013.","title":"Sparse representation classifier steered discriminative projection with applications to face recognition","authors":"Jian Yang, Delin Chu, Lei Zhang, Yong Xu, Jingyu Yang","year":"2013","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yang et al., 2017] Jian Yang, Lei Luo, Jianjun Qian, Ying Tai, Fan- long Zhang, and Yong Xu. Nuclear norm based matrix regression with applications to face recognition with occlusion and illumi- nation changes. IEEE TPAMI, 39(1):156-171, 2017.","title":"Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes","authors":"Jian Yang, Lei Luo, Jianjun Qian, Ying Tai, Fanlong Zhang, Yong Xu","year":"2017","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhu et al., 2015] Pengfei Zhu, Wangmeng Zuo, Lei Zhang, et al. Unsupervised feature selection by regularized self- representation. Pattern Recognition, 48(2):438-446, 2015.","title":"Unsupervised feature selection by regularized selfrepresentation","authors":"Pengfei Zhu, Wangmeng Zuo, Lei Zhang","year":"2015","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}