{"type":"ScholarlyArticle","title":"High-dimensional Similarity Learning via Dual-sparse Random Projection","authors":[{"name":"Dezhong Yao","slug":"dezhong-yao","given":"Dezhong","family":"Yao","affiliation":"Rolls-Royce @ NTU Corporate Lab , Nanyang Technological University , Singapore"},{"name":"Peilin Zhao","slug":"peilin-zhao","given":"Peilin","family":"Zhao","affiliation":"South China University of Technology ; Tencent AI Lab , China"},{"name":"Tuan-Anh Nguyen Pham","slug":"tuan-anh-nguyen-pham","given":"Tuan-Anh Nguyen","family":"Pham","affiliation":""},{"name":"Gao Cong","slug":"gao-cong","given":"Gao","family":"Cong","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"}],"abstract":"We investigate how to adopt dual random projection for high-dimensional similarity learning. For a high-dimensional similarity learning  problem, projection is usually adopted to map high-dimensional features into low-dimensional space, in order to reduce the computational cost. However, dimensionality reduction method sometimes results in unstable performance due to the suboptimal solution in original space. In this paper, we propose a dual random projection framework for similarity learning to recover the original optimal solution from subspace optimal solution. Previous dual random projection methods usually make strong assumptions about the data, which need to be low rank or have a large margin. Those assumptions limit dual random projection applications in similarity learning. Thus, we adopt a dual-sparse regularized random projection method that introduces a sparse regularizer into the reduced dual problem. As the original dual solution is a sparse one, applying a sparse regularizer in the reduced space relaxes the low-rank assumption. Experimental results show that our method enjoys higher effectiveness and efficiency than state-of-the-art solutions.","doi":"10.24963/ijcai.2018/417","url":"https://ijcai.org/proceedings/2018/417","pdf":"https://ijcai.org/proceedings/2018/0417.pdf","markdown":"https://ijcai.org/proceedings/2018/417.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/417","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":417,"pages":{"first":3005,"last":3011},"keywords":["Multidisciplinary Topics and Applications: Information Retrieval","Machine Learning Applications: Applications of Supervised Learning","Machine Learning Applications: Big data ; Scalability","Machine Learning: Learning Preferences or Rankings"],"videos":[],"references":[{"position":1,"raw":"[Balcan et al., 2006] Maria-Florina Balcan, Avrim Blum, and Santosh Vempala. Kernels as features: On kernels, margins, and low-dimensional mappings. Machine Learn- ing, 65(1):79-94, 2006.","title":"Kernels as features: On kernels, margins, and low-dimensional mappings","authors":"Maria-Florina Balcan, Avrim Blum, Santosh Vempala","year":"2006","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bellet et al., 2012] Aurélien Bellet, Amaury Habrard, and Marc Sebban. Similarity learning for provably accurate sparse linear classification. In Proc. ICML, pages 1491- 1498, Edinburgh, Scotland, UK, June 2012.","title":"Similarity learning for provably accurate sparse linear classification","authors":"Aurélien Bellet, Amaury Habrard, Marc Sebban","year":"2012","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bian and Tao, 2011] Wei Bian and Dacheng Tao. Learning a distance metric by empirical loss minimization. In Proc. IJCAI, pages 1186-1191, Barcelona, Spain, July 2011.","title":"Learning a distance metric by empirical loss minimization","authors":"Wei Bian, Dacheng Tao","year":"2011","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/202"},{"position":4,"raw":"[Bingham and Mannila, 2001] Ella Bingham and Heikki Mannila. Random projection in dimensionality reduction: applications to image and text data. In Proc. SIGKDD, pages 245-250, San Francisco, CA, USA, August 2001.","title":"Random projection in dimensionality reduction: applications to image and text data","authors":"Mannila Bingham, Ella Bingham, Heikki Mannila","year":"2001","venue":"Proc. SIGKDD","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chechik et al., 2010] Gal Chechik, Varun Sharma, Uri Shalit, and Samy Bengio. Large scale online learning of image similarity through ranking. Journal of Machine Learning Research, 11:1109-1135, 2010.","title":"Large scale online learning of image similarity through ranking","authors":"Chechik","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cheng, 2013] Li Cheng. Riemannian similarity learning. In Proc. ICML, pages 540-548, Atlanta, GA, USA, June 2013.","title":"Riemannian similarity learning","authors":"Li Cheng","year":"2013","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Crammer and Chechik, 2012] Koby Crammer and Gal Chechik. Adaptive regularization for weight matrices. In Proc. ICML, pages 425-432, Edinburgh, Scotland, UK, June 2012.","title":"Adaptive regularization for weight matrices","authors":"Chechik Crammer, Koby Crammer, Gal Chechik","year":"2012","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Davis and Dhillon, 2008] Jason V. Davis and Inderjit S. Dhillon. Structured metric learning for high dimensional problems. In Proc. SIGKDD, pages 195-203, Las Vegas, NV, USA, August 2008.","title":"Structured metric learning for high dimensional problems","authors":"Jason V Davis, Inderjit S Dhillon","year":"2008","venue":"Proc. SIGKDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fern and Brodley, 2003] Xiaoli Z Fern and Carla E Brodley. Random projection for high dimensional data clustering: A cluster ensemble approach. In Proc. ICML, pages 186- 193, Washington, DC, USA, August 2003.","title":"Random projection for high dimensional data clustering: A cluster ensemble approach","authors":"Z Xiaoli, Carla E Fern, Brodley","year":"2003","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fradkin and Madigan, 2003] Dmitriy Fradkin and David Madigan. Experiments with random projections for ma- chine learning. In Proc. SIGKDD, pages 517-522, Wash- ington, DC, USA, August 2003.","title":"Experiments with random projections for machine learning","authors":"Dmitriy Fradkin, David Madigan","year":"2003","venue":"Proc. SIGKDD","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jin et al., 2009] Rong Jin, Shijun Wang, and Yang Zhou. Regularized distance metric learning: Theory and algo- rithm. In Proc. NIPS, pages 862-870, Vancouver, British Columbia, Canada, December 2009.","title":"Regularized distance metric learning: Theory and algorithm","authors":"Rong Jin, Shijun Wang, Yang Zhou","year":"2009","venue":"Proc. NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kulis, 2013] Brian Kulis. Metric learning: A survey. Foun- dations and Trends in Machine Learning, 5(4):287-364, 2013.","title":"Metric learning: A survey","authors":"Brian Kulis","year":"2013","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Maillard and Munos, 2012] Odalric-Ambrym Maillard and Rémi Munos. Linear regression with random projections. Journal of Machine Learning Research, 13:2735-2772, 2012.","title":"Odalric-Ambrym Maillard and Rémi Munos. Linear regression with random projections","authors":"Munos Maillard","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Paul et al., 2013] Saurabh Paul, Christos Boutsidis, Malik Magdon-Ismail, and Petros Drineas. Random projections for support vector machines. In Proc. AISTATS, pages 498-506, Scottsdale, AZ, USA, April 2013.","title":"Random projections for support vector machines","authors":"Saurabh Paul, Christos Boutsidis, Malik Magdon-Ismail, Petros Drineas","year":"2013","venue":"Proc. AISTATS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Qian et al., 2015] Qi Qian, Rong Jin, Shenghuo Zhu, and Yuanqing Lin. Fine-grained visual categorization via multi-stage metric learning. In Proc. CVPR, pages 3716- 3724, Boston, MA, USA, June 2015.","title":"Fine-grained visual categorization via multi-stage metric learning","authors":"Rong Qi Qian, Shenghuo Jin, Yuanqing Zhu, Lin","year":"2015","venue":"Proc. CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shalev-Shwartz and Zhang, 2013] Shai Shalev-Shwartz and Tong Zhang. Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research, 14(1):567-599, 2013.","title":"Stochastic dual coordinate ascent methods for regularized loss minimization","authors":"Shalev-Shwartz Zhang, ; ] Shai Shalev-Shwartz, Tong Zhang","year":"2013","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shalev-Shwartz et al., 2004] Shai Shalev-Shwartz, Yoram Singer, and Andrew Y. Ng. Online and batch learning of pseudo-metrics. In Proc. ICML, page 94, Banff, Alberta, Canada, July 2004.","title":"Online and batch learning of pseudo-metrics","authors":"Shai Shalev-Shwartz, Yoram Singer, Andrew Y Ng","year":"2004","venue":"Proc. ICML, page 94","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi et al., 2012] Qinfeng Shi, Chunhua Shen, Rhys Hill, and Anton van den Hengel. Is margin preserved after ran- dom projection? In Proc. ICML, pages 643-650, Edin- burgh, Scotland, UK, June 2012.","title":"Is margin preserved after random projection?","authors":"Shi","year":"2012","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Venna et al., 2010] Jarkko Venna, Jaakko Peltonen, Kristian Nybo, Helena Aidos, and Samuel Kaski. Information re- trieval perspective to nonlinear dimensionality reduction for data visualization. Journal of Machine Learning Re- search, 11:451-490, 2010.","title":"Information retrieval perspective to nonlinear dimensionality reduction for data visualization","authors":"Venna","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Weinberger and Saul, 2009] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large mar- gin nearest neighbor classification. Journal of Machine Learning Research, 10:207-244, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"Q Kilian, Lawrence K Weinberger, Saul","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu et al., 2017] Yi Xu, Haiqin Yang, Lijun Zhang, and Tianbao Yang. Efficient non-oblivious randomized reduc- tion for risk minimization with improved excess risk guar- antee. In Proc. AAAI, pages 2796-2802, San Francisco, CA, USA, February 2017.","title":"Efficient non-oblivious randomized reduction for risk minimization with improved excess risk guarantee","authors":"Yi Xu, Haiqin Yang, Lijun Zhang, Tianbao Yang","year":"2017","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2015] Tianbao Yang, Lijun Zhang, Rong Jin, and Shenghuo Zhu. Theory of dual-sparse regularized ran- domized reduction. In Proc. ICML, pages 305-314, Lille, France, July 2015.","title":"Theory of dual-sparse regularized randomized reduction","authors":"Tianbao Yang, Lijun Zhang, Rong Jin, Shenghuo Zhu","year":"2015","venue":"Proc. ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2013] Lijun Zhang, Mehrdad Mahdavi, Rong Jin, Tianbao Yang, and Shenghuo Zhu. Recovering the optimal solution by dual random projection. In Proc. COLT, pages 135-157, Princeton University, NJ, USA, June 2013.","title":"Recovering the optimal solution by dual random projection","authors":"Lijun Zhang, Mehrdad Mahdavi, Rong Jin, Tianbao Yang, Shenghuo Zhu","year":"2013","venue":"Proc. COLT","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2014] Lijun Zhang, Mehrdad Mahdavi, Rong Jin, Tianbao Yang, and Shenghuo Zhu. Random projec- tions for classification: A recovery approach. IEEE Trans. Information Theory, 60(11):7300-7316, 2014.","title":"Random projections for classification: A recovery approach","authors":"Lijun Zhang, Mehrdad Mahdavi, Rong Jin, Tianbao Yang, Shenghuo Zhu","year":"2014","venue":"IEEE Trans. Information Theory","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2016] Weizhong Zhang, Lijun Zhang, Rong Jin, Deng Cai, and Xiaofei He. Accelerated sparse linear regression via random projection. In Proc. AAAI, pages 2337-2343, Phoenix, AZ, USA, February 2016.","title":"Accelerated sparse linear regression via random projection","authors":"Weizhong Zhang, Lijun Zhang, Rong Jin, Deng Cai, Xiaofei He","year":"2016","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}