{"type":"ScholarlyArticle","title":"Nonlinear Maximum Margin Multi-View Learning with Adaptive Kernel","authors":[{"name":"Jia He","slug":"jia-he","given":"Jia","family":"He","affiliation":"Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing 100190 , China; University of Chinese Academy of Sciences , Beijing 100049 , China"},{"name":"Changying Du","slug":"changying-du","given":"Changying","family":"Du","affiliation":"Lab of Parallel Software and Computational Science , Institute of Software , CAS , Beijing , China"},{"name":"Changde Du","slug":"changde-du","given":"Changde","family":"Du","affiliation":"Research Center for Brain-Inspired Intelligence , Institute of Automation , CAS , Beijing , China; University of Chinese Academy of Sciences , Beijing 100049 , China"},{"name":"Fuzhen Zhuang","slug":"fuzhen-zhuang","given":"Fuzhen","family":"Zhuang","affiliation":"Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing 100190 , China"},{"name":"Qing He","slug":"qing-he","given":"Qing","family":"He","affiliation":"Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing 100190 , China"},{"name":"Guoping Long","slug":"guoping-long","given":"Guoping","family":"Long","affiliation":"Lab of Parallel Software and Computational Science , Institute of Software , CAS , Beijing , China"}],"abstract":"Existing multi-view learning methods based on kernel function either require the user to select and tune a single predefined kernel or have to compute and store many Gram matrices to perform multiple kernel learning. Apart from the huge consumption of manpower, computation and memory resources, most of these models seek point estimation of their parameters, and are prone to overfitting to small training data.  This paper presents an adaptive kernel nonlinear max-margin multi-view learning model under the Bayesian framework. Specifically, we regularize the posterior of an efficient multi-view latent variable model by explicitly mapping the latent representations extracted from multiple data views to a random Fourier feature space where max-margin classification constraints are imposed. Assuming these random features are drawn from Dirichlet process Gaussian mixtures, we can adaptively learn shift-invariant kernels from data according to Bochners theorem. For inference, we employ the data augmentation idea for hinge loss, and design an efficient gradient-based MCMC sampler in the augmented space. Having no need to compute the Gram matrix, our algorithm scales linearly with the size of training set. Extensive experiments on real-world datasets demonstrate that our method has superior performance.","doi":"10.24963/ijcai.2017/254","url":"https://ijcai.org/proceedings/2017/254","pdf":"https://ijcai.org/proceedings/2017/0254.pdf","markdown":"https://ijcai.org/proceedings/2017/254.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/254","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":254,"pages":{"first":1830,"last":1836},"keywords":["Machine Learning: Kernel Methods","Machine Learning: Multi-instance/Multi-label/Multi-view learning","Machine Learning: Classification","Uncertainty in AI: Approximate Probabilistic Inference"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2012] Ning Chen, Jun Zhu, Fuchun Sun, and Eric Poe Xing. Large-margin predictive latent subspace learning for multiview data analysis. PAMI, 34(12):2365- 2378, 2012.","title":"Large-margin predictive latent subspace learning for multiview data analysis","authors":"Ning Chen, Jun Zhu, Fuchun Sun, Eric Poe Xing","year":"2012","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Fang and Zhang, 2012] Zheng Fang and Zhongfei Zhang. Simultaneously combining multi-view multi-label learn- ing with maximum margin classification. In ICDM, pages 864-869, 2012.","title":"Simultaneously combining multi-view multi-label learning with maximum margin classification","authors":"Zhang ; ] Zheng Fang, Zhongfei Fang, Zhang","year":"2012","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Farquhar et al., 2005] Jason Farquhar, David Hardoon, Hongying Meng, John S Shawe-taylor, and Sandor Szed- mak. Two view learning: Svm-2k, theory and practice. In Advances in neural information processing systems, pages 355-362, 2005.","title":"Two view learning: Svm-2k, theory and practice","authors":"Jason Farquhar, David Hardoon, Hongying Meng, Sandor John S Shawe-Taylor, Szedmak","year":"2005","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ge et al., 2015] Hong Ge, Yutian Chen, ENG CAM, Mo- quan Wan, and Zoubin Ghahramani. Distributed infer- ence for dirichlet process mixture models. In ICML, pages 2276-2284, 2015.","title":"Distributed inference for dirichlet process mixture models","authors":"Ge","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ghosh and Ramamoorthi, 2003] Jayanta K Ghosh and RV Ramamoorthi. Bayesian nonparametrics, volume 1. Springer New York, 2003.","title":"Bayesian nonparametrics","authors":"K Jayanta, R V Ghosh, Ramamoorthi","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gonen, 2012] Mehmet Gonen. Bayesian efficient multiple kernel learning. In ICML, pages 1-8, 2012.","title":"Bayesian efficient multiple kernel learning","authors":"Mehmet Gonen","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2016] Jia He, Changying Du, Fuzhen Zhuang, X- in Yin, Qing He, and Guoping Long. Online bayesian max-margin subspace multi-view learning. IJCAI, pages 1555-1561, 2016.","title":"Online bayesian max-margin subspace multi-view learning","authors":"Jia He, Changying Du, Fuzhen Zhuang, Xin Yin, Qing He, Guoping Long","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/223"},{"position":8,"raw":"[Hofmann et al., 2007] Thomas Hofmann, Bernhard Schölkopf, and Alexander J. Smola. Kernel methods in machine learning. Annals of Statistics, 36(3):1171-1220, 2007.","title":"Kernel methods in machine learning","authors":"Thomas Hofmann, Bernhard Schölkopf, Alexander J Smola","year":"2007","venue":"Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jaakkola et al., 1999] Tommi Jaakkola, Marina Meila, and Tony Jebara. Maximum entropy discrimination. In Ad- vances in neural information processing systems, 1999.","title":"Maximum entropy discrimination","authors":"Tommi Jaakkola, Marina Meila, Tony Jebara","year":"1999","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kalli et al., 2011] Maria Kalli, Jim E Griffin, and Stephen G Walker. Slice sampling mixture models. Statistics and computing, 21(1):93-105, 2011.","title":"Slice sampling mixture models","authors":"Maria Kalli, Jim E Griffin, Stephen G Walker","year":"2011","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lanckriet et al., 2004] Gert RG Lanckriet, Nello Cristiani- ni, Peter Bartlett, Laurent El Ghaoui, and Michael I Jor- dan. Learning the kernel matrix with semidefinite pro- gramming. JMLR, 5:27-72, 2004.","title":"Learning the kernel matrix with semidefinite programming","authors":"Gert Rg Lanckriet, Nello Cristianini, Peter Bartlett, Laurent El Ghaoui, Michael I Jordan","year":"2004","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Neal, 2011] Radford M Neal. Mcmc using hamiltonian dy- namics. Handbook of Markov Chain Monte Carlo, 2:113- 162, 2011.","title":"Mcmc using hamiltonian dynamics","authors":"M Radford, Neal","year":"2011","venue":"Handbook of Markov Chain Monte Carlo","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Oliva et al., 2016] Junier Oliva, Avinava Dubey, Barnabas Poczos, Jeff Schneider, and Eric P. Xing. Bayesian non- parametric kernel learning. AISTATS, 2016.","title":"Bayesian nonparametric kernel learning","authors":"Oliva","year":"2016","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Polson and Scott, 2011] Nicholas G Polson and Steven L Scott. Data augmentation for support vector machines. Bayesian Analysis, 6(1):1-23, 2011.","title":"Data augmentation for support vector machines","authors":"G Nicholas, Steven L Polson, Scott","year":"2011","venue":"Bayesian Analysis","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Quang et al., 2013] Minh H Quang, Loris Bazzani, and Vit- torio Murino. A unifying framework for vector-valued manifold regularization and multi-view learning. In ICML, pages 100-108, 2013.","title":"A unifying framework for vector-valued manifold regularization and multi-view learning","authors":"Loris Minh H Quang, Vittorio Bazzani, Murino","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rahimi and Recht, 2007] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Ad- vances in neural information processing systems, pages 1177-1184, 2007.","title":"Random features for large-scale kernel machines","authors":"Recht Rahimi, Ali Rahimi, Benjamin Recht","year":"2007","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rakotomamonjy et al., 2008] Alain Rakotomamonjy, Fran- cis R Bach, Stéphane Canu, and Yves Grandvalet. Sim- plemkl. JMLR, 9(Nov):2491-2521, 2008.","title":"Simplemkl","authors":"Alain Rakotomamonjy, Stéphane Francis R Bach, Yves Canu, Grandvalet","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rudin, 2011] Walter Rudin. Fourier analysis on groups. John Wiley & Sons, 2011.","title":"Fourier analysis on groups","authors":"Walter Rudin","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sonnenburg et al., 2006] Sören Sonnenburg, Gunnar Rätsch, Christin Schäfer, and Bernhard Schölkopf. Large scale multiple kernel learning. JMLR, 7(2006):1531- 1565, 2006.","title":"Large scale multiple kernel learning","authors":"Sören Sonnenburg, Gunnar Rätsch, Christin Schäfer, Bernhard Schölkopf","year":"2006","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sun and Chao, 2013] Shiliang Sun and Guoqing Chao. Multi-view maximum entropy discrimination. In IJCAI, pages 1706-1712, 2013.","title":"Multi-view maximum entropy discrimination","authors":"Shiliang Sun, Guoqing Chao","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/253"},{"position":21,"raw":"[Szedmak and Shawe-Taylor, 2007] Sandor Szedmak and John Shawe-Taylor. Synthesis of maximum margin and multiview learning using unlabeled data. Neurocomput- ing, 70(7-9):1254-1264, 2007.","title":"Sandor Szedmak and John Shawe-Taylor. Synthesis of maximum margin and multiview learning using unlabeled data","authors":"Shawe-Taylor Szedmak","year":"2007","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Walker, 2007] Stephen G Walker. Sampling the dirichlet mixture model with slices. Communications in Statistics- Simulation and Computation R , 36(1):45-54, 2007.","title":"Sampling the dirichlet mixture model with slices","authors":"G Stephen, Walker","year":"2007","venue":"Communications in Statistics-Simulation and Computation R","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang, 2007] Chong Wang. Variational bayesian approach to canonical correlation analysis. Neural Networks, 18(3):905-910, 2007.","title":"Variational bayesian approach to canonical correlation analysis","authors":"Chong Wang","year":"2007","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu et al., 2014] Chang Xu, Dacheng Tao, Yangxi Li, and Chao Xu. Large-margin multi-view gaussian process. Multimedia Systems, 21(2):147-157, 2014.","title":"Large-margin multi-view gaussian process","authors":"Chang Xu, Dacheng Tao, Yangxi Li, Chao Xu","year":"2014","venue":"Multimedia Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu et al., 2012] Jun Zhu, Amr Ahmed, and Eric P Xing. Medlda: maximum margin supervised topic models. JML- R, 13(1):2237-2278, 2012.","title":"Medlda: maximum margin supervised topic models","authors":"Jun Zhu, Amr Ahmed, Eric P Xing","year":"2012","venue":"JML-R","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhuang et al., 2012] Fuzhen Zhuang, George Karypis, Xia Ning, Qing He, and Zhongzhi Shi. Multi-view learning via probabilistic latent semantic analysis. Information Sci- ences, 199(15):20-30, 2012.","title":"Multi-view learning via probabilistic latent semantic analysis","authors":"Zhuang","year":"2012","venue":"Information Sciences","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}