{"type":"ScholarlyArticle","title":"Semi-supervised Max-margin Topic Model with Manifold Posterior Regularization","authors":[{"name":"Wenbo Hu","slug":"wenbo-hu","given":"Wenbo","family":"Hu","affiliation":"Tsinghua National Laboratory for Information Science and Technology (TNList) ,; State Key Lab for Intelligent Technology and Systems ,; Center for Brain-Inspired Computing Research (CBICR) , Department of Computer Science and Technology , Tsinghua University , Beijing , 100084 , China"},{"name":"Jun Zhu","slug":"jun-zhu","given":"Jun","family":"Zhu","affiliation":"Tsinghua National Laboratory for Information Science and Technology (TNList) ,; State Key Lab for Intelligent Technology and Systems ,; Center for Brain-Inspired Computing Research (CBICR) , Department of Computer Science and Technology , Tsinghua University , Beijing , 100084 , China"},{"name":"Hang Su","slug":"hang-su","given":"Hang","family":"Su","affiliation":"Tsinghua National Laboratory for Information Science and Technology (TNList) ,; State Key Lab for Intelligent Technology and Systems ,; Center for Brain-Inspired Computing Research (CBICR) , Department of Computer Science and Technology , Tsinghua University , Beijing , 100084 , China"},{"name":"Jingwei Zhuo","slug":"jingwei-zhuo","given":"Jingwei","family":"Zhuo","affiliation":"Tsinghua National Laboratory for Information Science and Technology (TNList) ,; State Key Lab for Intelligent Technology and Systems ,; Center for Brain-Inspired Computing Research (CBICR) , Department of Computer Science and Technology , Tsinghua University , Beijing , 100084 , China"},{"name":"Bo Zhang","slug":"bo-zhang","given":"Bo","family":"Zhang","affiliation":"Tsinghua National Laboratory for Information Science and Technology (TNList) ,; State Key Lab for Intelligent Technology and Systems ,; Center for Brain-Inspired Computing Research (CBICR) , Department of Computer Science and Technology , Tsinghua University , Beijing , 100084 , China"}],"abstract":"Supervised topic models leverage label information to learn discriminative latent topic representations. As collecting a fully labeled dataset is often time-consuming, semi-supervised learning is of high interest. In this paper, we present an effective semi-supervised max-margin topic model by naturally introducing manifold posterior regularization to a regularized Bayesian topic model, named LapMedLDA. The model jointly learns latent topics and a related classifier with only a small fraction of labeled documents. To perform the approximate inference, we derive an efficient stochastic gradient MCMC method. Unlike the previous semi-supervised topic models, our model adopts a tight coupling between the generative topic model and the discriminative classifier. Extensive experiments demonstrate that such tight coupling brings significant benefits in quantitative and qualitative performance.","doi":"10.24963/ijcai.2017/259","url":"https://ijcai.org/proceedings/2017/259","pdf":"https://ijcai.org/proceedings/2017/0259.pdf","markdown":"https://ijcai.org/proceedings/2017/259.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/259","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":259,"pages":{"first":1865,"last":1871},"keywords":["Natural Language Processing: Text Classification","Machine Learning: Semi-Supervised Learning","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Belkin and Niyogi, 2001] M. Belkin and P. Niyogi. Lapla- cian eigenmaps and spectral techniques for embedding and clustering. In NIPS, 2001.","title":"Laplacian eigenmaps and spectral techniques for embedding and clustering","authors":"M Belkin, P Niyogi","year":"2001","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Belkin et al., 2006] M. Belkin, P. Niyogi, and V. Sind- hwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. JMLR, 7:2399-2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"M Belkin, P Niyogi, V Sindhwani","year":"2006","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blei and McAuliffe, 2007] D. M. Blei and J. D. McAuliffe. Supervised topic models. In NIPS, 2007.","title":"Supervised topic models","authors":"D M Blei, J D Mcauliffe","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blei et al., 2003] D. Blei, A. Ng, and M. I. Jordan. La- tent dirichlet allocation. Journal of machine Learning re- search, 3:993-1022, 2003.","title":"Latent dirichlet allocation","authors":"D Blei, A Ng, M I Jordan","year":"2003","venue":"Journal of machine Learning research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cai et al., 2009] D. Cai, X. Wang, and X. He. Probabilistic dyadic data analysis with local and global consistency. In ICML, 2009.","title":"Probabilistic dyadic data analysis with local and global consistency","authors":"D Cai, X Wang, X He","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chapelle et al., 2006] O. Chapelle, B. Scholkopf, and A. Zien. Semi-supervised learning. MIT Press, 2006.","title":"Semi-supervised learning","authors":"O Chapelle, B Scholkopf, A Zien","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Crammer and Singer, 2001] K. Crammer and Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. JMLR, 2:265-292, 2001.","title":"On the algorithmic implementation of multiclass kernel-based vector machines","authors":"K Crammer, Y Singer","year":"2001","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duchi et al., 2011] J. Duchi, E. Hazan, and Y. Singer. Adap- tive subgradient methods for online learning and stochastic optimization. JMLR, 12:2121-2159, 2011.","title":"Adaptive subgradient methods for online learning and stochastic optimization","authors":"J Duchi, E Hazan, Y Singer","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Germain et al., 2009] P. Germain, A. Lacasse, F. Laviolette, and Marchand M. PAC-Bayesian learning of linear classi- fiers. In ICML, 2009.","title":"PAC-Bayesian learning of linear classifiers","authors":"P Germain, A Lacasse, F Laviolette, M Marchand","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goldberg et al., 2008] AB.","title":"AB","authors":"Goldberg","year":"2008","venue":"AB","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Goldberg, M. Li, and X. Zhu. Online manifold regularization: A new learning setting and empirical study. In ECML/PKDD, pages 393-407. Springer, 2008.","title":"Online manifold regularization: A new learning setting and empirical study","authors":"M Goldberg, X Li, Zhu","year":"2008","venue":"ECML/PKDD","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hu et al., 2017] W. Hu, J. Zhu, and B. Zhang. Fast sampling for Bayesian max-margin models. Expert Systems with Applications, 69:277-287, 2017.","title":"Fast sampling for Bayesian max-margin models","authors":"W Hu, J Zhu, B Zhang","year":"2017","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huh and Fienberg, 2012] Seungil Huh and Stephen E Fien- berg. Discriminative topic modeling based on manifold learning. ACM Transactions on Knowledge Discovery from Data (TKDD), 5(4):20, 2012.","title":"Seungil Huh and Stephen E Fienberg. Discriminative topic modeling based on manifold learning","authors":"Fienberg Huh","year":"2012","venue":"ACM Transactions on Knowledge Discovery from Data (TKDD)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Joachims, 1999] T. Joachims. Transductive inference for text classification using support vector machines. In ICML, volume 99, pages 200-209, 1999.","title":"Transductive inference for text classification using support vector machines","authors":"T Joachims","year":"1999","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Johnson and Zhang, 2015] R. Johnson and T. Zhang. Semi- supervised convolutional neural networks for text catego- rization via region embedding. In NIPS, 2015.","title":"Semisupervised convolutional neural networks for text categorization via region embedding","authors":"R Johnson, T Zhang","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Johnson and Zhang, 2016] R. Johnson and T. Zhang. Super- vised and semi-supervised text categorization using lstm for region embeddings. In ICML, 2016.","title":"Supervised and semi-supervised text categorization using lstm for region embeddings","authors":"R Johnson, T Zhang","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lacoste-Julien et al., 2008] S. Lacoste-Julien, F. Sha, and M. I. Jordan. DiscLDA: Discriminative learning for di- mensionality reduction and classification. In NIPS, 2008.","title":"DiscLDA: Discriminative learning for dimensionality reduction and classification","authors":"S Lacoste-Julien, F Sha, M I Jordan","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lu et al., 2013] Y. Lu, S. Okada, and K. Nitta. Semi- supervised latent dirichlet allocation for multi-label text classification. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pages 351-360. Springer, 2013.","title":"Semisupervised latent dirichlet allocation for multi-label text classification","authors":"Y Lu, S Okada, K Nitta","year":"2013","venue":"International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mao et al., 2012] X. Mao, Z. Ming, T. Chua, S. Li, H. Yan, and X. Li. SSHLDA: a semi-supervised hierarchical topic model. In EMNLP, 2012.","title":"SSHLDA: a semi-supervised hierarchical topic model","authors":"X Mao, Z Ming, T Chua, S Li, H Yan, X Li","year":"2012","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[McAllester, 2003] D. McAllester. Pac-bayesian stochastic model selection. Machine Learning, 51:5-21, 2003.","title":"Pac-bayesian stochastic model selection","authors":"D Mcallester","year":"2003","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Patterson and Teh, 2013] S. Patterson and Y. W. Teh. Stochastic gradient Riemannian langevin dynamics on the probability simplex. In NIPS, 2013.","title":"Stochastic gradient Riemannian langevin dynamics on the probability simplex","authors":"S Patterson, Y W Teh","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ramage et al., 2009] D. Ramage, D. Hall, R. Nallapati, and C. D. Manning. Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora. In EMNLP, 2009.","title":"Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora","authors":"D Ramage, D Hall, R Nallapati, C D Manning","year":"2009","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ramage et al., 2011] D. Ramage, C. D. Manning, and S. Dumais. Partially labeled topic models for interpretable text mining. In KDD, 2011.","title":"Partially labeled topic models for interpretable text mining","authors":"D Ramage, C D Manning, S Dumais","year":"2011","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2009] C. Wang, D. Blei, and F. Li. Simultane- ous image classification and annotation. In CVPR, 2009.","title":"Simultaneous image classification and annotation","authors":"C Wang, D Blei, F Li","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Welling and Teh, 2011] M Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In ICML, 2011.","title":"Bayesian learning via stochastic gradient Langevin dynamics","authors":"Y W Welling, Teh","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang and Wei, 2014] Y. Zhang and W. Wei. A jointly dis- tributed semi-supervised topic model. Neurocomputing, 134:38-45, 2014.","title":"A jointly distributed semi-supervised topic model","authors":"Y Zhang, W Wei","year":"2014","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zheng et al., 2015] X. Zheng, Y. Yu, and E. P. Xing. Linear time samplers for supervised topic models using composi- tional proposals. In KDD, 2015.","title":"Linear time samplers for supervised topic models using compositional proposals","authors":"Zheng","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhu et al., 2012] J. Zhu, A. Ahmed, and E. P. Xing. MedLDA: maximum margin supervised topic models. JMLR, 13(1):2237-2278, 2012.","title":"MedLDA: maximum margin supervised topic models","authors":"J Zhu, A Ahmed, E P Xing","year":"2012","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhu et al., 2014a] J. Zhu, N. Chen, H. Perkins, and E. P. Xing. Gibbs max-margin topic models with data augmen- tation. JMLR, 15:1073-1110, 2014.","title":"Gibbs max-margin topic models with data augmentation","authors":"Zhu","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhu et al., 2014b] J. Zhu, N. Chen, and E. P. Xing. Bayesian inference with posterior regularization and applications to infinite latent svms. JMLR, 15:1799-1847, 2014.","title":"Bayesian inference with posterior regularization and applications to infinite latent svms","authors":"Zhu","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhuang et al., 2013] L. Zhuang, H. Gao, J. Luo, and Z. Lin. Regularized semi-supervised latent dirichlet allocation for visual concept learning. Neurocomputing, 119:26-32, 2013.","title":"Regularized semi-supervised latent dirichlet allocation for visual concept learning","authors":"L Zhuang, H Gao, J Luo, Z Lin","year":"2013","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}