{"type":"ScholarlyArticle","title":"Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning","authors":[{"name":"Lidong Bing","slug":"lidong-bing","given":"Lidong","family":"Bing","affiliation":"AI Lab Tencent Inc."},{"name":"William W. Cohen","slug":"william-w-cohen","given":"William W.","family":"Cohen","affiliation":"Machine Learning Department Carnegie Mellon University"},{"name":"Bhuwan Dhingra","slug":"bhuwan-dhingra","given":"Bhuwan","family":"Dhingra","affiliation":"Language Technologies Institute Carnegie Mellon University"}],"abstract":"We propose a general approach to modeling semi-supervised learning (SSL) algorithms. Specifically, we present a declarative language for modeling both traditional supervised classification tasks and many SSL heuristics, including both well-known heuristics such as co-training and novel domain-specific heuristics. In addition to representing individual SSL heuristics, we show that multiple heuristics can be automatically combined using Bayesian optimization methods. We experiment with two classes of tasks, link-based text classification and relation extraction.  We show modest improvements on well-studied link-based classification benchmarks, and state-of-the-art results on relation-extraction tasks for two realistic domains.","doi":"10.24963/ijcai.2017/201","url":"https://ijcai.org/proceedings/2017/201","pdf":"https://ijcai.org/proceedings/2017/0201.pdf","markdown":"https://ijcai.org/proceedings/2017/201.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/201","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":201,"pages":{"first":1454,"last":1460},"keywords":["Machine Learning: Semi-Supervised Learning","Natural Language Processing: Text Classification","Natural Language Processing: Information Extraction","Machine Learning: Multi-instance/Multi-label/Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled exam- ples. The Journal of Machine Learning Research, 7:2399- 2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"Mikhail Belkin, Partha Niyogi, Vikas Sindhwani","year":"2006","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bing et al., 2015] Lidong Bing, Sneha Chaudhari, Richard Wang C, and William W Cohen. Improving distant su- pervision for information extraction using label propaga- tion through lists. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 524-529, Lisbon, Portugal, September 2015. Asso- ciation for Computational Linguistics.","title":"Improving distant supervision for information extraction using label propagation through lists","authors":"Lidong Bing, Sneha Chaudhari, Richard Wang, C, William W Cohen","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bing et al., 2016] Lidong Bing, Mingyang Ling, Richard C. Wang, and William W. Cohen. Distant IE by bootstrap- ping using lists and document structure. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA., pages 2899-2905, 2016.","title":"Distant IE by bootstrapping using lists and document structure","authors":"Lidong Bing, Mingyang Ling, Richard C Wang, William W Cohen","year":"2016","venue":"Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bing et al., 2017] Lidong Bing, Bhuwan Dhingra, Kathryn Mazaitis, Jong Hyuk Park, and William W. Cohen. Boot- strapping distantly supervised ie using joint learning and small well-structured corpora. In Proceedings of the Thirty First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA., 2017.","title":"Bootstrapping distantly supervised ie using joint learning and small well-structured corpora","authors":"Lidong Bing, Bhuwan Dhingra, Kathryn Mazaitis, Jong Hyuk Park, William W Cohen","year":"2017","venue":"Proceedings of the Thirty First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Grandvalet and Bengio, 2004] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimiza- tion. In Advances in neural information processing sys- tems, pages 529-536, 2004.","title":"Semi-supervised learning by entropy minimization","authors":"Bengio Grandvalet, Yves Grandvalet, Yoshua Bengio","year":"2004","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hoffmann et al., 2011] Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. Knowledge-based weak supervision for information ex- traction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 541-550. Association for Computational Linguis- tics, 2011.","title":"Knowledge-based weak supervision for information extraction of overlapping relations","authors":"Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, Daniel S Weld","year":"2011","venue":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Joachims, 1999] Thorsten Joachims. Transductive inference for text classification using support vector machines. In Proceedings of the Sixteenth International Conference on Machine Learning, ICML '99, pages 200-209, San Fran- cisco, CA, USA, 1999.","title":"Transductive inference for text classification using support vector machines","authors":"Thorsten Joachims","year":"1999","venue":"Proceedings of the Sixteenth International Conference on Machine Learning, ICML '99","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Morgan Kaufmann Publishers Inc. Mintz et al., 2009] Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. Distant supervision for relation extrac- tion without labeled data. In Proceedings of the Joint Con- ference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Pro- cessing of the AFNLP: Volume 2-Volume 2, pages 1003- 1011. Association for Computational Linguistics, 2009.","title":"Distant supervision for relation extraction without labeled data","authors":"Mike Mintz, Steven Bills, Rion Snow, Dan Jurafsky","year":"2009","venue":"Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Riedel et al., 2010] Sebastian Riedel, Limin Yao, and An- drew McCallum. Modeling relations and their mentions without labeled text. In Machine Learning and Knowledge Discovery in Databases, pages 148-163. Springer, 2010.","title":"Modeling relations and their mentions without labeled text","authors":"Riedel","year":"2010","venue":"Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Mark Namata, Mustafa Bilgic, Lise Getoor, Brian Gallagher, and Tina Eliassi-Rad. Collective classification in network data. AI Magazine, 29(3):93-106, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Mark Galileo, Mustafa Namata, Lise Bilgic, Brian Getoor, Tina Gallagher, Eliassi-Rad","year":"2008","venue":"AI Magazine","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Snoek et al., 2012] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of ma- chine learning algorithms. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 2951- 2959.","title":"Practical bayesian optimization of machine learning algorithms","authors":"Snoek","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"Curran Associates, Inc., 2012.","title":null,"authors":"Curran Associates, Inc","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Surdeanu et al., 2012] Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. Multi- instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computa- tional Natural Language Learning, EMNLP-CoNLL '12, pages 455-465, Stroudsburg, PA, USA, 2012. Association for Computational Linguistics.","title":"Multiinstance multi-label learning for relation extraction","authors":"Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, Christopher D Manning","year":"2012","venue":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '12","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Talukdar and Crammer, 2009] Partha Pratim Talukdar and Koby Crammer. New regularized algorithms for transduc- tive learning. In Machine Learning and Knowledge Dis- covery in Databases, pages 442-457. Springer, 2009.","title":"New regularized algorithms for transductive learning","authors":"Partha Pratim, Talukdar, Koby Crammer","year":"2009","venue":"Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Wang et al., 2013] William Yang Wang, Kathryn Mazaitis, and William W Cohen. Programming with personalized pagerank: a locally groundable first-order probabilistic logic. In Proceedings of the 22nd ACM international con- ference on Conference on information & knowledge man- agement, pages 2129-2138. ACM, 2013.","title":"Programming with personalized pagerank: a locally groundable first-order probabilistic logic","authors":"William Yang, Wang, Kathryn Mazaitis, William W Cohen","year":"2013","venue":"Proceedings of the 22nd ACM international conference on Conference on information & knowledge management","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Weston et al., 2012] Jason Weston, Frédéric Ratle, Hossein Mobahi, and Ronan Collobert. Deep learning via semi- supervised embedding. In Neural Networks: Tricks of the Trade -Second Edition, pages 639-655. 2012.","title":"Deep learning via semisupervised embedding","authors":"Jason Weston, Frédéric Ratle, Hossein Mobahi, Ronan Collobert","year":"2012","venue":"Neural Networks: Tricks of the Trade -Second Edition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Zhou et al., 2003] Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. In Advances in Neural Information Processing Systems 16 [Neural Information Processing Systems, NIPS 2003, December 8-13, 2003, Vancouver and Whistler, British Columbia, Canada], pages 321-328, 2003.","title":"Learning with local and global consistency","authors":"Zhou","year":"2003","venue":"Advances in Neural Information Processing Systems 16 [Neural Information Processing Systems, NIPS 2003","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhu et al., 2003] X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using Gaussian fields and har- monic functions. In Proceedings of ICML-03, the 20th International Conference on Machine Learning, 2003.","title":"Semi-supervised learning using Gaussian fields and harmonic functions","authors":"X Zhu, Z Ghahramani, J Lafferty","year":"2003","venue":"Proceedings of ICML-03, the 20th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zhu, 2005] Xiaojin Zhu. Semi-supervised learning litera- ture survey. 2005.","title":"Semi-supervised learning literature survey","authors":"Xiaojin Zhu","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}