{"type":"ScholarlyArticle","title":"Active Learning for Black-Box Semantic Role Labeling with Neural Factors","authors":[{"name":"Chenguang Wang","slug":"chenguang-wang","given":"Chenguang","family":"Wang","affiliation":"IBM Research -Almaden"},{"name":"Laura Chiticariu","slug":"laura-chiticariu","given":"Laura","family":"Chiticariu","affiliation":"IBM Research -Almaden"},{"name":"Yunyao Li","slug":"yunyao-li","given":"Yunyao","family":"Li","affiliation":"IBM Research -Almaden"}],"abstract":"Active learning is a useful technique for tasks for which unlabeled data is abundant but manual labeling is expensive. One example of such a task is semantic role labeling (SRL), which relies heavily on labels from trained linguistic experts. One challenge in applying active learning algorithms for SRL is that the complete knowledge of the SRL model is often unavailable, against the common assumption that active learning methods are aware of the details of the underlying models. In this paper, we present an active learning framework for black-box SRL models (i.e., models whose details are unknown). In lieu of a query strategy based on model details, we propose a neural query strategy model that embeds both language and semantic information to automatically learn the query strategy from predictions of an SRL model alone. Our experimental results demonstrate the effectiveness of both this new active learning framework and the neural query strategy model.","doi":"10.24963/ijcai.2017/405","url":"https://ijcai.org/proceedings/2017/405","pdf":"https://ijcai.org/proceedings/2017/0405.pdf","markdown":"https://ijcai.org/proceedings/2017/405.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/405","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":405,"pages":{"first":2908,"last":2914},"keywords":["Natural Language Processing: Tagging, chunking, syntax, and parsing","Machine Learning: Active Learning","Natural Language Processing: Natural Language Semantics","Machine Learning: Neural Networks"],"videos":[],"references":[{"position":1,"raw":"[Akbik and Li, 2016] Alan Akbik and Yunyao Li. K-srl: Instance- based learning for semantic role labeling. In COLING, pages 599-608, 2016.","title":"Yunyao Li. K-srl: Instancebased learning for semantic role labeling","authors":"Li Akbik, Alan Akbik","year":"2016","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Björkelund et al., 2010] Anders Björkelund, Bernd Bohnet, Love Hafdell, and Pierre Nugues. A high-performance syntactic and semantic dependency parser. In COLING, pages 33-36, 2010.","title":"A high-performance syntactic and semantic dependency parser","authors":"Anders Björkelund, Bernd Bohnet, Love Hafdell, Pierre Nugues","year":"2010","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bordes et al., 2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Trans- lating embeddings for modeling multi-relational data. In NIPS, pages 2787-2795, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, Oksana Yakhnenko","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown et al., 1992] Peter F Brown, Peter V Desouza, Robert L Mercer, Vincent J Della Pietra, and Jenifer C Lai. Class-based n-gram models of natural language. CL, 18(4):467-479, 1992.","title":"Class-based n-gram models of natural language","authors":"Peter V Peter F Brown, Robert L Desouza, Vincent J Mercer, Pietra Della, Jenifer C Lai","year":"1992","venue":"CL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Charniak, 1997] Eugene Charniak. Statistical parsing with a context-free grammar and word statistics. AAAI/IAAI, page 18, 1997.","title":"Statistical parsing with a context-free grammar and word statistics","authors":"Eugene Charniak, Charniak","year":"1997","venue":"AAAI/IAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi and Palmer, 2011] Jinho D Choi and Martha Palmer. Transition-based semantic role labeling using predicate ar- gument clustering. In ACL 2011 (Workshop), pages 37-45, 2011.","title":"Transition-based semantic role labeling using predicate argument clustering","authors":"D Jinho, Martha Choi, Palmer","year":"2011","venue":"ACL 2011 (Workshop)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Clark et al., 2003] Stephen Clark, James R Curran, and Miles Os- borne. Bootstrapping pos taggers using unlabelled data. In HLT- NAACL, pages 49-55, 2003.","title":"Bootstrapping pos taggers using unlabelled data","authors":"Stephen Clark, James R Curran, Miles Osborne","year":"2003","venue":"HLT-NAACL","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Collobert et al., 2011] Ronan Collobert, Jason Weston, Léon Bot- tou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. JMLR, 12(Aug):2493-2537, 2011.","title":"Natural language processing (almost) from scratch","authors":"Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa","year":"2011","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Culotta and McCallum, 2005] Aron Culotta and Andrew McCal- lum. Reducing labeling effort for structured prediction tasks. In AAAI, pages 746-51, 2005.","title":"Reducing labeling effort for structured prediction tasks","authors":"Mccallum Culotta, Aron Culotta, Andrew Mccallum","year":"2005","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dagan and Engelson, 1995] Ido Dagan and Sean P Engelson. Committee-based sampling for training probabilistic classifiers. In ICML, pages 150-157, 1995.","title":"Committee-based sampling for training probabilistic classifiers","authors":"Ido Dagan, Sean P Engelson","year":"1995","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dhillon et al., 2011] Paramveer Dhillon, Dean P Foster, and Lyle H Ungar. Multi-view learning of word embeddings via cca. In NIPS, pages 199-207, 2011.","title":"Multi-view learning of word embeddings via cca","authors":"Dhillon","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[FitzGerald et al., 2015] Nicholas FitzGerald, Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. Semantic role labeling with neural network factors. In EMNLP, pages 960-970, 2015.","title":"Semantic role labeling with neural network factors","authors":"Nicholas Fitzgerald, Oscar Täckström, Kuzman Ganchev, Dipanjan Das","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gildea and Jurafsky, 2002] Daniel Gildea and Daniel Jurafsky. Automatic labeling of semantic roles. CL, 28(3):245-288, 2002.","title":"Automatic labeling of semantic roles","authors":"Daniel Gildea, Daniel Jurafsky","year":"2002","venue":"CL","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hajič et al., 2009] Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martí, Lluís Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, et al. The conll-2009 shared task: Syntactic and se- mantic dependencies in multiple languages. In CoNLL, pages 1-18, 2009.","title":"The conll-2009 shared task: Syntactic and semantic dependencies in multiple languages","authors":"Hajič","year":"2009","venue":"CoNLL","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2012] Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng. Improving word representations via global context and multiple word prototypes. In ACL, pages 873-882, 2012.","title":"Improving word representations via global context and multiple word prototypes","authors":"Eric H Huang, Richard Socher, Christopher D Manning, Andrew Y Ng","year":"2012","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lebret and Collobert, 2013] Rémi Lebret and Ronan Collobert. Word emdeddings through hellinger pca. arXiv preprint arXiv:1312.5542, 2013.","title":"Word emdeddings through hellinger pca","authors":"Collobert Lebret, Rémi Lebret, Ronan Collobert","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lewis and Gale, 1994] David D Lewis and William A Gale. A se- quential algorithm for training text classifiers. In SIGIR, pages 3-12, 1994.","title":"A sequential algorithm for training text classifiers","authors":"D David, William A Lewis, Gale","year":"1994","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[McCallumzy and Nigamy, 1998] Andrew Kachites McCallumzy and Kamal Nigamy. Employing em and pool-based active learn- ing for text classification. In ICML, pages 359-367, 1998.","title":"Andrew Kachites McCallumzy and Kamal Nigamy. Employing em and pool-based active learning for text classification","authors":"Nigamy Mccallumzy","year":"1998","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Palangi et al., 2016] Hamid Palangi, Li Deng, Yelong Shen, Jian- feng Gao, Xiaodong He, Jianshu Chen, Xinying Song, and Rabab Ward. Deep sentence embedding using long short-term mem- ory networks: Analysis and application to information retrieval. TASLP, 24(4):694-707, 2016.","title":"Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval","authors":"Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, Rabab Ward","year":"2016","venue":"TASLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word repre- sentation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Recht et al., 2011] Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. Hogwild: A lock-free approach to par- allelizing stochastic gradient descent. In NIPS, pages 693-701, 2011.","title":"Hogwild: A lock-free approach to parallelizing stochastic gradient descent","authors":"Benjamin Recht, Christopher Re, Stephen Wright, Feng Niu","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Roth and Lapata, 2016] Michael Roth and Mirella Lapata. Neu- ral semantic role labeling with dependency path embeddings. In ACL, page to appear, 2016.","title":"Neural semantic role labeling with dependency path embeddings","authors":"Michael Roth, Mirella Lapata","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Roth and Woodsend, 2014] Michael Roth and Kristian Woodsend. Composition of word representations improves semantic role la- belling. In EMNLP, pages 407-413, 2014.","title":"Composition of word representations improves semantic role labelling","authors":"Michael Roth, Kristian Woodsend","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Scheffer et al., 2001] Tobias Scheffer, Christian Decomain, and Stefan Wrobel. Active hidden markov models for information extraction. In IDA, pages 309-318, 2001.","title":"Active hidden markov models for information extraction","authors":"Scheffer","year":"2001","venue":"IDA","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Settles, 2010] Burr Settles. Active learning literature survey. Uni- versity of Wisconsin, Madison, 52(55-66):11, 2010.","title":"Burr Settles. Active learning literature survey","authors":"Settles","year":"2010","venue":"University of Wisconsin","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Seung et al., 1992] H Sebastian Seung, Manfred Opper, and Haim Sompolinsky. Query by committee. In Annual workshop on Com- putational learning theory, pages 287-294, 1992.","title":"Query by committee","authors":"Sebastian Seung, Manfred Opper, Haim Sompolinsky","year":"1992","venue":"Annual workshop on Computational learning theory","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tai et al., 2015] Kai Sheng Tai, Richard Socher, and Christo- pher D. Manning. Improved semantic representations from tree- structured long short-term memory networks. In ACL, pages 1556-1566, 2015.","title":"Improved semantic representations from treestructured long short-term memory networks","authors":"Kai Sheng, Tai, Richard Socher, Christopher D Manning","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Thompson et al., 1999] Cynthia A Thompson, Mary Elaine Califf, and Raymond J Mooney. Active learning for natural language parsing and information extraction. In ICML, pages 406-414, 1999.","title":"Active learning for natural language parsing and information extraction","authors":"Cynthia A Thompson, Mary Elaine Califf, Raymond J Mooney","year":"1999","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2014] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph and text jointly embedding. In EMNLP, pages 1591-1601, 2014.","title":"Knowledge graph and text jointly embedding","authors":"Zhen Wang, Jianwen Zhang, Jianlin Feng, Zheng Chen","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2015a] Chenguang Wang, Yangqiu Song, Ahmed El- Kishky, Dan Roth, Ming Zhang, and Jiawei Han. Incorporating world knowledge to document clustering via heterogeneous in- formation networks. In KDD, pages 1215-1224, 2015.","title":"Incorporating world knowledge to document clustering via heterogeneous information networks","authors":"Wang","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2015b] Chenguang Wang, Yangqiu Song, Dan Roth, Chi Wang, Jiawei Han, Heng Ji, and Ming Zhang. Constrained information-theoretic tripartite graph clustering to identify se- mantically similar relations. In IJCAI, pages 3882-3889, 2015.","title":"Constrained information-theoretic tripartite graph clustering to identify semantically similar relations","authors":"Wang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/545"},{"position":33,"raw":"[Wang et al., 2016] Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, and Jiawei Han. Text classification with heteroge- neous information network kernels. In AAAI, pages 2130-2136, 2016.","title":"Text classification with heterogeneous information network kernels","authors":"Chenguang Wang, Yangqiu Song, Haoran Li, Ming Zhang, Jiawei Han","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhou and Xu, 2015] Jie Zhou and Wei Xu. End-to-end learning of semantic role labeling using recurrent neural networks. In ACL, pages 1127-1137, 2015.","title":"End-to-end learning of semantic role labeling using recurrent neural networks","authors":"Xu ; ] Jie Zhou, Wei Zhou, Xu","year":"2015","venue":"ACL","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}