{"type":"ScholarlyArticle","title":"Textual Membership Queries","authors":[{"name":"Jonathan Zarecki","slug":"jonathan-zarecki","given":"Jonathan","family":"Zarecki","affiliation":"Department of Computer Science , Technion -Israel Institute of Technology"},{"name":"Shaul Markovitch","slug":"shaul-markovitch","given":"Shaul","family":"Markovitch","affiliation":"Department of Computer Science , Technion -Israel Institute of Technology"}],"abstract":"Human labeling of data can be very time-consuming and expensive, yet, in many cases it is critical for the success of the learning process.\r\nIn order to minimize human labeling efforts, we propose a novel active learning solution that does not rely on existing sources of unlabeled data. It uses a small amount of labeled data as the core set for the synthesis of useful membership queries (MQs) — unlabeled instances generated by an algorithm for human labeling.\r\n\r\nOur solution uses modification operators, functions that modify instances to some extent. We apply the operators on a small set of instances (core set), creating a set of new membership queries.\r\nUsing this framework, we look at the instance space as a search space and apply search algorithms in order to generate new examples highly relevant to the learner.\r\nWe implement this framework in the textual domain and test it on several text classification tasks and show improved classifier performance as more MQs are labeled and incorporated into the training set.\r\nTo the best of our knowledge, this is the first work on membership queries in the textual domain.","doi":"10.24963/ijcai.2020/369","url":"https://ijcai.org/proceedings/2020/369","pdf":"https://ijcai.org/proceedings/2020/0369.pdf","markdown":"https://ijcai.org/proceedings/2020/369.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/369","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":369,"pages":{"first":2662,"last":2668},"keywords":["Heuristic Search and Game Playing: Heuristic Search and Machine Learning","Natural Language Processing: Natural Language Processing","Machine Learning: Active Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25473/file_25473.m3u8","length_sec":260,"provider":"hls"}],"references":[{"position":1,"raw":"[Angluin, 1987] Dana Angluin. Learning Regular Sets from Queries and Counterexamples. Inf. Comput., 75(2):87- 106, 1987.","title":"Dana Angluin. Learning Regular Sets from Queries and Counterexamples","authors":"Angluin","year":"1987","venue":"Inf. Comput","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Angluin, 1988] Dana Angluin. Queries and Concept Learn- ing. Machine Learning, 2(4):319-342, 1988.","title":"Dana Angluin. Queries and Concept Learning","authors":"Angluin","year":"1988","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Atlas et al., 1990] Les E Atlas, David A Cohn, and Richard E Ladner. Training Connectionist Networks with Queries and Selective Sampling. In D S Touretzky, edi- tor, Advances in Neural Information Processing Systems 2, pages 566-573.","title":"Training Connectionist Networks with Queries and Selective Sampling","authors":"Atlas","year":"1990","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Awasthi et al., 2013] Pranjal Awasthi, Vitaly Feldman, and Varun Kanade. Learning using local membership queries. COLT, 30:1-34, 2013.","title":"Learning using local membership queries","authors":"Pranjal Awasthi, Vitaly Feldman, Varun Kanade","year":"2013","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bary, 2015] Galit Bary. Learning Using 1-Local Member- ship Queries. PhD thesis, The Hebrew University of Jerusalem, 2015.","title":"Learning Using 1-Local Membership Queries","authors":"Bary","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bshouty, 1995] N H Bshouty. Exact Learning Boolean Functions via the Monotone Theory. Information and Computation, 123(1):146-153, 1995.","title":"Exact Learning Boolean Functions via the Monotone Theory","authors":"N H Bshouty","year":"1995","venue":"Information and Computation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chawla et al., 2002] Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321-357, 2002.","title":"SMOTE: Synthetic minority over-sampling technique","authors":"V Nitesh, Kevin W Chawla, Lawrence O Bowyer, W Philip Hall, Kegelmeyer","year":"2002","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Davidson et al., 2017] Thomas Davidson, Dana Warmsley, Michael W. Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language. In ICWSM, 2017.","title":"Automated hate speech detection and the problem of offensive language","authors":"Thomas Davidson, Dana Warmsley, Michael W Macy, Ingmar Weber","year":"2017","venue":"ICWSM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Druck et al., 2009] Gregory Druck, Burr Settles, and An- drew McCallum. Active Learning by Labeling Features. EMNLP '09, pages 81-90, Stroudsburg, PA, USA, 2009. Association for Computational Linguistics.","title":"Active Learning by Labeling Features","authors":"Gregory Druck, Burr Settles, Andrew Mccallum","year":"2009","venue":"EMNLP '09","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gabrilovich and Markovitch, 2007] Evgeniy Gabrilovich and Shaul Markovitch. Computing semantic relatedness using wikipedia-based explicit semantic analysis. In IJCAI, 2007.","title":"Computing semantic relatedness using wikipedia-based explicit semantic analysis","authors":"Evgeniy Gabrilovich, Shaul Markovitch","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/259"},{"position":12,"raw":"[Gurevich et al., 2006] Nela Gurevich, Shaul Markovitch, and Ehud Rivlin. Active learning with near misses. In AAAI, 2006.","title":"Active learning with near misses","authors":"Nela Gurevich, Shaul Markovitch, Ehud Rivlin","year":"2006","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Harris, 1954] Zellig S. Harris. Distributional Structure. Word, 10(2-3):146-162, 1954.","title":"Distributional Structure","authors":"S Zellig, Harris","year":"1954","venue":"Word","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9:1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Houlsby et al., 2011] Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learn- ing for classification and preference learning. ArXiv, abs/1112.5745, 2011.","title":"Bayesian active learning for classification and preference learning","authors":"Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, Máté Lengyel","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Iacobacci et al., 2015] Ignacio Iacobacci, Moham- mad Taher Pilehvar, and Roberto Navigli. Sensembed: Learning sense embeddings for word and relational similarity. volume 1 of EMNLP '09, pages 95-105, 2015.","title":"Ignacio Iacobacci, Mohammad Taher Pilehvar, and Roberto Navigli. Sensembed: Learning sense embeddings for word and relational similarity","authors":"Iacobacci","year":"2015","venue":"EMNLP '09","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lang and Baum, 1992] Kevin J. Lang and Eric B Baum. Query Learning Can Work Poorly when a Human Oracle is Used. In IJCNN 1992, pages 335-340, 1992.","title":"Query Learning Can Work Poorly when a Human Oracle is Used","authors":"Kevin J Lang, Eric B Baum","year":"1992","venue":"IJCNN 1992","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Levy and Goldberg, 2014] Omer Levy and Yoav Goldberg. Dependency-based word embeddings. In ACL, 2014.","title":"Dependency-based word embeddings","authors":"Goldberg Levy, Omer Levy, Yoav Goldberg","year":"2014","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lewis and Gale, 1994] David D Lewis and William A Gale. A Sequential Algorithm For Training Text Classifiers. Pro- ceedings of the 17th International Conference on Research and Development in Information Retrieval (SIGIR'94), 94:3-12, 1994.","title":"A Sequential Algorithm For Training Text Classifiers","authors":"D David, William A Lewis, Gale","year":"1994","venue":"Proceedings of the 17th International Conference on Research and Development in Information Retrieval (SIGIR'94)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lindenbaum et al., 1999] Michael Lindenbaum, Shaul Markovitch, and Dmitry Rusakov. Selective sampling for nearest neighbor classifiers. Machine Learning, 54:125-152, 1999.","title":"Selective sampling for nearest neighbor classifiers","authors":"Michael Lindenbaum, Shaul Markovitch, Dmitry Rusakov","year":"1999","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their composi- tionality. In NIPS, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pang and Lee, 2004] Bo Pang and Lillian Lee. A Senti- mental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts. In ACL, 2004.","title":"A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2004","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pang and Lee, 2005] Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the ACL, 2005.","title":"Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales","authors":"Lee ; Pang, Bo Pang, Lillian Lee","year":"2005","venue":"Proceedings of the ACL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1532-1543, 2014.","title":"GloVe: Global Vectors for Word Representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Radford et al., 2017] Alec Radford, Rafal Józefowicz, and Ilya Sutskever. Learning to Generate Reviews and Dis- covering Sentiment. CoRR, abs/1704.0, 2017.","title":"Learning to Generate Reviews and Discovering Sentiment","authors":"Radford","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rosario, 2017] Ryan Rosario. A Data Augmentation Ap- proach to Short Text Classification. PhD thesis, University of California, Los Angeles, 2017.","title":"A Data Augmentation Approach to Short Text Classification","authors":"Ryan Rosario","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Socher et al., 2013] Richard Socher, Alex Perelygin, and Jy Wu. Recursive deep models for semantic composition- ality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, 2013.","title":"Recursive deep models for semantic compositionality over a sentiment treebank","authors":"Richard Socher, Alex Perelygin, Jy Wu","year":"2013","venue":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Turney and Pantel, 2010] Peter D. Turney and Patrick Pan- tel. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37:141-188, 2010.","title":"From frequency to meaning: Vector space models of semantics","authors":"D Peter, Patrick Turney, Pantel","year":"2010","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Valiant, 1984] L. G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134-1142, 11 1984.","title":"A theory of the learnable","authors":"L G Valiant","year":"1984","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang and LeCun, 2015] Xiang Zhang and Yann LeCun. Text Understanding from Scratch. CoRR, abs/1502.0, 2015.","title":"Text Understanding from Scratch","authors":"Xiang Zhang, Yann Lecun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}