{"type":"ScholarlyArticle","title":"Scaling Active Search using Linear Similarity Functions","authors":[{"name":"Sibi Venkatesan","slug":"sibi-venkatesan","given":"Sibi","family":"Venkatesan","affiliation":"Robotics Institute , Carnegie Mellon University , Pittsburgh , PA"},{"name":"James K. Miller","slug":"james-k-miller","given":"James K.","family":"Miller","affiliation":"Robotics Institute , Carnegie Mellon University , Pittsburgh , PA"},{"name":"Jeff Schneider","slug":"jeff-schneider","given":"Jeff","family":"Schneider","affiliation":"Robotics Institute , Carnegie Mellon University , Pittsburgh , PA"},{"name":"Artur Dubrawski","slug":"artur-dubrawski","given":"Artur","family":"Dubrawski","affiliation":"Robotics Institute , Carnegie Mellon University , Pittsburgh , PA"}],"abstract":"Active Search has become an increasingly useful tool in information retrieval problems where the goal is to discover as many target elements as possible using only limited label queries. With the advent of big data, there is a growing emphasis on the scalability of such techniques to handle very large and very complex datasets.\r\n\r\nIn this paper, we consider the problem of Active Search where we are given a similarity function between data points. We look at an algorithm introduced by Wang et al. [Wang et al., 2013] known as Active Search on Graphs and propose crucial modifications which allow it to scale significantly. Their approach selects points by minimizing an energy function over the graph induced by the similarity function on the data. Our modifications require the similarity function to be a dot-product between feature vectors of data points, equivalent to having a linear kernel for the adjacency matrix. With this, we are able to scale tremendously: for n data points, the original algorithm runs in O(n^2) time per iteration while ours runs in only O(nr + r^2) given r-dimensional features.\r\n\r\nWe also describe a simple alternate approach using a weighted-neighbor predictor which also scales well. In our experiments, we show that our method is competitive with existing semi-supervised approaches. We also briefly discuss conditions under which our algorithm performs well.","doi":"10.24963/ijcai.2017/401","url":"https://ijcai.org/proceedings/2017/401","pdf":"https://ijcai.org/proceedings/2017/0401.pdf","markdown":"https://ijcai.org/proceedings/2017/401.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/401","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":401,"pages":{"first":2878,"last":2884},"keywords":["Machine Learning: Semi-Supervised Learning","Machine Learning: Active Learning"],"videos":[],"references":[{"position":1,"raw":"[Cesa-Bianchi et al., 2013] Nicolo Cesa-Bianchi, Claudio Gentile, Fabio Vitale, and Giovanni Zappella. Ac- tive learning on trees and graphs. arXiv preprint arXiv:1301.5112, 2013.","title":"Active learning on trees and graphs","authors":"Cesa-Bianchi","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Fujiwara and Irie, 2014] Yasuhiro Fujiwara and Go Irie. Ef- ficient label propagation. In Proceedings of the 31st in- ternational conference on machine learning (ICML-14), pages 784-792, 2014.","title":"Efficient label propagation","authors":"Irie Fujiwara, Yasuhiro Fujiwara, Go Irie","year":"2014","venue":"Proceedings of the 31st international conference on machine learning (ICML-14)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Garnett et al., 2012] Roman Garnett, Yamuna Krishna- murthy, Xuehan Xiong, Jeff Schneider, and Richard Mann. Bayesian optimal active search and surveying. arXiv preprint arXiv:1206.6406, 2012.","title":"Roman Garnett, Yamuna Krishnamurthy","authors":"Garnett","year":"2012","venue":"Bayesian optimal active search and surveying","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Guillory and Bilmes, 2009] Andrew Guillory and Jeff A Bilmes. Label selection on graphs. In Advances in Neural Information Processing Systems, 2009.","title":"Label selection on graphs","authors":"Andrew Guillory, Jeff A Bilmes","year":"2009","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Kushnir, 2014] Dan Kushnir. Active-transductive learning with label-adapted kernels. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 462-471. ACM, 2014.","title":"Active-transductive learning with label-adapted kernels","authors":"Dan Kushnir, Kushnir","year":"2014","venue":"Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Liu et al., 2010] Wei Liu, Junfeng He, and Shih-Fu Chang. Large graph construction for scalable semi-supervised learning. In Proceedings of the 27th international con- ference on machine learning (ICML-10), pages 679-686, 2010.","title":"Large graph construction for scalable semi-supervised learning","authors":"Wei Liu, Junfeng He, Shih-Fu Chang","year":"2010","venue":"Proceedings of the 27th international conference on machine learning (ICML-10)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ma et al., 2013] Yifei Ma, Roman Garnett, and Jeff Schnei- der. σ-optimality for active learning on gaussian random fields. In Advances in Neural Information Processing Sys- tems, 2013.","title":"σ-optimality for active learning on gaussian random fields","authors":"Yifei Ma, Roman Garnett, Jeff Schneider","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ma et al., 2015] Yifei Ma, Tzu-Kuo Huang, and Jeff Schneider. Active search and bandits on graphs using sigma-optimality. 2015.","title":"Active search and bandits on graphs using sigma-optimality","authors":"Yifei Ma, Tzu-Kuo Huang, Jeff Schneider","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Melacci and Belkin, 2011] Stefano Melacci and Mikhail Belkin. Laplacian Support Vector Machines Trained in the Primal. Journal of Machine Learning Research, 12:1149- 1184, March 2011.","title":"Laplacian Support Vector Machines Trained in the Primal","authors":"Stefano Melacci, Mikhail Belkin","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Rahimi and Recht, 2007] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Ad- vances in neural information processing systems, pages 1177-1184, 2007.","title":"Random features for large-scale kernel machines","authors":"Recht Rahimi, Ali Rahimi, Benjamin Recht","year":"2007","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Wang et al., 2013] Xuezhi Wang, Roman Garnett, and Jeff Schneider. Active search on graphs. In Proceedings of the 19th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, KDD '13, 2013.","title":"Active search on graphs","authors":"Wang","year":"2013","venue":"Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '13","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Zhang et al., 2009] Kai Zhang, James T Kwok, and Bahram Parvin. Prototype vector machine for large scale semi- supervised learning. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 1233-1240. ACM, 2009.","title":"Prototype vector machine for large scale semisupervised learning","authors":"Kai Zhang, James T Kwok, Bahram Parvin","year":"2009","venue":"Proceedings of the 26th Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Zhu and Lafferty, 2005] Xiaojin Zhu and John Lafferty. Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi- supervised learning. In Proceedings of the 22nd interna- tional conference on Machine learning. ACM, 2005.","title":"Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semisupervised learning","authors":"Xiaojin Zhu, John Lafferty","year":"2005","venue":"Proceedings of the 22nd international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Zhu et al., 2003a] Xiaojin Zhu, Zoubin Ghahramani, John Lafferty, et al. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, 2003.","title":"Semi-supervised learning using gaussian fields and harmonic functions","authors":"Zhu","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Zhu et al., 2003b] Xiaojin Zhu, John Lafferty, and Zoubin Ghahramani. Combining active learning and semi- supervised learning using gaussian fields and harmonic functions. In ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, 2003.","title":"Combining active learning and semisupervised learning using gaussian fields and harmonic functions","authors":"Zhu","year":"2003","venue":"ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}