{"type":"ScholarlyArticle","title":"Boosting for Comparison-Based Learning","authors":[{"name":"Michael Perrot","slug":"michael-perrot","given":"Michael","family":"Perrot","affiliation":"Max-Planck-Institute for Intelligent Systems , Tübingen , Germany"},{"name":"Ulrike von Luxburg","slug":"ulrike-von-luxburg","given":"Ulrike","family":"von Luxburg","affiliation":"Max-Planck-Institute for Intelligent Systems , Tübingen , Germany; University of Tübingen , Department of Computer Science , Tübingen , Germany"}],"abstract":"We consider the problem of classification in a comparison-based setting: given a set of objects, we only have access to triplet comparisons of the form ``object A is closer to object B than to object C.'' In this paper we introduce TripletBoost, a new method that can learn a classifier just from such triplet comparisons. The main idea is to aggregate the triplets information into weak classifiers, which can subsequently be boosted to a strong classifier. Our method has two main advantages: (i) it is applicable to data from any metric space, and (ii) it can deal with large scale problems using only passively obtained and noisy triplets. We derive theoretical generalization guarantees and a lower bound on the number of necessary triplets, and we empirically show that our method is both competitive with state of the art approaches and resistant to noise.","doi":"10.24963/ijcai.2019/255","url":"https://ijcai.org/proceedings/2019/255","pdf":"https://ijcai.org/proceedings/2019/0255.pdf","markdown":"https://ijcai.org/proceedings/2019/255.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/255","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Knowledge Representation and Reasoning","track":"Main track","number":255,"pages":{"first":1844,"last":1850},"keywords":["Humans and AI: Human Computation and Crowdsourcing","Knowledge Representation and Reasoning: Preference Modelling and Preference-Based Reasoning"],"videos":[],"references":[{"position":1,"raw":"[Agarwal et al., 2007] Sameer Agarwal, Josh Wills, Lawrence Cayton, Gert Lanckriet, David Kriegman, and Serge Belongie. Generalized non-metric multidimensional scaling. In Artificial Intelligence and Statistics, pages 11-18, 2007.","title":"Generalized non-metric multidimensional scaling","authors":"Sameer Agarwal, Josh Wills, Lawrence Cayton, Gert Lanckriet, David Kriegman, Serge Belongie","year":"2007","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ailon, 2012] Nir Ailon. An active learning algorithm for ranking from pairwise preferences with an almost optimal query complexity. Journal of Machine Learning Research, 13(Jan):137-164, 2012.","title":"Nir Ailon. An active learning algorithm for ranking from pairwise preferences with an almost optimal query complexity","authors":"Ailon","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Amid and Ukkonen, 2015] Ehsan Amid and Antti Ukko- nen. Multiview triplet embedding: Learning attributes in multiple maps. In International Conference on Machine Learning, pages 1472-1480, 2015.","title":"Ehsan Amid and Antti Ukkonen. Multiview triplet embedding: Learning attributes in multiple maps","authors":"Ukkonen Amid","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bellet et al., 2015] Aurélien Bellet, Amaury Habrard, and Marc Sebban. Metric learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 9(1):1-151, 2015.","title":"Metric learning","authors":"Aurélien Bellet, Amaury Habrard, Marc Sebban","year":"2015","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Breiman, 1999] Leo Breiman. Prediction games and arcing algorithms. Neural computation, 11(7):1493-1517, 1999.","title":"Prediction games and arcing algorithms","authors":"Leo Breiman, Breiman","year":"1999","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Freund and Schapire, 1997] Yoav Freund and Robert E Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of com- puter and system sciences, 55(1):119-139, 1997.","title":"A decision-theoretic generalization of on-line learning and an application to boosting","authors":"Schapire Freund, Yoav Freund, Robert E Schapire","year":"1997","venue":"Journal of computer and system sciences","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gao and Zhou, 2013] Wei Gao and Zhi-Hua Zhou. On the doubt about margin explanation of boosting. Artificial In- telligence, 203:1-18, 2013.","title":"On the doubt about margin explanation of boosting","authors":"Wei Gao, Zhi-Hua Zhou","year":"2013","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Haghiri et al., 2017] Siavash Haghiri, Debarghya Ghosh- dastidar, and Ulrike von Luxburg. Comparison-based nearest neighbor search. In Artificial Intelligence and Statistics, pages 851-859, 2017.","title":"Comparison-based nearest neighbor search","authors":"Siavash Haghiri, Debarghya Ghoshdastidar, Ulrike Von, Luxburg","year":"2017","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Harper and Konstan, 2016] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems, 5(4):19, 2016.","title":"The movielens datasets: History and context","authors":"Maxwell Harper, Joseph A Konstan","year":"2016","venue":"ACM Transactions on Interactive Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hastie et al., 2009] Trevor Hastie, Saharon Rosset, Ji Zhu, and Hui Zou. Multi-class adaboost. Statistics and its In- terface, 2(3):349-360, 2009.","title":"Multi-class adaboost","authors":"Trevor Hastie, Saharon Rosset, Ji Zhu, Hui Zou","year":"2009","venue":"Statistics and its Interface","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Heikinheimo and Ukkonen, 2013] Hannes Heikinheimo and Antti Ukkonen. The crowd-median algorithm. In First AAAI Conference on Human Computation and Crowdsourcing, 2013.","title":"The crowd-median algorithm","authors":"Ukkonen Heikinheimo, Hannes Heikinheimo, Antti Ukkonen","year":"2013","venue":"First AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jain et al., 2016] Lalit Jain, Kevin G Jamieson, and Rob Nowak. Finite sample prediction and recovery bounds for ordinal embedding. In Neural Information Processing Sys- tems, pages 2711-2719, 2016.","title":"Finite sample prediction and recovery bounds for ordinal embedding","authors":"Lalit Jain, Kevin G Jamieson, Rob Nowak","year":"2016","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jamieson and Nowak, 2011] Kevin G Jamieson and Robert D Nowak. Low-dimensional embedding using adaptively selected ordinal data. In Conference on Com- munication, Control, and Computing, pages 1077-1084, 2011.","title":"Low-dimensional embedding using adaptively selected ordinal data","authors":"G Kevin, Robert D Jamieson, Nowak","year":"2011","venue":"Conference on Communication, Control, and Computing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kane et al., 2017] Daniel M Kane, Shachar Lovett, Shay Moran, and Jiapeng Zhang. Active classification with comparison queries. In Foundations of Computer Science, pages 355-366, 2017.","title":"Active classification with comparison queries","authors":"Shachar Daniel M Kane, Shay Lovett, Jiapeng Moran, Zhang","year":"2017","venue":"Foundations of Computer Science","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kazemi et al., 2018] Ehsan Kazemi, Lin Chen, Sanjoy Das- gupta, and Amin Karbasi. Comparison based learning from weak oracles. arXiv preprint arXiv:1802.06942v1, 2018.","title":"Comparison based learning from weak oracles","authors":"Ehsan Kazemi, Lin Chen, Sanjoy Dasgupta, Amin Karbasi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kleindessner and Luxburg, 2015] Matthäus Kleindessner and Ulrike Luxburg. Dimensionality estimation without distances. In Artificial Intelligence and Statistics, pages 471-479, 2015.","title":"Matthäus Kleindessner and Ulrike Luxburg. Dimensionality estimation without distances","authors":"Luxburg Kleindessner","year":"2015","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pedregosa et al., 2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blon- del, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.","title":"Scikit-learn: Machine learning in Python","authors":"F Pedregosa, G Varoquaux, A Gramfort, V Michel, B Thirion, O Grisel, M Blondel, P Prettenhofer, R Weiss, V Dubourg, J Vanderplas, A Passos, D Cournapeau, M Brucher, M Perrot, E Duchesnay","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Perrot and von Luxburg, 2018] Michaël Perrot and Ulrike von Luxburg. Boosting for comparison-based learning. arXiv preprint arXiv:1810.13333, 2018.","title":"Boosting for comparison-based learning","authors":"Von Luxburg ; Michaël Perrot, Ulrike Perrot, Luxburg Von","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schapire and Freund, 2012] Robert E Schapire and Yoav Freund. Boosting: Foundations and algorithms. MIT press, 2012.","title":"Boosting: Foundations and algorithms","authors":"E Robert, Yoav Schapire, Freund","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Schapire and Singer, 1999] Robert E Schapire and Yoram Singer. Improved boosting algorithms using confidence- rated predictions. Machine learning, 37(3):297-336, 1999.","title":"Improved boosting algorithms using confidencerated predictions","authors":"E Robert, Yoram Schapire, Singer","year":"1999","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schapire and Singer, 2000] Robert E Schapire and Yoram Singer. Boostexter: A boosting-based system for text cat- egorization. Machine learning, 39(2-3):135-168, 2000.","title":"Boostexter: A boosting-based system for text categorization","authors":"E Robert, Yoram Schapire, Singer","year":"2000","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schapire et al., 1998] Robert E Schapire, Yoav Freund, Pe- ter Bartlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of statistics, pages 1651-1686, 1998.","title":"Boosting the margin: A new explanation for the effectiveness of voting methods","authors":"Yoav Robert E Schapire, Peter Freund, Wee Bartlett, Lee Sun","year":"1998","venue":"Annals of statistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tamuz et al., 2011] Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, and Adam Tauman Kalai. Adaptively learn- ing the crowd kernel. In International Conference on Ma- chine Learning, pages 673-680, 2011.","title":"Adaptively learning the crowd kernel","authors":"Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, Adam Tauman, Kalai","year":"2011","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Terada and von Luxburg, 2014] Yoshikazu Terada and Ul- rike von Luxburg. Local ordinal embedding. In Interna- tional Conference on Machine Learning, pages 847-855, 2014.","title":"Yoshikazu Terada and Ulrike von Luxburg","authors":"Von Terada, Luxburg","year":"2014","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tschopp et al., 2011] Dominique Tschopp, Suhas Diggavi, Payam Delgosha, and Soheil Mohajer. Randomized algo- rithms for comparison-based search. In Neural Informa- tion Processing Systems, pages 2231-2239, 2011.","title":"Randomized algorithms for comparison-based search","authors":"Dominique Tschopp, Suhas Diggavi, Payam Delgosha, Soheil Mohajer","year":"2011","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Van Der Maaten and Weinberger, 2012] Laurens Van Der Maaten and Kilian Weinberger. Stochastic triplet embedding. In Machine Learning for Signal Processing, pages 1-6, 2012.","title":"Stochastic triplet embedding","authors":"Van Der Maaten, Laurens Weinberger, Der Van, Kilian Maaten, Weinberger","year":"2012","venue":"Machine Learning for Signal Processing","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2011] Liwei Wang, Masashi Sugiyama, Zhaox- iang Jing, Cheng Yang, Zhi-Hua Zhou, and Jufu Feng. A refined margin analysis for boosting algorithms via equi- librium margin. Journal of Machine Learning Research, 12(Jun):1835-1863, 2011.","title":"A refined margin analysis for boosting algorithms via equilibrium margin","authors":"Liwei Wang, Masashi Sugiyama, Zhaoxiang Jing, Cheng Yang, Zhi-Hua Zhou, Jufu Feng","year":"2011","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}