{"type":"ScholarlyArticle","title":"Unbiased Learning-to-Rank with Biased Feedback","authors":[{"name":"Thorsten Joachims","slug":"thorsten-joachims","given":"Thorsten","family":"Joachims","affiliation":"Cornell University , Ithaca , NY"},{"name":"Adith Swaminathan","slug":"adith-swaminathan","given":"Adith","family":"Swaminathan","affiliation":"Microsoft Research , Redmond , WA"},{"name":"Tobias Schnabel","slug":"tobias-schnabel","given":"Tobias","family":"Schnabel","affiliation":"Cornell University , Ithaca , NY"}],"abstract":"Implicit feedback (e.g., clicks, dwell times, etc.) is an abundant source of data in human-interactive systems. While implicit feedback has many advantages (e.g., it is inexpensive to collect, user-centric, and timely), its inherent biases are a key obstacle to its effective use. For example, position bias in search rankings strongly influences how many clicks a result receives, so that directly using click data as a training signal in Learning-to-Rank (LTR) methods yields \r\nsub-optimal results. To overcome this bias problem, we present a counterfactual inference framework that provides the theoretical basis for unbiased LTR via Empirical Risk Minimization despite biased data.\r\nUsing this framework, we derive a propensity-weighted ranking SVM for discriminative learning from implicit feedback, where click models take the role of the propensity estimator. \r\nBeyond the theoretical support, we show empirically that the proposed learning method is highly effective in dealing with biases, that it is robust to noise and propensity model mis-specification, and that it scales efficiently. We also demonstrate the real-world applicability of our approach on an operational search engine, where it substantially improves retrieval performance.","doi":"10.24963/ijcai.2018/738","url":"https://ijcai.org/proceedings/2018/738","pdf":"https://ijcai.org/proceedings/2018/0738.pdf","markdown":"https://ijcai.org/proceedings/2018/738.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/738","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Sister Conferences Best Papers","track":"Best Sister Conferences","number":738,"pages":{"first":5284,"last":5288},"keywords":["Multidisciplinary Topics and Applications: Information Retrieval","Machine Learning: Machine Learning","Natural Language Processing: Information Retrieval"],"videos":[],"references":[{"position":1,"raw":"[Chapelle et al., 2012] Oliver Chapelle, Thorsten Joachims, Filip Radlinski, and Yisong Yue. Large-scale validation and analysis of interleaved search evaluation. ACM Trans- actions on Information Systems (TOIS), 30(1):6:1-6:41, 2012.","title":"Large-scale validation and analysis of interleaved search evaluation","authors":"Oliver Chapelle, Thorsten Joachims, Filip Radlinski, Yisong Yue","year":"2012","venue":"ACM Transactions on Information Systems (TOIS)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chuklin et al., 2015] Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. Click Models for Web Search. Syn- thesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers, 2015.","title":"Click Models for Web Search","authors":"Aleksandr Chuklin, Ilya Markov, Maarten De Rijke","year":"2015","venue":"Synthesis Lectures on Information Concepts","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hofmann et al., 2013] Katja Hofmann, Anne Schuth, Shi- mon Whiteson, and Maarten de Rijke. Reusing historical interaction data for faster online learning to rank for IR. In International Conference on Web Search and Data Mining (WSDM), pages 183-192, 2013.","title":"Reusing historical interaction data for faster online learning to rank for IR","authors":"Katja Hofmann, Anne Schuth, Shimon Whiteson, Maarten De Rijke","year":"2013","venue":"International Conference on Web Search and Data Mining (WSDM)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Horvitz and Thompson, 1952] Daniel Horvitz and Donovan Thompson. A generalization of sampling without replace- ment from a finite universe. Journal of the American Sta- tistical Association, 47(260):663-685, 1952.","title":"A generalization of sampling without replacement from a finite universe","authors":"Thompson ; Horvitz, Donovan Horvitz, Thompson","year":"1952","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Imbens and Rubin, 2015] Guido Imbens and Donald Rubin. Causal Inference for Statistics, Social, and Biomedical Sciences. Cambridge University Press, 2015.","title":"Causal Inference for Statistics, Social, and Biomedical Sciences","authors":"Rubin ; Imbens, Guido Imbens, Donald Rubin","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Joachims et al., 2007] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, and Geri Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Transactions on Information Systems (TOIS), 25(2), April 2007.","title":"Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search","authors":"Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Filip Radlinski, Geri Gay","year":"2007","venue":"ACM Transactions on Information Systems (TOIS)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Joachims et al., 2017] Thorsten Joachims, Adith Swami- nathan, and Tobias Schnabel. Unbiased learning-to-rank with biased feedback. In ACM Conference on Web Search and Data Mining (WSDM), pages 781-789, 2017.","title":"Unbiased learning-to-rank with biased feedback","authors":"Thorsten Joachims, Adith Swaminathan, Tobias Schnabel","year":"2017","venue":"ACM Conference on Web Search and Data Mining (WSDM)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Joachims, 2002] Thorsten Joachims. Optimizing search en- gines using clickthrough data. In ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining (KDD), pages 133-142, 2002.","title":"Optimizing search engines using clickthrough data","authors":"Thorsten Joachims","year":"2002","venue":"ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Joachims, 2006] Thorsten Joachims. Training linear SVMs in linear time. In ACM SIGKDD International Conference On Knowledge Discovery and Data Mining (KDD), pages 217-226, 2006.","title":"Training linear SVMs in linear time","authors":"Thorsten Joachims","year":"2006","venue":"ACM SIGKDD International Conference On Knowledge Discovery and Data Mining (KDD)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Little and Rubin, 2002] Roderick J. A. Little and Donald B. Rubin. Statistical Analysis with Missing Data. John Wiley, 2002.","title":"Statistical Analysis with Missing Data","authors":"J A Roderick, Donald B Little, Rubin","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu, 2009] Tie-Yan Liu. Learning to rank for information retrieval. Foundations and Trends in Information Re- trieval, 3(3):225-331, March 2009.","title":"Learning to rank for information retrieval","authors":"Tie-Yan Liu, Liu","year":"2009","venue":"Foundations and Trends in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Raman et al., 2013] Karthik Raman, Thorsten Joachims, P. Shivaswamy, and Tobias Schnabel. Stable coactive learning via perturbation. In International Conference on Machine Learning (ICML), pages 837-845, 2013.","title":"Stable coactive learning via perturbation","authors":"Karthik Raman, Thorsten Joachims, P Shivaswamy, Tobias Schnabel","year":"2013","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Richardson et al., 2007] Matthew Richardson, Ewa Domi- nowska, and Robert Ragno. Predicting clicks: Estimat- ing the click-through rate for new ads. In International Conference on World Wide Web (WWW), pages 521-530. ACM, 2007.","title":"Predicting clicks: Estimating the click-through rate for new ads","authors":"Matthew Richardson, Ewa Dominowska, Robert Ragno","year":"2007","venue":"International Conference on World Wide Web (WWW)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rosenbaum and Rubin, 1983] Paul R. Rosenbaum and Donald B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.","title":"The central role of the propensity score in observational studies for causal effects","authors":"Paul R Rosenbaum, Donald B Rubin","year":"1983","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Schnabel et al., 2016] Tobias Schnabel, Adith Swami- nathan, Peter Frazier, and Thorsten Joachims. Unbiased comparative evaluation of ranking functions. In ACM International Conference on the Theory of Information Retrieval (ICTIR), pages 109-118, 2016.","title":"Unbiased comparative evaluation of ranking functions","authors":"Schnabel","year":"2016","venue":"ACM International Conference on the Theory of Information Retrieval (ICTIR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sparck-Jones and van Rijsbergen, 1975] Karen Sparck- Jones and Cornelis J. van Rijsbergen. Report on the need for and provision of an \"ideal\" information retrieval test collection. Technical report, University of Cambridge, 1975.","title":"Report on the need for and provision of an \"ideal\" information retrieval test collection","authors":"Sparck-Jones And Van Rijsbergen ; Karen, Sparck-Jones, Cornelis J Van Rijsbergen","year":"1975","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Swaminathan and Joachims, 2015] Adith Swaminathan and Thorsten Joachims. Batch learning from logged bandit feedback through counterfactual risk minimization. Jour- nal of Machine Learning Research (JMLR), 16:1731- 1755, Sep 2015. Special Issue in Memory of Alexey Cher- vonenkis.","title":"Batch learning from logged bandit feedback through counterfactual risk minimization","authors":"Joachims Swaminathan, Adith Swaminathan, Thorsten Joachims","year":"2015","venue":"Journal of Machine Learning Research (JMLR)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vapnik, 1998] Vladimir Vapnik. Statistical Learning The- ory. Wiley, Chichester, GB, 1998.","title":"Statistical Learning Theory","authors":"Vladimir Vapnik","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2011] Lidan Wang, Jimmy J. Lin, and Donald Metzler. A cascade ranking model for efficient ranked re- trieval. In ACM Conference on Research and Development in Information Retrieval (SIGIR), pages 105-114, 2011.","title":"A cascade ranking model for efficient ranked retrieval","authors":"Lidan Wang, Jimmy J Lin, Donald Metzler","year":"2011","venue":"ACM Conference on Research and Development in Information Retrieval (SIGIR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2016] Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork. Learning to rank with selection bias in personal search. In ACM Conference on Research and Development in Information Retrieval (SI- GIR). ACM, 2016.","title":"Learning to rank with selection bias in personal search","authors":"Wang","year":"2016","venue":"ACM Conference on Research and Development in Information Retrieval (SI-GIR)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2018] Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc Najork. Position bias estimation for unbiased learning to rank in personal search. In Conference on Web Search and Data Mining (WSDM), 2018.","title":"Position bias estimation for unbiased learning to rank in personal search","authors":"Wang","year":"2018","venue":"Conference on Web Search and Data Mining (WSDM)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}