{"type":"ScholarlyArticle","title":"Learning with Sparse and Biased Feedback for Personal Search","authors":[{"name":"Michael Bendersky","slug":"michael-bendersky","given":"Michael","family":"Bendersky","affiliation":""},{"name":"Xuanhui Wang","slug":"xuanhui-wang","given":"Xuanhui","family":"Wang","affiliation":""},{"name":"Marc Najork","slug":"marc-najork","given":"Marc","family":"Najork","affiliation":""},{"name":"Donald Metzler","slug":"donald-metzler","given":"Donald","family":"Metzler","affiliation":""}],"abstract":"Personal search, including email, on-device, and personal media search, has recently attracted a considerable attention from the information retrieval community. In this paper, we provide an overview of challenges and opportunities of learning with implicit user feedback (e.g., click data) in personal search. Implicit user feedback provides a convenient source of supervision for ranking models in personal search. This feedback, however, has two major drawbacks: it is highly sparse and biased due to the personal nature of queries and documents. We demonstrate how these drawbacks can be overcome, and empirically demonstrate the benefits of learning with implicit feedback in the context of a large-scale email search engine.","doi":"10.24963/ijcai.2018/725","url":"https://ijcai.org/proceedings/2018/725","pdf":"https://ijcai.org/proceedings/2018/0725.pdf","markdown":"https://ijcai.org/proceedings/2018/725.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/725","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Sister Conferences Best Papers","track":"Best Sister Conferences","number":725,"pages":{"first":5219,"last":5223},"keywords":["Natural Language Processing: Information Retrieval","Humans and AI: Personalization and User Modeling","Multidisciplinary Topics and Applications: Information Retrieval"],"videos":[],"references":[{"position":1,"raw":"[Agarwal, 2014] Shalini Agarwal. Official Gmail Blog: A bit about Bundles in Inbox. https://gmail.googleblog.com/2014/11/a-bit-about- bundles-in-inbox.html, 2014.","title":"Official Gmail Blog: A bit about Bundles in Inbox","authors":"Shalini Agarwal","year":null,"venue":null,"doi":null,"url":"https://gmail.googleblog.com/2014/11/a-bit-about-bundles-in-inbox.html,2014","internal_url":null},{"position":2,"raw":"[Agichtein et al., 2006] Eugene Agichtein, Eric Brill, and Susan Dumais. Improving web search ranking by incorpo- rating user behavior information. In Proceedings of SIGIR, pages 19-26, 2006.","title":"Improving web search ranking by incorporating user behavior information","authors":"Eugene Agichtein, Eric Brill, Susan Dumais","year":"2006","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ailon et al., 2013] Nir Ailon, Zohar S Karnin, Edo Liberty, and Yoelle Maarek. Threading machine generated email. In Proceedings of WSDM, pages 405-414, 2013.","title":"Threading machine generated email","authors":"Nir Ailon, Edo Zohar S Karnin, Yoelle Liberty, Maarek","year":"2013","venue":"Proceedings of WSDM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Anguera et al., 2008] Xavier Anguera, JieJun Xu, and Nuria Oliver. Multimodal photo annotation and retrieval on a mobile phone. In Proceedings of the 1st ACM Inter- national Conference on Multimedia Information Retrieval (MIR), pages 188-194, 2008.","title":"Multimodal photo annotation and retrieval on a mobile phone","authors":"Xavier Anguera, Jiejun Xu, Nuria Oliver","year":"2008","venue":"Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval (MIR)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bendersky et al., 2017] Michael Bendersky, Xuanhui Wang, Donald Metzler, and Marc Najork. Learning from user interactions in personal search via attribute param- eterization. In Proceedings of WSDM, pages 791-799, 2017.","title":"Learning from user interactions in personal search via attribute parameterization","authors":"Michael Bendersky, Xuanhui Wang, Donald Metzler, Marc Najork","year":"2017","venue":"Proceedings of WSDM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Carmel et al., 2015] David Carmel, Guy Halawi, Liane Lewin-Eytan, Yoelle Maarek, and Ariel Raviv. Rank by time or by relevance?: Revisiting email search. In Pro- ceedings of CIKM, pages 283-292, 2015.","title":"Rank by time or by relevance?: Revisiting email search","authors":"David Carmel, Guy Halawi, Liane Lewin-Eytan, Yoelle Maarek, Ariel Raviv","year":"2015","venue":"Proceedings of CIKM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chuklin et al., 2015] Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. Click Models for Web Search. Morgan & Claypool, 2015.","title":"Click Models for Web Search","authors":"Aleksandr Chuklin, Ilya Markov, Maarten De Rijke","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dumais et al., 2003] Susan Dumais, Edward Cutrell, JJ Cadiz, Gavin Jancke, Raman Sarin, and Daniel C. Robbins. Stuff I've seen: A system for personal informa- tion retrieval and re-use. In Proceedings of SIGIR, pages 72-79, 2003.","title":"Stuff I've seen: A system for personal information retrieval and re-use","authors":"Susan Dumais, Edward Cutrell, Gavin Cadiz, Raman Jancke, Daniel C Sarin, Robbins","year":"2003","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guo and Agichtein, 2012] Qi Guo and Eugene Agichtein. Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior. In Proceedings of WWW, pages 569-578, 2012.","title":"Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior","authors":"Agichtein ; ] Qi Guo, Eugene Guo, Agichtein","year":"2012","venue":"Proceedings of WWW","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guo et al., 2013] Qi Guo, Haojian Jin, Dmitry Lagun, Shuai Yuan, and Eugene Agichtein. Mining touch interaction data on mobile devices to predict web search result rele- vance. In Proceedings of SIGIR, pages 153-162, 2013.","title":"Mining touch interaction data on mobile devices to predict web search result relevance","authors":"Guo","year":"2013","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guy et al., 2018] Ido Guy, Alexander Nus, Dan Pelleg, and Idan Szpektor. Care to share?: Learning to rank personal photos for public sharing. In Proceedings of WSDM, pages 207-215, 2018.","title":"Care to share?: Learning to rank personal photos for public sharing","authors":"Ido Guy, Alexander Nus, Dan Pelleg, Idan Szpektor","year":"2018","venue":"Proceedings of WSDM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hastie et al., 2001] Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer, 2001.","title":"The Elements of Statistical Learning","authors":"Trevor Hastie, Robert Tibshirani, Jerome Friedman","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Joachims et al., 2005] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. Accurately interpreting clickthrough data as implicit feedback. In Pro- ceedings of SIGIR, pages 154-161, 2005.","title":"Accurately interpreting clickthrough data as implicit feedback","authors":"Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Geri Gay","year":"2005","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Joachims, 2002] Thorsten Joachims. Optimizing search en- gines using clickthrough data. In Proceedings of KDD, pages 133-142, 2002.","title":"Optimizing search engines using clickthrough data","authors":"Thorsten Joachims, Joachims","year":"2002","venue":"Proceedings of KDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kamvar et al., 2009] Maryam Kamvar, Melanie Kellar, Ra- jan Patel, and Ya Xu. Computers and iPhones and mobile phones, oh my!: A logs-based comparison of search users on different devices. In Proceedings of WWW, pages 801- 810, 2009.","title":"Computers and iPhones and mobile phones, oh my!: A logs-based comparison of search users on different devices","authors":"Maryam Kamvar, Melanie Kellar, Rajan Patel, Ya Xu","year":"2009","venue":"Proceedings of WWW","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu, 2009] Tie-Yan Liu. Learning to rank for information retrieval. Foundations and Trends in Information Re- trieval, 3(3):225-331, 2009.","title":"Learning to rank for information retrieval","authors":"Tie-Yan Liu, Liu","year":"2009","venue":"Foundations and Trends in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Richardson et al., 2006] Matthew Richardson, Amit Prakash, and Eric Brill. Beyond PageRank: machine learning for static ranking. In Proceedings of WWW, pages 707-715, 2006.","title":"Beyond PageRank: machine learning for static ranking","authors":"Matthew Richardson, Amit Prakash, Eric Brill","year":"2006","venue":"Proceedings of WWW","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Richardson et al., 2007] Matthew Richardson, Ewa Domi- nowska, and Robert Ragno. Predicting clicks: Estimat- ing the click-through rate for new ads. In Proceedings of WWW, pages 521-530, 2007.","title":"Predicting clicks: Estimating the click-through rate for new ads","authors":"Matthew Richardson, Ewa Dominowska, Robert Ragno","year":"2007","venue":"Proceedings of WWW","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tromba et al., 2017] Isabella Tromba, John Gal- lagher, and Jason Liszka. Search at Slack. https://slack.engineering/search-at-slack-431f8c80619e, 2017.","title":"Search at Slack","authors":"Isabella Tromba, John Gallagher, Jason Liszka","year":"2017","venue":null,"doi":null,"url":"https://slack.engineering/search-at-slack-431f8c80619e","internal_url":null},{"position":20,"raw":"[Wang et al., 2016] Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork. Learning to rank with selection bias in personal search. In Proceedings of SIGIR, pages 115-124, 2016.","title":"Learning to rank with selection bias in personal search","authors":"Wang","year":"2016","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2018] Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc Najork. Position bias estimation for unbiased learning to rank in personal search. In Proceedings of WSDM, pages 610-618, 2018.","title":"Position bias estimation for unbiased learning to rank in personal search","authors":"Wang","year":"2018","venue":"Proceedings of WSDM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yue et al., 2010] Yisong Yue, Rajan Patel, and Hein Roehrig. Beyond position bias: Examining result attrac- tiveness as a source of presentation bias in clickthrough data. In Proceedings of WWW, 2010.","title":"Beyond position bias: Examining result attractiveness as a source of presentation bias in clickthrough data","authors":"Yue","year":"2010","venue":"Proceedings of WWW","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}