{"type":"ScholarlyArticle","title":"Thompson Sampling for Bandit Learning in Matching Markets","authors":[{"name":"Fang Kong","slug":"fang-kong","given":"Fang","family":"Kong","affiliation":"Center for Computer Science , Shanghai Jiao Tong University"},{"name":"Junming Yin","slug":"junming-yin","given":"Junming","family":"Yin","affiliation":"Tepper School of Business , Carnegie Mellon University"},{"name":"Shuai Li","slug":"shuai-li","given":"Shuai","family":"Li","affiliation":"Center for Computer Science , Shanghai Jiao Tong University"}],"abstract":"The problem of two-sided matching markets has a wide range of real-world applications and has been extensively studied in the literature. A line of recent works have focused on the problem setting where the preferences of one-side market participants are unknown a priori and are learned by iteratively interacting with the other side of participants. All these works are based on explore-then-commit (ETC) and upper confidence bound (UCB) algorithms, two common strategies in multi-armed bandits (MAB). Thompson sampling (TS) is another popular approach, which attracts lots of attention due to its easier implementation and better empirical performances. In many problems, even when UCB and ETC-type algorithms have already been analyzed, researchers are still trying to study TS for its benefits. However, the convergence analysis of TS is much more challenging and remains open in many problem settings. In this paper, we provide the first regret analysis for TS in the new setting of iterative matching markets. Extensive experiments demonstrate the practical advantages of the TS-type algorithm over the ETC and UCB-type baselines.","doi":"10.24963/ijcai.2022/439","url":"https://ijcai.org/proceedings/2022/439","pdf":"https://ijcai.org/proceedings/2022/0439.pdf","markdown":"https://ijcai.org/proceedings/2022/439.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/439","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":439,"pages":{"first":3164,"last":3170},"keywords":["Machine Learning: Online Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986537","length_sec":77,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984680","length_sec":308,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abdulkadiroglu and Sönmez, 1999] Atila Abdulkadiroglu and Tayfun Sönmez. House allocation with existing ten- ants. Journal of Economic Theory, 88(2):233-260, 1999.","title":"Atila Abdulkadiroglu and Tayfun Sönmez. House allocation with existing tenants","authors":"Sönmez Abdulkadiroglu","year":"1999","venue":"Journal of Economic Theory","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agrawal and Goyal, 2012] Shipra Agrawal and Navin Goyal. Analysis of thompson sampling for the multi- armed bandit problem. In Conference on Learning Theory, pages 39-1, 2012.","title":"Analysis of thompson sampling for the multiarmed bandit problem","authors":"Shipra Agrawal, Navin Goyal","year":"2012","venue":"Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Agrawal and Goyal, 2013] Shipra Agrawal and Navin Goyal. Further optimal regret bounds for thompson sampling. In AISTATS, pages 99-107, 2013.","title":"Further optimal regret bounds for thompson sampling","authors":"Shipra Agrawal, Navin Goyal","year":"2013","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Auer et al., 2002] Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2):235-256, 2002.","title":"Finite-time analysis of the multiarmed bandit problem","authors":"Auer","year":"2002","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Basu et al., 2021] Soumya Basu, Karthik Abinav Sankararaman, and Abishek Sankararaman. Beyond log 2 (t) regret for decentralized bandits in matching markets. In ICML, pages 705-715, 2021.","title":"Beyond log 2 (t) regret for decentralized bandits in matching markets","authors":"Soumya Basu, Karthik Abinav Sankararaman, Abishek Sankararaman","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bistritz and Leshem, 2018] Ilai Bistritz and Amir Leshem. Distributed multi-player bandits-a game of thrones ap- proach. In Advances in Neural Information Processing Systems, pages 7222-7232, 2018.","title":"Distributed multi-player bandits-a game of thrones approach","authors":"Ilai Bistritz, Amir Leshem","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2020] Wei Chen, Yihan Du, Longbo Huang, and Haoyu Zhao. Combinatorial pure exploration for du- eling bandit. In ICML, pages 1531-1541, 2020.","title":"Combinatorial pure exploration for dueling bandit","authors":"Wei Chen, Yihan Du, Longbo Huang, Haoyu Zhao","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cheung et al., 2019] Wang Chi Cheung, Vincent Tan, and Zixin Zhong. A thompson sampling algorithm for cascad- ing bandits. In AISTATS, pages 438-447, 2019.","title":"A thompson sampling algorithm for cascading bandits","authors":"Chi Wang, Vincent Cheung, Zixin Tan, Zhong","year":"2019","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dai and Jordan, 2021] Xiaowu Dai and Michael I Jordan. Learning strategies in decentralized matching markets un- der uncertain preferences. Journal of Machine Learning Research, 22(260):1-50, 2021.","title":"Learning strategies in decentralized matching markets under uncertain preferences","authors":"Jordan ; Dai, Xiaowu Dai, Michael I Jordan","year":"2021","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Das and Kamenica, 2005] Sanmay Das and Emir Ka- menica. Two-sided bandits and the dating market. In Inter- national Joint Conference on Artificial Intelligence, pages 947-952, 2005.","title":"Sanmay Das and Emir Kamenica. Two-sided bandits and the dating market","authors":"Kamenica Das","year":"2005","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Epple et al., 2006] Dennis Epple, Richard Romano, and Holger Sieg. Admission, tuition, and financial aid poli- cies in the market for higher education. Econometrica, 74(4):885-928, 2006.","title":"Admission, tuition, and financial aid policies in the market for higher education","authors":"Dennis Epple, Richard Romano, Holger Sieg","year":"2006","venue":"Econometrica","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fu, 2014] Chao Fu. Equilibrium tuition, applications, ad- missions, and enrollment in the college market. Journal of Political Economy, 122(2):225-281, 2014.","title":"Equilibrium tuition, applications, admissions, and enrollment in the college market","authors":"Chao Fu","year":"2014","venue":"Journal of Political Economy","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gale and Shapley, 1962] David Gale and Lloyd S Shapley. College admissions and the stability of marriage. The American Mathematical Monthly, 69(1):9-15, 1962.","title":"College admissions and the stability of marriage","authors":"Shapley Gale","year":"1962","venue":"The American Mathematical Monthly","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Haeringer and Wooders, 2011] Guillaume Haeringer and Myrna Wooders. Decentralized job matching. Interna- tional Journal of Game Theory, 40(1):1-28, 2011.","title":"Decentralized job matching","authors":"Wooders Haeringer, Myrna Wooders","year":"2011","venue":"International Journal of Game Theory","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kaufmann et al., 2012] Emilie Kaufmann, Nathaniel Ko- rda, and Rémi Munos. Thompson sampling: An asymptot- ically optimal finite-time analysis. In International Con- ference on Algorithmic Learning Theory, pages 199-213. Springer, 2012.","title":"Thompson sampling: An asymptotically optimal finite-time analysis","authors":"Kaufmann","year":"2012","venue":"International Conference on Algorithmic Learning Theory","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kong et al., 2021] Fang Kong, Yueran Yang, Wei Chen, and Shuai Li. The hardness analysis of thompson sampling for combinatorial semi-bandits with greedy oracle. In NeurIPS, volume 34, 2021.","title":"The hardness analysis of thompson sampling for combinatorial semi-bandits with greedy oracle","authors":"Fang Kong, Yueran Yang, Wei Chen, Shuai Li","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lattimore and Szepesvári, 2020] Tor Lattimore and Csaba Szepesvári. Bandit Algorithms. Cambridge University Press, 2020.","title":"Tor Lattimore and Csaba Szepesvári. Bandit Algorithms","authors":"Szepesvári Lattimore","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu and Zhao, 2010] Keqin Liu and Qing Zhao. Distributed learning in multi-armed bandit with multiple players. IEEE Transactions on Signal Processing, 58(11):5667- 5681, 2010.","title":"Distributed learning in multi-armed bandit with multiple players","authors":"Zhao ; ] Keqin Liu, Qing Liu, Zhao","year":"2010","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2020] Lydia T Liu, Horia Mania, and Michael Jordan. Competing bandits in matching markets. In AIS- TATS, pages 1618-1628, 2020.","title":"Competing bandits in matching markets","authors":"Lydia T Liu, Horia Mania, Michael Jordan","year":"2020","venue":"AIS-TATS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2021] Lydia T Liu, Feng Ruan, Horia Mania, and Michael I Jordan. Bandit learning in decentralized match- ing markets. Journal of Machine Learning Research, 22(211):1-34, 2021.","title":"Bandit learning in decentralized matching markets","authors":"Lydia T Liu, Feng Ruan, Horia Mania, Michael I Jordan","year":"2021","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Perrault et al., 2020] Pierre Perrault, Etienne Boursier, Vianney Perchet, and Michal Valko. Statistical efficiency of thompson sampling for combinatorial semi-bandits. In NeurIPS, 2020.","title":"Statistical efficiency of thompson sampling for combinatorial semi-bandits","authors":"Perrault","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rosenski et al., 2016] Jonathan Rosenski, Ohad Shamir, and Liran Szlak. Multi-player bandits-a musical chairs approach. In ICML, pages 155-163. PMLR, 2016.","title":"Multi-player bandits-a musical chairs approach","authors":"Jonathan Rosenski, Ohad Shamir, Liran Szlak","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Roth and Sotomayor, 1992] Alvin E Roth and Marilda So- tomayor. Two-sided matching. Handbook of Game Theory with Economic Applications, 1:485-541, 1992.","title":"Two-sided matching","authors":"Alvin E Roth, Marilda","year":"1992","venue":"Handbook of Game Theory with Economic Applications","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Roth and Xing, 1997] Alvin E Roth and Xiaolin Xing. Turnaround time and bottlenecks in market clearing: De- centralized matching in the market for clinical psychol- ogists. Journal of Political Economy, 105(2):284-329, 1997.","title":"Turnaround time and bottlenecks in market clearing: Decentralized matching in the market for clinical psychologists","authors":"Alvin E Roth, Xiaolin Xing","year":"1997","venue":"Journal of Political Economy","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Roth, 1984] Alvin E Roth. The evolution of the labor mar- ket for medical interns and residents: a case study in game theory. Journal of Political Economy, 92(6):991-1016, 1984.","title":"The evolution of the labor market for medical interns and residents: a case study in game theory","authors":"Alvin E Roth","year":"1984","venue":"Journal of Political Economy","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sankararaman et al., 2021] Abishek Sankararaman, Soumya Basu, and Karthik Abinav Sankararaman. Dominate or delete: Decentralized competing bandits in serial dictatorship. In AISTATS, pages 1252-1260, 2021.","title":"Dominate or delete: Decentralized competing bandits in serial dictatorship","authors":"Abishek Sankararaman, Soumya Basu, Karthik Abinav Sankararaman","year":"2021","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Thompson, 1933] William R Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4):285- 294, 1933.","title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","authors":"William R Thompson","year":"1933","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang and Chen, 2018] Siwei Wang and Wei Chen. Thomp- son sampling for combinatorial semi-bandits. In ICML, pages 5114-5122, 2018.","title":"Thompson sampling for combinatorial semi-bandits","authors":"Siwei Wang, Wei Chen","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}