{"type":"ScholarlyArticle","title":"FedConPE: Efficient Federated Conversational Bandits with Heterogeneous Clients","authors":[{"name":"Zhuohua Li","slug":"zhuohua-li","given":"Zhuohua","family":"Li","affiliation":"The Chinese University of Hong Kong"},{"name":"Maoli Liu","slug":"maoli-liu","given":"Maoli","family":"Liu","affiliation":"The Chinese University of Hong Kong"},{"name":"John C. S. Lui","slug":"john-c-s-lui","given":"John C.S.","family":"Lui","affiliation":"The Chinese University of Hong Kong"}],"abstract":"Conversational recommender systems have emerged as a potent solution for efficiently eliciting user preferences. These systems interactively present queries associated with \"key terms\" to users and leverage user feedback to estimate user preferences more efficiently. Nonetheless, most existing algorithms adopt a centralized approach. In this paper, we introduce FedConPE, a phase elimination-based federated conversational bandit algorithm, where M agents collaboratively solve a global contextual linear bandit problem with the help of a central server while ensuring secure data management. To effectively coordinate all the clients and aggregate their collected data, FedConPE uses an adaptive approach to construct key terms that minimize uncertainty across all dimensions in the feature space. Furthermore, compared with existing federated linear bandit algorithms, FedConPE offers improved computational and communication efficiency as well as enhanced privacy protections. Our theoretical analysis shows that FedConPE is minimax near-optimal in terms of cumulative regret. We also establish upper bounds for communication costs and conversation frequency. Comprehensive evaluations demonstrate that FedConPE outperforms existing conversational bandit algorithms while using fewer conversations.","doi":"10.24963/ijcai.2024/501","url":"https://ijcai.org/proceedings/2024/501","pdf":"https://ijcai.org/proceedings/2024/0501.pdf","markdown":"https://ijcai.org/proceedings/2024/501.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/501","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":501,"pages":{"first":4533,"last":4541},"keywords":["Machine Learning: ML: Multi-armed bandits","Machine Learning: ML: Federated learning","Machine Learning: ML: Online learning","Data Mining: DM: Recommender systems"],"videos":[],"references":[{"position":1,"raw":"[Abbasi-Yadkori et al., 2011] Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for lin- ear stochastic bandits. In Proceedings of the 24th Inter- national Conference on Neural Information Processing Systems, NIPS'11, page 2312-2320, 2011.","title":"Improved algorithms for linear stochastic bandits","authors":"Yasin Abbasi-Yadkori, Dávid Pál, Csaba Szepesvári","year":"2011","venue":"Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS'11","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bretagnolle and Huber, 1978] J. Bretagnolle and C. Huber. Estimation des densités : Risque minimax. In C. Del- lacherie, P. A. Meyer, and M. Weil, editors, Séminaire de Probabilités XII, pages 342-363, Berlin, Heidelberg, 1978. Springer Berlin Heidelberg.","title":"Estimation des densités : Risque minimax","authors":"J Bretagnolle, C Huber","year":"1978","venue":"Séminaire de Probabilités XII","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cantador et al., 2011] Ivan Cantador, Peter Brusilovsky, and Tsvi Kuflik. Second workshop on information heterogene- ity and fusion in recommender systems (hetrec2011). In Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys '11, page 387-388, 2011.","title":"Second workshop on information heterogeneity and fusion in recommender systems (hetrec2011)","authors":"Ivan Cantador, Peter Brusilovsky, Tsvi Kuflik","year":"2011","venue":"Proceedings of the Fifth ACM Conference on Recommender Systems, RecSys '11","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2023] Yu-Zhen Janice Chen, Lin Yang, Xuchuang Wang, Xutong Liu, Mohammad Hajiesmaili, John CS Lui, and Don Towsley. On-demand communica- tion for asynchronous multi-agent bandits. In International Conference on Artificial Intelligence and Statistics, pages 3903-3930. PMLR, 2023.","title":"On-demand communication for asynchronous multi-agent bandits","authors":"Yu-Zhen Janice Chen, Lin Yang, Xuchuang Wang, Xutong Liu, Mohammad Hajiesmaili, John Cs Lui, Don Towsley","year":"2023","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Christakopoulou et al., 2016] Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. Towards conversa- tional recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, page 815-824, 2016.","title":"Towards conversational recommender systems","authors":"Konstantina Christakopoulou, Filip Radlinski, Katja Hofmann","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chu et al., 2011] Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 208-214, 2011.","title":"Contextual bandits with linear payoff functions","authors":"Wei Chu, Lihong Li, Lev Reyzin, Robert Schapire","year":"2011","venue":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dubey and Pentland, 2020] Abhimanyu Dubey and Alex Pentland. Differentially-private federated linear bandits. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, 2020.","title":"Abhimanyu Dubey and Alex Pentland. Differentially-private federated linear bandits","authors":"Pentland Dubey","year":"2020","venue":"Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Frank and Wolfe, 1956] Marguerite Frank and Philip Wolfe. An algorithm for quadratic programming. Naval Research Logistics Quarterly, 3(1-2):95-110, 1956.","title":"An algorithm for quadratic programming","authors":"Wolfe Frank, Marguerite Frank, Philip Wolfe","year":"1956","venue":"Naval Research Logistics Quarterly","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gao et al., 2021] Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. Advances and challenges in conversational recommender systems: A sur- vey. AI Open, 2:100-126, 2021.","title":"Advances and challenges in conversational recommender systems: A survey","authors":"Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten De Rijke, Tat-Seng Chua","year":"2021","venue":"AI Open","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Harper and Konstan, 2015] F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst., 5(4), dec 2015.","title":"The movielens datasets: History and context","authors":"F, Maxwell Harper, Joseph A Konstan","year":"2015","venue":"ACM Trans. Interact. Intell. Syst","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2022] Jiafan He, Tianhao Wang, Yifei Min, and Quanquan Gu. A simple and provably efficient algorithm for asynchronous federated contextual linear bandits. In Advances in Neural Information Processing Systems, vol- ume 35, pages 4762-4775, 2022.","title":"A simple and provably efficient algorithm for asynchronous federated contextual linear bandits","authors":"He","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2021] Ruiquan Huang, Weiqiang Wu, Jing Yang, and Cong Shen. Federated linear contextual ban- dits. Advances in neural information processing systems, 34:27057-27068, 2021.","title":"Federated linear contextual bandits","authors":"Huang","year":"2021","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kiefer and Wolfowitz, 1960] J. Kiefer and J. Wolfowitz. The equivalence of two extremum problems. Canadian Journal of Mathematics, 12:363-366, 1960.","title":"The equivalence of two extremum problems","authors":"J Kiefer, J Wolfowitz","year":"1960","venue":"Canadian Journal of Mathematics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lattimore and Szepesvári, 2020] Tor Lattimore and Csaba Szepesvári. Bandit Algorithms. Cambridge University Press, 2020.","title":"Tor Lattimore and Csaba Szepesvári. Bandit Algorithms","authors":"Szepesvári Lattimore","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lei et al., 2020] Wenqiang Lei, Xiangnan He, Maarten de Ri- jke, and Tat-Seng Chua. Conversational recommendation: Formulation, methods, and evaluation. In Proceedings of the 43rd International ACM SIGIR Conference on Re- search and Development in Information Retrieval, SIGIR '20, page 2425-2428, 2020.","title":"Conversational recommendation: Formulation, methods, and evaluation","authors":"Lei","year":"2020","venue":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li and Wang, 2022] Chuanhao Li and Hongning Wang. Asynchronous upper confidence bound algorithms for fed- erated linear bandits. In International Conference on Artifi- cial Intelligence and Statistics, pages 6529-6553, 2022.","title":"Asynchronous upper confidence bound algorithms for federated linear bandits","authors":"Wang ; Li, Chuanhao Li, Hongning Wang","year":"2022","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2010] Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual-bandit approach to per- sonalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web, WWW '10, page 661-670, 2010.","title":"A contextual-bandit approach to personalized news article recommendation","authors":"Lihong Li, Wei Chu, John Langford, Robert E Schapire","year":"2010","venue":"Proceedings of the 19th International Conference on World Wide Web, WWW '10","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2024] Zhuohua Li, Maoli Liu, and John C. S. Lui. Fedconpe: Efficient federated conversational bandits with heterogeneous clients, 2024.","title":"Fedconpe: Efficient federated conversational bandits with heterogeneous clients","authors":"Zhuohua Li, Maoli Liu, John C S Lui","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin and Moothedath, 2023] Jiabin Lin and Shana Moothe- dath. Federated stochastic bandit learning with unobserved context, 2023.","title":"Federated stochastic bandit learning with unobserved context","authors":"Lin, Moothedath ;, Jiabin Lin, Shana Moothedath","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2022] Xutong Liu, Haoru Zhao, Tong Yu, Shuai Li, and John CS Lui. Federated online clustering of bandits. In Uncertainty in Artificial Intelligence, pages 1221-1231, 2022.","title":"Federated online clustering of bandits","authors":"Liu","year":"2022","venue":"Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun and Zhang, 2018] Yueming Sun and Yi Zhang. Conver- sational recommender system. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR '18, page 235-244, 2018.","title":"Conversational recommender system","authors":"Yueming Sun, Yi Zhang","year":"2018","venue":"The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR '18","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Todd, 2016] Michael J. Todd. Minimum-Volume Ellipsoids. Society for Industrial and Applied Mathematics, 2016.","title":"Minimum-Volume Ellipsoids","authors":"J Michael, Todd","year":"2016","venue":"Society for Industrial and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2020] Yuanhao Wang, Jiachen Hu, Xiaoyu Chen, and Liwei Wang. Distributed bandit learning: Near- optimal regret with efficient communication. In 8th Interna- tional Conference on Learning Representations, ICLR'20, 2020.","title":"Distributed bandit learning: Nearoptimal regret with efficient communication","authors":"Wang","year":"2020","venue":"8th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2023] Zhiyong Wang, Xutong Liu, Shuai Li, and John C. S. Lui. Efficient explorative key-term selec- tion strategies for conversational contextual bandits. Pro- ceedings of the AAAI Conference on Artificial Intelligence, 37(8):10288-10295, Jun. 2023.","title":"Efficient explorative key-term selection strategies for conversational contextual bandits","authors":"Zhiyong Wang, Xutong Liu, Shuai Li, John C S Lui","year":"2023","venue":"Pro-ceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Weyl, 1912] Hermann Weyl. Das asymptotische verteilungs- gesetz der eigenwerte linearer partieller differentialgle- ichungen (mit einer anwendung auf die theorie der hohlraumstrahlung). Mathematische Annalen, 71(4):441 - 479, 1912.","title":"Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung)","authors":"Hermann Weyl","year":"1912","venue":"Mathematische Annalen","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2021] Junda Wu, Canzhe Zhao, Tong Yu, Jingyang Li, and Shuai Li. Clustering of conversational bandits for user preference learning and elicitation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM '21, page 2129-2139, 2021.","title":"Clustering of conversational bandits for user preference learning and elicitation","authors":"Junda Wu, Canzhe Zhao, Tong Yu, Jingyang Li, Shuai Li","year":"2021","venue":"Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM '21","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xia et al., 2023] Yu Xia, Junda Wu, Tong Yu, Sungchul Kim, Ryan A. Rossi, and Shuai Li. User-regulation decon- founded conversational recommender system with bandit feedback. In Proceedings of the 29th ACM SIGKDD Con- ference on Knowledge Discovery and Data Mining, KDD '23, page 2694-2704, 2023.","title":"User-regulation deconfounded conversational recommender system with bandit feedback","authors":"Yu Xia, Junda Wu, Tong Yu, Sungchul Kim, Ryan A Rossi, Shuai Li","year":"2023","venue":"Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '23","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xie et al., 2021] Zhihui Xie, Tong Yu, Canzhe Zhao, and Shuai Li. Comparison-based conversational recommender system with relative bandit feedback. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '21, page 1400-1409, 2021.","title":"Comparison-based conversational recommender system with relative bandit feedback","authors":"Zhihui Xie, Tong Yu, Canzhe Zhao, Shuai Li","year":"2021","venue":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '21","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yang et al., 2024] Hantao Yang, Xutong Liu, Zhiyong Wang, Hong Xie, John C. S. Lui, Defu Lian, and Enhong Chen. Federated contextual cascading bandits with asyn- chronous communication and heterogeneous users. Pro- ceedings of the AAAI Conference on Artificial Intelligence, 38(18):20596-20603, Mar. 2024.","title":"Federated contextual cascading bandits with asynchronous communication and heterogeneous users","authors":"Hantao Yang, Xutong Liu, Zhiyong Wang, Hong Xie, C S John, Defu Lui, Enhong Lian, Chen","year":"2024","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2020] Xiaoying Zhang, Hong Xie, Hang Li, and John C.S. Lui. Conversational contextual bandit: Algo- rithm and application. In Proceedings of The Web Confer- ence 2020, WWW '20, page 662-672, 2020.","title":"Conversational contextual bandit: Algorithm and application","authors":"Xiaoying Zhang, Hong Xie, Hang Li, John C S Lui","year":"2020","venue":"Proceedings of The Web Conference 2020","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhao et al., 2022] Canzhe Zhao, Tong Yu, Zhihui Xie, and Shuai Li. Knowledge-aware conversational preference elic- itation with bandit feedback. In Proceedings of the ACM Web Conference 2022, page 483-492, 2022.","title":"Knowledge-aware conversational preference elicitation with bandit feedback","authors":"Canzhe Zhao, Tong Yu, Zhihui Xie, Shuai Li","year":"2022","venue":"Proceedings of the ACM Web Conference 2022","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhou and Chowdhury, 2023] Xingyu Zhou and Sayak Ray Chowdhury. On differentially private federated linear con- textual bandits, 2023.","title":"On differentially private federated linear contextual bandits","authors":"Chowdhury Zhou, Xingyu Zhou, Sayak Ray, Chowdhury","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zuo et al., 2022] Jinhang Zuo, Songwen Hu, Tong Yu, Shuai Li, Handong Zhao, and Carlee Joe-Wong. Hierarchical conversational preference elicitation with bandit feedback. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, CIKM '22, page 2827-2836, 2022.","title":"Hierarchical conversational preference elicitation with bandit feedback","authors":"Zuo","year":"2022","venue":"Proceedings of the 31st ACM International Conference on Information & Knowledge Management, CIKM '22","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}