{"type":"ScholarlyArticle","title":"Multinomial Logit Bandit with Linear Utility Functions","authors":[{"name":"Mingdong Ou","slug":"mingdong-ou","given":"Mingdong","family":"Ou","affiliation":"Alibaba Group , Hang Zhou , China"},{"name":"Nan Li","slug":"nan-li","given":"Nan","family":"Li","affiliation":"Alibaba Group , Hang Zhou , China"},{"name":"Shenghuo Zhu","slug":"shenghuo-zhu","given":"Shenghuo","family":"Zhu","affiliation":"Alibaba Group , Hang Zhou , China"},{"name":"Rong Jin","slug":"rong-jin","given":"Rong","family":"Jin","affiliation":"Alibaba Group , Hang Zhou , China"}],"abstract":"Multinomial logit bandit is a sequential subset selection problem which arises in many applications. In each round, the player selects a K-cardinality subset from N candidate items, and receives a reward which is governed by a multinomial logit (MNL) choice model considering both item utility and substitution property among items. The player's objective is to dynamically learn the parameters of MNL model and maximize cumulative reward over a finite horizon T. This problem faces the exploration-exploitation dilemma, and the involved combinatorial nature makes it non-trivial. In recent years, there have developed some algorithms by exploiting specific characteristics of the MNL model, but all of them estimate the parameters of MNL model separately and incur a regret bound which is not preferred for large candidate set size N. In this paper, we consider the linear utility MNL choice model whose item utilities are represented as linear functions of d-dimension item features, and propose an algorithm, titled LUMB, to exploit the underlying structure. It is proven that the proposed algorithm achieves regret which is free of candidate set size.  Experiments show the superiority of the proposed algorithm.","doi":"10.24963/ijcai.2018/361","url":"https://ijcai.org/proceedings/2018/361","pdf":"https://ijcai.org/proceedings/2018/0361.pdf","markdown":"https://ijcai.org/proceedings/2018/361.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/361","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":361,"pages":{"first":2602,"last":2608},"keywords":["Machine Learning: Reinforcement Learning","Uncertainty in AI: Sequential Decision Making"],"videos":[],"references":[{"position":1,"raw":"[Abbasi-Yadkori et al., 2011] Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In NIPS, pages 2312-2320, 2011.","title":"Improved algorithms for linear stochastic bandits","authors":"Yasin Abbasi-Yadkori, Dávid Pál, Csaba Szepesvári","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agrawal and Goyal, 2012] Shipra Agrawal and Navin Goyal. Analysis of thompson sampling for the multi- armed bandit problem. In COLT, pages 39-1, 2012.","title":"Analysis of thompson sampling for the multiarmed bandit problem","authors":"Shipra Agrawal, Navin Goyal","year":"2012","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Agrawal and Goyal, 2013] Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear payoffs. In ICML, pages 127-135, 2013.","title":"Thompson sampling for contextual bandits with linear payoffs","authors":"Shipra Agrawal, Navin Goyal","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Agrawal et al., 2017a] Shipra Agrawal, Vashist Avadhan- ula, Vineet Goyal, and Assaf Zeevi. Mnl-bandit: A dy- namic learning approach to assortment selection. arXiv preprint arXiv:1706.03880, 2017.","title":"Mnl-bandit: A dynamic learning approach to assortment selection","authors":"Agrawal","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Agrawal et al., 2017b] Shipra Agrawal, Vashist Avadhan- ula, Vineet Goyal, and Assaf Zeevi. Thompson sampling for the mnl-bandit. arXiv preprint arXiv:1706.00977, 2017.","title":"Thompson sampling for the mnl-bandit","authors":"Agrawal","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Audibert et al., 2013] Jean-Yves Audibert, Sébastien Bubeck, and Gábor Lugosi. Regret in online combinato- rial optimization. Mathematics of Operations Research, 39(1):31-45, 2013.","title":"Regret in online combinatorial optimization","authors":"Audibert","year":"2013","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Auer, 2002] Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. JMLR, 3(Nov):397- 422, 2002.","title":"Using confidence bounds for exploitation-exploration trade-offs","authors":"Peter Auer","year":"2002","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bernstein et al., 2017] Fernando Bernstein, Sajad Modaresi, and Denis Sauré. A dynamic clustering approach to data-driven assortment personalization. 2017.","title":"A dynamic clustering approach to data-driven assortment personalization","authors":"Fernando Bernstein, Sajad Modaresi, Denis Sauré","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Bubeck and Cesa-Bianchi, 2012] Sébastien Bubeck and Nicolò Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. CoRR, abs/1204.5721, 2012.","title":"Regret analysis of stochastic and nonstochastic multi-armed bandit problems","authors":"Cesa-Bianchi ; Bubeck, Sébastien Bubeck, Nicolò Cesa-Bianchi","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Caro and Gallien, 2007] Felipe Caro and Jérémie Gallien. Dynamic assortment with demand learning for seasonal consumer goods. Management Science, 53(2):276-292, 2007.","title":"Dynamic assortment with demand learning for seasonal consumer goods","authors":"Caro, Gallien, Felipe Caro, Jérémie Gallien","year":"2007","venue":"Management Science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Cesa-Bianchi and Lugosi, 2012] Nicolo Cesa-Bianchi and Gábor Lugosi. Combinatorial bandits. Journal of Com- puter and System Sciences, 78(5):1404-1422, 2012.","title":"Nicolo Cesa-Bianchi and Gábor Lugosi","authors":"Cesa-Bianchi, Lugosi","year":"2012","venue":"Journal of Computer and System Sciences","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Chen et al., 2013] Wei Chen, Yajun Wang, and Yang Yuan. Combinatorial multi-armed bandit: General framework and applications. In ICML, pages 151-159, 2013.","title":"Combinatorial multi-armed bandit: General framework and applications","authors":"Wei Chen, Yajun Wang, Yang Yuan","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Chen et al., 2016] Wei Chen, Wei Hu, Fu Li, Jian Li, Yu Liu, and Pinyan Lu. Combinatorial multi-armed bandit with general reward functions. In NIPS, pages 1659-1667, 2016.","title":"Combinatorial multi-armed bandit with general reward functions","authors":"Wei Chen, Wei Hu, Fu Li, Jian Li, Yu Liu, Pinyan Lu","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Cheung and Simchi-Levi, 2017] Wang Chi Cheung and David Simchi-Levi. Assortment optimization under un- known multinomial logit choice models. arXiv preprint arXiv:1704.00108, 2017.","title":"Assortment optimization under unknown multinomial logit choice models","authors":"Chi Wang, David Cheung, Simchi-Levi","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Chu et al., 2011] Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In IJCAI, pages 208-214, 2011.","title":"Contextual bandits with linear payoff functions","authors":"Wei Chu, Lihong Li, Lev Reyzin, Robert Schapire","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Dani et al., 2008] Varsha Dani, Thomas Hayes, and Sham Kakade. Stochastic linear optimization under bandit feed- back. In COLT, pages 355-366, 2008.","title":"Stochastic linear optimization under bandit feedback","authors":"Varsha Dani, Thomas Hayes, Sham Kakade","year":"2008","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Davis et al., 2013] James Davis, Guillermo Gallego, and Huseyin Topaloglu. Assortment planning under the multi- nomial logit model with totally unimodular constraint structures. Technical Report, Cornell University, pages 335-357, 2013.","title":"Assortment planning under the multinomial logit model with totally unimodular constraint structures","authors":"James Davis, Guillermo Gallego, Huseyin Topaloglu","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gai et al., 2012] Yi Gai, Bhaskar Krishnamachari, and Rahul Jain. Combinatorial network optimization with unknown variables: Multi-armed bandits with linear re- wards and individual observations. TON, 20(5):1466- 1478, 2012.","title":"Combinatorial network optimization with unknown variables: Multi-armed bandits with linear rewards and individual observations","authors":"Yi Gai, Bhaskar Krishnamachari, Rahul Jain","year":"2012","venue":"TON","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Golrezaei et al., 2014] Negin Golrezaei, Hamid Naz- erzadeh, and Paat Rusmevichientong. Real-time optimization of personalized assortments. Management Science, 60(6):1532-1551, 2014.","title":"Real-time optimization of personalized assortments","authors":"Negin Golrezaei, Hamid Nazerzadeh, Paat Rusmevichientong","year":"2014","venue":"Management Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kallus and Udell, 2016] Nathan Kallus and Madeleine Udell. Dynamic assortment personalization in high dimensions. arXiv preprint arXiv:1610.05604, 2016.","title":"Dynamic assortment personalization in high dimensions","authors":"Udell ; Kallus, Nathan Kallus, Madeleine Udell","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kveton et al., 2015] Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvari. Tight regret bounds for stochastic combinatorial semi-bandits. In Artificial Intelli- gence and Statistics, pages 535-543, 2015.","title":"Tight regret bounds for stochastic combinatorial semi-bandits","authors":"Zheng Branislav Kveton, Azin Wen, Csaba Ashkan, Szepesvari","year":"2015","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lagrée et al., 2016] Paul Lagrée, Claire Vernade, and Olivier Cappe. Multiple-play bandits in the position-based model. In NIPS, pages 1597-1605. 2016.","title":"Multiple-play bandits in the position-based model","authors":"Paul Lagrée, Claire Vernade, Olivier Cappe","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Luce, 2005] Duncan Luce. Individual choice behavior: A theoretical analysis. Courier Corporation, 2005.","title":"Individual choice behavior: A theoretical analysis","authors":"Duncan Luce","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Plackett, 1975] Robin L Plackett. The analysis of permuta- tions. Applied Statistics, pages 193-202, 1975.","title":"The analysis of permutations","authors":"Robin L Plackett","year":"1975","venue":"Applied Statistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Rusmevichientong et al., 2010] Paat Rusmevichientong, Zuo-Jun Max Shen, and David Shmoys. Dynamic assortment optimization with a multinomial logit choice model and capacity constraint. Operations research, 58(6):1666-1680, 2010.","title":"Dynamic assortment optimization with a multinomial logit choice model and capacity constraint","authors":"Paat Rusmevichientong, Max Zuo-Jun, David Shen, Shmoys","year":"2010","venue":"Operations research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Russo and Van Roy, 2014] Daniel Russo and Benjamin Van Roy. Learning to optimize via posterior sampling. Mathematics of Operations Research, 39(4):1221-1243, 2014.","title":"Learning to optimize via posterior sampling","authors":"Van Russo, Daniel Roy, Benjamin Russo, Roy Van","year":"2014","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sauré and Zeevi, 2013] Denis Sauré and Assaf Zeevi. Op- timal dynamic assortment planning with demand learn- ing. Manufacturing & Service Operations Management, 15(3):387-404, 2013.","title":"Optimal dynamic assortment planning with demand learning","authors":"Zeevi Sauré, Denis Sauré, Assaf Zeevi","year":"2013","venue":"Manufacturing & Service Operations Management","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Thompson, 1933] William Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285- 294, 1933.","title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","authors":"William Thompson","year":"1933","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2017] Yingfei Wang, Hua Ouyang, Chu Wang, Jianhui Chen, Tsvetan Asamov, and Yi Chang. Efficient ordered combinatorial semi-bandits for whole-page rec- ommendation. In AAAI, pages 2746-2753, 2017.","title":"Efficient ordered combinatorial semi-bandits for whole-page recommendation","authors":"Yingfei Wang, Hua Ouyang, Chu Wang, Jianhui Chen, Tsvetan Asamov, Yi Chang","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wen et al., 2015] Zheng Wen, Branislav Kveton, and Azin Ashkan. Efficient learning in large-scale combinatorial semi-bandits. In ICML, pages 1113-1122, 2015.","title":"Efficient learning in large-scale combinatorial semi-bandits","authors":"Zheng Wen, Branislav Kveton, Azin Ashkan","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}