{"type":"ScholarlyArticle","title":"Learning with Posterior Sampling for Revenue Management under Time-varying Demand","authors":[{"name":"Kazuma Shimizu","slug":"kazuma-shimizu","given":"Kazuma","family":"Shimizu","affiliation":"NEC Corporation"},{"name":"Junya Honda","slug":"junya-honda","given":"Junya","family":"Honda","affiliation":"Kyoto University; RIKEN AIP"},{"name":"Shinji Ito","slug":"shinji-ito","given":"Shinji","family":"Ito","affiliation":"NEC Corporation; RIKEN AIP"},{"name":"Shinji Nakadai","slug":"shinji-nakadai","given":"Shinji","family":"Nakadai","affiliation":"NEC Corporation; Intent Exchange , Inc."}],"abstract":"This paper discusses the revenue management (RM) problem to maximize revenue by pricing items or services. One challenge in this problem is that the demand distribution is unknown and varies over time in real applications such as airline and retail industries. In particular, the time-varying demand has not been well studied under scenarios of unknown demand due to the difficulty of jointly managing the remaining inventory and estimating the demand. To tackle this challenge, we first introduce an episodic generalization of the RM problem motivated by typical application scenarios. We then propose a computationally efficient algorithm based on posterior sampling, which effectively optimizes prices by solving linear programming. We derive a Bayesian regret upper bound of this algorithm for general models where demand parameters can be correlated between time periods, while also deriving a regret lower bound for generic algorithms. Our empirical study shows that the proposed algorithm performs better than other benchmark algorithms and comparably to the optimal policy in hindsight. We also propose a heuristic modification of the proposed algorithm, which further efficiently learns the pricing policy in the experiments. An extended version of this paper with appendixes is available at: http://arxiv.org/abs/2405.04910.","doi":"10.24963/ijcai.2024/543","url":"https://ijcai.org/proceedings/2024/543","pdf":"https://ijcai.org/proceedings/2024/0543.pdf","markdown":"https://ijcai.org/proceedings/2024/543.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/543","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":543,"pages":{"first":4911,"last":4919},"keywords":["Machine Learning: ML: Multi-armed bandits","Machine Learning: ML: Online learning","Machine Learning: ML: Bayesian learning"],"videos":[],"references":[{"position":1,"raw":"[Anjos et al., 2005] Miguel F Anjos, Russell CH Cheng, and Christine SM Currie. Optimal pricing policies for perish- able products. European Journal of Operational Research, 166(1):246-254, 2005.","title":"Optimal pricing policies for perishable products","authors":"Russell Ch Miguel F Anjos, Christine Sm Cheng, Currie","year":"2005","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Badanidiyuru et al., 2013] Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science, pages 207-216. IEEE, 2013.","title":"Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks","authors":"Badanidiyuru","year":"2013","venue":"IEEE 54th Annual Symposium on Foundations of Computer Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bandalouski et al., 2021] Andrei M Bandalouski, Natalja G Egorova, Mikhail Y Kovalyov, Erwin Pesch, and S Ar- magan Tarim. Dynamic pricing with demand disaggrega- tion for hotel revenue management. Journal of Heuristics, 27(5):869-885, 2021.","title":"Dynamic pricing with demand disaggregation for hotel revenue management","authors":"Andrei M Bandalouski, G Natalja, Mikhail Y Egorova, Erwin Kovalyov, S Armagan Pesch, Tarim","year":"2021","venue":"Journal of Heuristics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bitran and Caldentey, 2003] Gabriel Bitran and René Caldentey. An overview of pricing models for revenue management. Manufacturing & Service Operations Management, 5(3):203-229, 2003.","title":"Gabriel Bitran and René Caldentey. An overview of pricing models for revenue management","authors":"Caldentey Bitran","year":"2003","venue":"Manufacturing & Service Operations Management","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022] Boxiao Chen, Menglong Li, and David Simchi-Levi. Dynamic pricing with infrequent inventory replenishments. Available at SSRN 4240137, 2022.","title":"Dynamic pricing with infrequent inventory replenishments","authors":"Chen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chiang et al., 2007] Wen-Chyuan Chiang, Jason CH Chen, and Xiaojing Xu. An overview of research on revenue management: current issues and future research. Interna- tional journal of revenue management, 1(1):97-128, 2007.","title":"An overview of research on revenue management: current issues and future research","authors":"Wen-Chyuan Chiang, Jason Ch Chen, Xiaojing Xu","year":"2007","venue":"International journal of revenue management","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[den Boer and Zwart, 2015] Arnoud V den Boer and Bert Zwart. Dynamic pricing and learning with finite inven- tories. Operations research, 63(4):965-978, 2015.","title":"Dynamic pricing and learning with finite inventories","authors":"Zwart ; Arnoud V Den Den Boer, Bert Boer, Zwart","year":"2015","venue":"Operations research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Den Boer, 2015] Arnoud V Den Boer. Dynamic pricing and learning: historical origins, current research, and new di- rections. Surveys in operations research and management science, 20(1):1-18, 2015.","title":"Dynamic pricing and learning: historical origins, current research, and new directions","authors":"Arnoud V Den Boer","year":"2015","venue":"Surveys in operations research and management science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ferreira et al., 2018] Kris Johnson Ferreira, David Simchi- Levi, and He Wang. Online network revenue manage- ment using thompson sampling. Operations research, 66(6):1586-1602, 2018.","title":"Online network revenue management using thompson sampling","authors":"Kris Johnson Ferreira, David Simchi-Levi, He Wang","year":"2018","venue":"Operations research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gallego and Van Ryzin, 1994] Guillermo Gallego and Gar- rett Van Ryzin. Optimal dynamic pricing of inventories with stochastic demand over finite horizons. Management science, 40(8):999-1020, 1994.","title":"Optimal dynamic pricing of inventories with stochastic demand over finite horizons","authors":"Van Gallego, ; Ryzin, Garrett Gallego, Van Ryzin","year":"1994","venue":"Management science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gallego and Van Ryzin, 1997] Guillermo Gallego and Gar- rett Van Ryzin. A multiproduct dynamic pricing problem and its applications to network yield management. Opera- tions research, 45(1):24-41, 1997.","title":"A multiproduct dynamic pricing problem and its applications to network yield management","authors":"Van Gallego, ; Ryzin, Van Ryzin","year":"1997","venue":"Operations research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Garber, 2017] Dan Garber. Efficient online linear optimiza- tion with approximation algorithms. In Advances in Neu- ral Information Processing Systems, volume 30, 2017.","title":"Efficient online linear optimization with approximation algorithms","authors":"Dan Garber","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hao and Lattimore, 2022] Botao Hao and Tor Lattimore. Regret bounds for information-directed reinforcement learning. In Advances in Neural Information Processing Systems, volume 35, pages 28575-28587, 2022.","title":"Botao Hao and Tor Lattimore. Regret bounds for information-directed reinforcement learning","authors":"Lattimore Hao","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Immorlica et al., 2022] Nicole Immorlica, Karthik Sankararaman, Robert Schapire, and Aleksandrs Slivkins. Adversarial bandits with knapsacks. Journal of the ACM, 69(6):1-47, 2022.","title":"Adversarial bandits with knapsacks","authors":"Nicole Immorlica, Karthik Sankararaman, Robert Schapire, Aleksandrs Slivkins","year":"2022","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Klein et al., 2020] Robert Klein, Sebastian Koch, Claudius Steinhardt, and Arne K. Strauss. A review of revenue man- agement: Recent generalizations and advances in industry applications. European Journal of Operational Research, 284(2):397-412, 2020.","title":"A review of revenue management: Recent generalizations and advances in industry applications","authors":"Robert Klein, Sebastian Koch, Claudius Steinhardt, Arne K Strauss","year":"2020","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lattimore and Szepesvári, 2020] Tor Lattimore and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020.","title":"Tor Lattimore and Csaba Szepesvári. Bandit algorithms","authors":"Szepesvári Lattimore","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lazarev, 2013] John Lazarev. The welfare effects of in- tertemporal price discrimination: an empirical analysis of airline pricing in us monopoly markets. New York Univer- sity, 2013.","title":"The welfare effects of intertemporal price discrimination: an empirical analysis of airline pricing in us monopoly markets","authors":"John Lazarev, Lazarev","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2022] Shang Liu, Jiashuo Jiang, and Xiaocheng Li. Non-stationary bandits with knapsacks. Advances in Neural Information Processing Systems, 35:16522-16532, 2022.","title":"Non-stationary bandits with knapsacks","authors":"Shang Liu, Jiashuo Jiang, Xiaocheng Li","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lu and Van Roy, 2019] Xiuyuan Lu and Benjamin Van Roy. Information-theoretic confidence bounds for reinforce- ment learning. In Advances in Neural Information Pro- cessing Systems, volume 32, 2019.","title":"Xiuyuan Lu and Benjamin Van Roy. Information-theoretic confidence bounds for reinforcement learning","authors":"Lu, Van Roy","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Malighetti et al., 2009] Paolo Malighetti, Stefano Paleari, and Renato Redondi. Pricing strategies of low-cost air- lines: The ryanair case study. Journal of Air Transport Management, 15(4):195-203, 2009.","title":"Pricing strategies of low-cost airlines: The ryanair case study","authors":"Paolo Malighetti, Stefano Paleari, Renato Redondi","year":"2009","venue":"Journal of Air Transport Management","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Moradipari et al., 2023] Ahmadreza Moradipari, Moham- mad Pedramfar, Modjtaba Shokrian Zini, and Vaneet Ag- garwal. Improved Bayesian regret bounds for Thomp- son sampling in reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 23557-23569, 2023.","title":"Ahmadreza Moradipari, Mohammad Pedramfar, Modjtaba Shokrian Zini, and Vaneet Aggarwal. Improved Bayesian regret bounds for Thompson sampling in reinforcement learning","authors":"Moradipari","year":"2023","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Osband and Van Roy, 2017] Ian Osband and Benjamin Van Roy. Why is posterior sampling better than optimism for reinforcement learning? In International Conference on Machine Learning, pages 2701-2710. PMLR, 2017.","title":"Why is posterior sampling better than optimism for reinforcement learning?","authors":"Van Osband, Ian Roy, Benjamin Osband, Roy Van","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rasmussen and Williams, 2005] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 11 2005.","title":"Gaussian Processes for Machine Learning","authors":"Carl Edward Rasmussen, Christopher K I Williams","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Russo and Van Roy, 2014] Daniel Russo and Benjamin Van Roy. Learning to optimize via posterior sampling. Mathematics of Operations Research, 39(4):1221-1243, 2014.","title":"Learning to optimize via posterior sampling","authors":"Van Russo, Daniel Roy, Benjamin Russo, Roy Van","year":"2014","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Strauss et al., 2018] Arne K. Strauss, Robert Klein, and Claudius Steinhardt. A review of choice-based revenue management: Theory and methods. European Journal of Operational Research, 271(2):375-387, 2018.","title":"A review of choice-based revenue management: Theory and methods","authors":"Arne K Strauss, Robert Klein, Claudius Steinhardt","year":"2018","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Su, 2007] Xuanming Su. Intertemporal pricing with strate- gic customer behavior. Management Science, 53(5):726- 741, 2007.","title":"Xuanming Su. Intertemporal pricing with strategic customer behavior","authors":"Su","year":"2007","venue":"Management Science","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Thompson, 1933] William R Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285- 294, 1933.","title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","authors":"William R Thompson","year":"1933","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wen et al., 2017] Zheng Wen, Branislav Kveton, Michal Valko, and Sharan Vaswani. Online influence maximiza- tion under independent cascade model with semi-bandit feedback. In Advances in Neural Information Processing Systems, volume 30, 2017.","title":"Online influence maximization under independent cascade model with semi-bandit feedback","authors":"Zheng Wen, Branislav Kveton, Michal Valko, Sharan Vaswani","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Williams, 2020] Kevin R Williams. Dynamic airline pricing and seat availability. Technical report, Cowles Foundation for Research in Economics, Yale University, 2020.","title":"Dynamic airline pricing and seat availability","authors":"Kevin R Williams","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhao and Zheng, 2000] Wen Zhao and Yu-Sheng Zheng. Optimal dynamic pricing for perishable assets with non- homogeneous demand. Management science, 46(3):375- 388, 2000.","title":"Optimal dynamic pricing for perishable assets with nonhomogeneous demand","authors":"Wen Zhao, Yu-Sheng Zheng","year":"2000","venue":"Management science","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}