Logistic Markov Decision Processes

Logistic Markov Decision Processes

Martin Mladenov, Craig Boutilier, Dale Schuurmans, Ofer Meshi, Gal Elidan, Tyler Lu

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 2486-2493. https://doi.org/10.24963/ijcai.2017/346

User modeling in advertising and recommendation has typically focused on myopic predictors of user responses. In this work, we consider the long-term decision problem associated with user interaction. We propose a concise specification of long-term interaction dynamics by combining factored dynamic Bayesian networks with logistic predictors of user responses, allowing state-of-the-art prediction models to be seamlessly extended. We show how to solve such models at scale by providing a constraint generation approach for approximate linear programming that overcomes the variable coupling and non-linearity induced by the logistic regression predictor. The efficacy of the approach is demonstrated on advertising domains with up to 2^54 states and 2^39 actions.
Keywords:
Machine Learning: Classification
Machine Learning: Reinforcement Learning
Planning and Scheduling: Markov Decisions Processes