{"type":"ScholarlyArticle","title":"Split Q Learning: Reinforcement Learning with Two-Stream Rewards","authors":[{"name":"Baihan Lin","slug":"baihan-lin","given":"Baihan","family":"Lin","affiliation":"Center for Theoretical Neuroscience , Columbia University , New York , NY 10027 , USA"},{"name":"Djallel Bouneffouf","slug":"djallel-bouneffouf","given":"Djallel","family":"Bouneffouf","affiliation":"IBM Thomas J. Watson Research Center , Yorktown Heights , NY 10598 , USA"},{"name":"Guillermo Cecchi","slug":"guillermo-cecchi","given":"Guillermo","family":"Cecchi","affiliation":"IBM Thomas J. Watson Research Center , Yorktown Heights , NY 10598 , USA"}],"abstract":"Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for a reinforcement learning problem, which extends the standard Q-learning approach to incorporate a two-stream framework of reward processing with biases biologically associated with several neurological and psychiatric conditions, including Parkinson's and Alzheimer's diseases, attention-deficit/hyperactivity disorder (ADHD), addiction, and chronic pain. For AI community, the development of agents that react differently to different types of rewards can enable us to understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems. Moreover, from the behavioral modeling perspective, our parametric framework can be viewed as a first step towards a unifying computational model capturing reward processing abnormalities across multiple mental conditions and user preferences in long-term recommendation systems.","doi":"10.24963/ijcai.2019/913","url":"https://ijcai.org/proceedings/2019/913","pdf":"https://ijcai.org/proceedings/2019/0913.pdf","markdown":"https://ijcai.org/proceedings/2019/913.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/913","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":913,"pages":{"first":6448,"last":6449},"keywords":["Machine Learning: Reinforcement Learning","Machine Learning Applications: Applications of Reinforcement Learning","Humans and AI: Cognitive Modeling","Humans and AI: Brain Sciences"],"videos":[],"references":[{"position":1,"raw":"[Abbeel and Ng, 2004] Pieter Abbeel and Andrew Y Ng. Appren- ticeship learning via inverse reinforcement learning. In Pro- ceedings of the twenty-first international conference on Machine learning, page 1. ACM, 2004.","title":"Apprenticeship learning via inverse reinforcement learning","authors":"Ng Abbeel, Pieter Abbeel, Andrew Y Ng","year":"2004","venue":"Proceedings of the twenty-first international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bouneffouf et al., 2017] Djallel Bouneffouf, Irina Rish, and Guillermo A Cecchi. Bandit models of human behavior: Reward processing in mental disorders. In International Conference on Artificial General Intelligence, pages 237-248. Springer, 2017.","title":"Bandit models of human behavior: Reward processing in mental disorders","authors":"Bouneffouf","year":"2017","venue":"International Conference on Artificial General Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Frank et al., 2004] Michael J Frank, Lauren C Seeberger, and Ran- dall C O'reilly. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306(5703):1940-1943, 2004.","title":"By carrot or by stick: cognitive reinforcement learning in parkinsonism","authors":"J Michael, Lauren C Frank, Randall C O' Seeberger, Reilly","year":"2004","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Perry and Kramer, 2015] David C Perry and Joel H Kramer. Re- ward processing in neurodegenerative disease. Neurocase, 21(1):120-133, 2015.","title":"Reward processing in neurodegenerative disease","authors":"C David, Joel H Perry, Kramer","year":"2015","venue":"Neurocase","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Redish et al., 2007] A David Redish, Steve Jensen, Adam John- son, and Zeb Kurth-Nelson. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychological review, 114(3):784, 2007.","title":"Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling","authors":"David Redish, Steve Jensen, Adam Johnson, Zeb Kurth-Nelson","year":"2007","venue":"Psychological review","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Srinivas et al., 2009] Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995, 2009.","title":"Gaussian process optimization in the bandit setting: No regret and experimental design","authors":"Niranjan Srinivas, Andreas Krause, Matthias Sham M Kakade, Seeger","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}