{"type":"ScholarlyArticle","title":"Game Redesign in No-regret Game Playing","authors":[{"name":"Yuzhe Ma","slug":"yuzhe-ma","given":"Yuzhe","family":"Ma","affiliation":"University of Wisconsin-Madison"},{"name":"Young Wu","slug":"young-wu","given":"Young","family":"Wu","affiliation":"University of Wisconsin-Madison"},{"name":"Xiaojin Zhu","slug":"xiaojin-zhu","given":"Xiaojin","family":"Zhu","affiliation":"University of Wisconsin-Madison"}],"abstract":"We study the game redesign problem in which an external designer has the ability to change the payoff function in each round, but incurs a design cost for deviating from the original game. The players apply no-regret learning algorithms to repeatedly play the changed games with limited feedback. The goals of the designer are to (i) incentivize players to take a specific target action profile frequently; (ii) incur small cumulative design cost. We present game redesign algorithms with the guarantee that the target action profile is played in T-o(T) rounds while incurring only o(T) cumulative design cost. Simulations on four classic games confirm the ef- fectiveness of our proposed redesign algorithms.","doi":"10.24963/ijcai.2022/461","url":"https://ijcai.org/proceedings/2022/461","pdf":"https://ijcai.org/proceedings/2022/0461.pdf","markdown":"https://ijcai.org/proceedings/2022/461.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/461","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Machine Learning","track":"Main Track","number":461,"pages":{"first":3321,"last":3327},"keywords":["Agent-based and Multi-agent Systems: Multi-agent Learning","Machine Learning: Adversarial Machine Learning"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38984956","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38986813","length_sec":350,"provider":"slideslive"}],"references":[{"position":1,"raw":"[BLUM, 2007] A BLUM. Learning, regret minimization, and equilibria. Algorithmic Game Theory, pages 79-102, 2007.","title":"Learning, regret minimization, and equilibria","authors":"A Blum","year":"2007","venue":"Algorithmic Game Theory","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bogunovic et al., 2021] Ilija Bogunovic, Arpan Losalka, Andreas Krause, and Jonathan Scarlett. Stochastic lin- ear bandits robust to adversarial attacks. In International Conference on Artificial Intelligence and Statistics, pages 991-999. PMLR, 2021.","title":"Stochastic linear bandits robust to adversarial attacks","authors":"Ilija Bogunovic, Arpan Losalka, Andreas Krause, Jonathan Scarlett","year":"2021","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bubeck and Cesa-Bianchi, 2012] Sébastien Bubeck and Nicolò Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends R in Machine Learning, 5(1):1-122, 2012.","title":"Regret analysis of stochastic and nonstochastic multi-armed bandit problems","authors":"Cesa-Bianchi ; Bubeck, Sébastien Bubeck, Nicolò Cesa-Bianchi","year":"2012","venue":"Foundations and Trends R in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Garcelon et al., 2020] Evrard Garcelon, Baptiste Roziere, Laurent Meunier, Olivier Teytaud, Alessandro Lazaric, and Matteo Pirotta. Adversarial attacks on linear contex- tual bandits. arXiv preprint arXiv:2002.03839, 2020.","title":"Adversarial attacks on linear contextual bandits","authors":"Evrard Garcelon, Baptiste Roziere, Laurent Meunier, Olivier Teytaud, Alessandro Lazaric, Matteo Pirotta","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Guan et al., 2020] Ziwei Guan, Kaiyi Ji, Donald J Bucci Jr, Timothy Y Hu, Joseph Palombo, Michael Liston, and Yingbin Liang. Robust stochastic bandit algorithms under probabilistic unbounded adversarial attack. arXiv preprint arXiv:2002.07214, 2020.","title":"Robust stochastic bandit algorithms under probabilistic unbounded adversarial attack","authors":"Kaiyi Ziwei Guan, Donald J Ji, Timothy Y Bucci, Joseph Hu, Michael Palombo, Yingbin Liston, Liang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hart and Mas-Colell, 2000] Sergiu Hart and Andreu Mas- Colell. A simple adaptive procedure leading to correlated equilibrium. Econometrica, 68(5):1127-1150, 2000.","title":"A simple adaptive procedure leading to correlated equilibrium","authors":"Sergiu Hart, Andreu Mas-Colell","year":"2000","venue":"Econometrica","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang and Zhu, 2019] Yunhan Huang and Quanyan Zhu. Deceptive reinforcement learning under adversarial ma- nipulations on cost signals. In International Conference on Decision and Game Theory for Security, pages 217- 237. Springer, 2019.","title":"Deceptive reinforcement learning under adversarial manipulations on cost signals","authors":"Zhu ; ] Yunhan Huang, Quanyan Huang, Zhu","year":"2019","venue":"International Conference on Decision and Game Theory for Security","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jun et al., 2018] Kwang-Sung Jun, Lihong Li, Yuzhe Ma, and Xiaojin Zhu. Adversarial attacks on stochastic ban- dits. In Advances in Neural Information Processing Sys- tems, pages 3640-3649, 2018.","title":"Adversarial attacks on stochastic bandits","authors":"Kwang-Sung Jun, Lihong Li, Yuzhe Ma, Xiaojin Zhu","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu and Shroff, 2019] Fang Liu and Ness Shroff. Data poi- soning attacks on stochastic bandits. In International Con- ference on Machine Learning, pages 4042-4050, 2019.","title":"Data poisoning attacks on stochastic bandits","authors":"Shroff Liu, Fang Liu, Ness Shroff","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lu et al., 2021] Shiyin Lu, Guanghui Wang, and Lijun Zhang. Stochastic graphical bandits with adversarial cor- ruptions. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, volume 35, pages 8749-8757, 2021.","title":"Stochastic graphical bandits with adversarial corruptions","authors":"Shiyin Lu, Guanghui Wang, Lijun Zhang","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ma et al., 2018] Yuzhe Ma, Kwang-Sung Jun, Lihong Li, and Xiaojin Zhu. Data poisoning attacks in contextual ban- dits. In International Conference on Decision and Game Theory for Security, pages 186-204. Springer, 2018.","title":"Data poisoning attacks in contextual bandits","authors":"Yuzhe Ma, Kwang-Sung Jun, Lihong Li, Xiaojin Zhu","year":"2018","venue":"International Conference on Decision and Game Theory for Security","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ma et al., 2019] Yuzhe Ma, Xuezhou Zhang, Wen Sun, and Xiaojin Zhu. Policy poisoning in batch reinforcement learning and control. In Advances in Neural Information Processing Systems, pages 14570-14580, 2019.","title":"Policy poisoning in batch reinforcement learning and control","authors":"Yuzhe Ma, Xuezhou Zhang, Wen Sun, Xiaojin Zhu","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Monderer and Tennenholtz, 2004] Dov Monderer and Moshe Tennenholtz. k-implementation. Journal of Artificial Intelligence Research, 21:37-62, 2004.","title":"Dov Monderer and Moshe Tennenholtz. k-implementation","authors":"Tennenholtz Monderer","year":"2004","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rakhsha et al., 2020] Amin Rakhsha, Goran Radanovic, Rati Devidze, Xiaojin Zhu, and Adish Singla. Policy teaching via environment poisoning: Training-time ad- versarial attacks against reinforcement learning. arXiv preprint arXiv:2003.12909, 2020.","title":"Policy teaching via environment poisoning: Training-time adversarial attacks against reinforcement learning","authors":"Amin Rakhsha, Goran Radanovic, Rati Devidze, Xiaojin Zhu, Adish Singla","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Slivkins, 2019] Aleksandrs Slivkins. Introduction to multi- armed bandits. arXiv preprint arXiv:1904.07272, 2019.","title":"Introduction to multiarmed bandits","authors":"Aleksandrs Slivkins","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Sun et al., 2020] Yanchao Sun, Da Huo, and Furong Huang. Vulnerability-aware poisoning mechanism for online rl with unknown dynamics. arXiv preprint arXiv:2009.00774, 2020.","title":"Vulnerability-aware poisoning mechanism for online rl with unknown dynamics","authors":"Sun","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Yang et al., 2021] Lin Yang, Mohammad Hajiesmaili, Mo- hammad Sadegh Talebi, John Lui, and Wing Shing Wong. Adversarial bandits with corruptions: Regret lower bound and no-regret algorithm. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2021.","title":"Adversarial bandits with corruptions: Regret lower bound and no-regret algorithm","authors":"Lin Yang, Mohammad Hajiesmaili, Mohammad Sadegh Talebi, John Lui, Wing Shing Wong","year":"2021","venue":"Advances in Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zhang et al., 2020] Xuezhou Zhang, Yuzhe Ma, Adish Singla, and Xiaojin Zhu. Adaptive reward-poisoning attacks against reinforcement learning. arXiv preprint arXiv:2003.12613, 2020.","title":"Adaptive reward-poisoning attacks against reinforcement learning","authors":"Zhang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Zuo, 2020] Shiliang Zuo. Near optimal adversarial attack on ucb bandits. arXiv preprint arXiv:2008.09312, 2020.","title":"Near optimal adversarial attack on ucb bandits","authors":"Shiliang Zuo, Zuo","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}