{"type":"ScholarlyArticle","title":"Playing Repeated Network Interdiction Games with Semi-Bandit Feedback","authors":[{"name":"Qingyu Guo","slug":"qingyu-guo","given":"Qingyu","family":"Guo","affiliation":"Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly , NTU , Singapore; ,"},{"name":"Bo An","slug":"bo-an","given":"Bo","family":"An","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"Long Tran-Thanh","slug":"long-tran-thanh","given":"Long","family":"Tran-Thanh","affiliation":"Electronics and Computer Science , University of Southampton , UK"}],"abstract":"We study repeated network interdiction games with no prior knowledge of the adversary and the environment, which can model many real world network security domains. Existing works often require plenty of available information for the defender and neglect the frequent interactions between both players, which are unrealistic and impractical, and thus, are not suitable for our settings. As such, we provide the first defender strategy, that enjoys nice theoretical and practical performance guarantees, by applying the adversarial online learning approach. In particular, we model the repeated network interdiction game with no prior knowledge as an online linear optimization problem, for which a novel and efficient online learning algorithm, SBGA, is proposed, which exploits the unique semi-bandit feedback in network security domains. We prove that SBGA achieves sublinear regret against adaptive adversary, compared with both the best fixed strategy in hindsight and a near optimal adaptive strategy. Extensive experiments also show that SBGA significantly outperforms existing approaches with fast convergence rate.","doi":"10.24963/ijcai.2017/515","url":"https://ijcai.org/proceedings/2017/515","pdf":"https://ijcai.org/proceedings/2017/0515.pdf","markdown":"https://ijcai.org/proceedings/2017/515.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/515","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":515,"pages":{"first":3682,"last":3690},"keywords":["Multidisciplinary Topics and Applications: AI&Security and Privacy","Agent-based and Multi-agent Systems: Noncooperative Games"],"videos":[],"references":[{"position":1,"raw":"[Abernethy et al., 2008] Jacob D. Abernethy, Elad Hazan, and Alexander Rakhlin. Competing in the dark: An ef- ficient algorithm for bandit linear optimization. In COLT, pages 263-274, 2008.","title":"Competing in the dark: An efficient algorithm for bandit linear optimization","authors":"Jacob D Abernethy, Elad Hazan, Alexander Rakhlin","year":"2008","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[An et al., 2013] Bo An, Matthew Brown, Yevgeniy Vorob- eychik, and Milind Tambe. Security games with surveil- lance cost and optimal timing of attack execution. In AA- MAS, pages 223-230, 2013.","title":"Security games with surveillance cost and optimal timing of attack execution","authors":"Bo An, Matthew Brown, Yevgeniy Vorobeychik, Milind Tambe","year":"2013","venue":"AA-MAS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Assimakopoulos, 1987] Nikitas Assimakopoulos. A net- work interdiction model for hospital infection control. Computers in Biology and Medicine, 17(6):413-422, 1987.","title":"A network interdiction model for hospital infection control","authors":"Nikitas Assimakopoulos","year":"1987","venue":"Computers in Biology and Medicine","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Auer et al., 2002] Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, and Robert E Schapire. The nonstochastic mul- tiarmed bandit problem. SIAM Journal on Computing, 32(1):48-77, 2002.","title":"The nonstochastic multiarmed bandit problem","authors":"Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, Robert E Schapire","year":"2002","venue":"SIAM Journal on Computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Awerbuch and Kleinberg, 2004] Baruch Awerbuch and Robert D. Kleinberg. Adaptive routing with end-to-end feedback: Distributed learning and geometric approaches. In STOC, pages 45-53, 2004.","title":"Adaptive routing with end-to-end feedback: Distributed learning and geometric approaches","authors":"Kleinberg Awerbuch, Baruch Awerbuch, Robert D Kleinberg","year":"2004","venue":"STOC","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bartlett et al., 2008] Peter L. Bartlett, Varsha Dani, Thomas P. Hayes, Sham Kakade, Alexander Rakhlin, and Ambuj Tewari. High-probability regret bounds for bandit online linear optimization. In COLT, pages 335-342, 2008.","title":"High-probability regret bounds for bandit online linear optimization","authors":"L Peter, Varsha Bartlett, Thomas P Dani, Sham Hayes, Alexander Kakade, Ambuj Rakhlin, Tewari","year":"2008","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Beittel, 2015] June","title":null,"authors":"Beittel","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"S. Beittel. Mexico: Organzied Crime and Drug Trafficking Organizations. Congressional Re- search Service, 2015.","title":"Mexico: Organzied Crime and Drug Trafficking Organizations","authors":"S Beittel","year":"2015","venue":"Congressional Research Service","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Blum et al., 2014] Avrim Blum, Nika Haghtalab, and Ariel D. Procaccia. Learning optimal commitment to over- come insecurity. In NIPS, pages 1826-1834, 2014.","title":"Learning optimal commitment to overcome insecurity","authors":"Avrim Blum, Nika Haghtalab, Ariel D Procaccia","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Camerer, 2003] Colin Camerer. Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, 2003.","title":"Behavioral Game Theory: Experiments in Strategic Interaction","authors":"Colin Camerer","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Cesa-Bianchi and Lugosi, 2006] Nicolò Cesa-Bianchi and Gábor Lugosi. Prediction, Learning, and Games. Cam- bridge University Press, 2006.","title":"Nicolò Cesa-Bianchi and Gábor Lugosi","authors":"Cesa-Bianchi, Lugosi","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[CTR., 2010] US DEPT OF JUSTICE NATL DRUG IN- TELLIGENCE CTR. National drug threat assessment, 2010.","title":"US DEPT OF JUSTICE NATL DRUG IN-TELLIGENCE CTR. National drug threat assessment","authors":"Ctr","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dani and Hayes, 2006] Varsha Dani and Thomas P Hayes. Robbing the bandit: Less regret in online geometric optimization against an adaptive adversary. In ACM- SIAM/SODA, pages 937-943. Society for Industrial and Applied Mathematics, 2006.","title":"Robbing the bandit: Less regret in online geometric optimization against an adaptive adversary","authors":"Dani Varsha, Thomas P Hayes","year":"2006","venue":"ACM-SIAM/SODA","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Dani et al., 2007] Varsha Dani, Thomas P. Hayes, and Sham Kakade. The price of bandit information for online opti- mization. In NIPS, pages 345-352, 2007.","title":"The price of bandit information for online optimization","authors":"Varsha Dani, Thomas P Hayes, Sham Kakade","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Guo et al., 2016a] Qingyu Guo, Bo An, Yevgeniy Vorobey- chik, Long Tran-Thanh, Jiarui Gan, and Chunyan Miao. Coalitional security games. In AAMAS, pages 159-167, 2016.","title":"Coalitional security games","authors":"Guo","year":"2016","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Guo et al., 2016b] Qingyu Guo, Bo An, Yair Zick, and Chunyan Miao. Optimal interdiction of illegal network flow. In IJCAI, pages 2507-2513, 2016.","title":"Optimal interdiction of illegal network flow","authors":"Guo","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/357"},{"position":17,"raw":"[Haskell et al., 2014] William B. Haskell, Debarun Kar, Fei Fang, Milind Tambe, Sam Cheung, and Elizabeth Denico- la. Robust protection of fisheries with compass. In AAAI, pages 2978-2983, 2014.","title":"Robust protection of fisheries with compass","authors":"William B Haskell, Debarun Kar, Fei Fang, Milind Tambe, Sam Cheung, Elizabeth Denicola","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Jain et al., 2011] Manish Jain, Dmytro Korzhyk, Ondrej Vanek, Vincent Conitzer, Michal Pechoucek, and Milind Tambe. A double oracle algorithm for zero-sum security games on graphs. In AAMAS, pages 327-334, 2011.","title":"A double oracle algorithm for zero-sum security games on graphs","authors":"Manish Jain, Dmytro Korzhyk, Ondrej Vanek, Vincent Conitzer, Michal Pechoucek, Milind Tambe","year":"2011","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kakade et al., 2009] Sham M. Kakade, Adam Tauman Kalai, and Katrina Ligett. Playing games with approxi- mation algorithms. SIAM J. Comput., 39(3):1088-1106, 2009.","title":"Playing games with approximation algorithms","authors":"M Sham, Adam Kakade, Katrina Tauman Kalai, Ligett","year":"2009","venue":"SIAM J. Comput","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kalai and Vempala, 2003] Adam Kalai and Santosh Vem- pala. Efficient algorithms for online decision problems. In COLT, pages 26-40, 2003.","title":"Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems","authors":"Vempala Kalai","year":"2003","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kar et al., 2015] Debarun Kar, Fei Fang, Francesco Mari- a Delle Fave, Nicole Sintov, and Milind Tambe. \"a game of thrones\": When human behavior models compete in repeated Stackelberg security games. In AAMAS, pages 1381-1390, 2015.","title":"a game of thrones\": When human behavior models compete in repeated Stackelberg security games","authors":"Kar","year":"2015","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[McFadden, 1976] Daniel L McFadden. Quantal choice analaysis: A survey. In Annals of Economic and Social Measurement, Volume 5, number 4, pages 363-390. 1976.","title":"Quantal choice analaysis: A survey","authors":"Daniel L Mcfadden","year":"1976","venue":"Annals of Economic and Social Measurement","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[McMahan and Blum, 2004] H. Brendan McMahan and Avrim Blum. Online geometric optimization in the ban- dit setting against an adaptive adversary. In COLT, pages 109-123, 2004.","title":"Online geometric optimization in the bandit setting against an adaptive adversary","authors":"H, Brendan Mcmahan, Avrim Blum","year":"2004","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Neu and Bartók, 2015] Gergely Neu and Gábor Bartók. Im- portance weighting without importance weights: An effi- cient algorithm for combinatorial semi-bandits. CoRR, ab- s/1503.05087, 2015.","title":"Importance weighting without importance weights: An efficient algorithm for combinatorial semi-bandits","authors":"Bartók Neu, Gergely Neu, Gábor Bartók","year":"2015","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Nguyen et al., 2013] Thanh Hong Nguyen, Rong Yang, Amos Azaria, Sarit Kraus, and Milind Tambe. Analyzing the effectiveness of adversary modeling in security games. In AAAI, 2013.","title":"Analyzing the effectiveness of adversary modeling in security games","authors":"Thanh Hong Nguyen, Rong Yang, Amos Azaria, Sarit Kraus, Milind Tambe","year":"2013","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Office, 2009] United States Government Accountability Of- fice. Border patrol: Checkpoints contribute to border patrols mission, but more consistent data collection and performance measurement could improve effectiveness, 2009.","title":"Border patrol: Checkpoints contribute to border patrols mission, but more consistent data collection and performance measurement could improve effectiveness","authors":null,"year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Streeter and Golovin, 2008] Matthew J. Streeter and Daniel Golovin. An online algorithm for maximizing submodular functions. In NIPS, pages 1577-1584, 2008.","title":"An online algorithm for maximizing submodular functions","authors":"Matthew J Streeter, Daniel Golovin","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2016] Zhen Wang, Yue Yin, and Bo An. Com- puting optimal monitoring strategy for detecting terrorist plots. In AAAI, pages 637-643, 2016.","title":"Computing optimal monitoring strategy for detecting terrorist plots","authors":"Zhen Wang, Yue Yin, Bo An","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2017] Xinrun Wang, Qingyu Guo, and Bo An. Stop nuclear smuggling through efficient container inspec- tion. In AAMAS, pages 669-677, 2017.","title":"Stop nuclear smuggling through efficient container inspection","authors":"Wang","year":"2017","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Waxman, 1988] Bernard M Waxman. Routing of multipoint connections. IEEE Journal on Selected Areas in Commu- nications, 6(9):1617-1622, 1988.","title":"Routing of multipoint connections","authors":"Bernard M Waxman","year":"1988","venue":"IEEE Journal on Selected Areas in Communications","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wood, 1993] R Kevin Wood. Deterministic network inter- diction. Mathematical and Computer Modelling, 17(2):1- 18, 1993.","title":"Deterministic network interdiction","authors":"Kevin Wood","year":"1993","venue":"Mathematical and Computer Modelling","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xu et al., 2016] Haifeng Xu, Long Tran-Thanh, and Nicholas R. Jennings. Playing repeated security games with no prior knowledge. In AAMAS, pages 104-112, 2016.","title":"Playing repeated security games with no prior knowledge","authors":"Haifeng Xu, Long Tran-Thanh, Nicholas R Jennings","year":"2016","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yang et al., 2014] Rong Yang, Benjamin J. Ford, Milind Tambe, and Andrew Lemieux. Adaptive resource alloca- tion for wildlife protection against illegal poachers. In AA- MAS, pages 453-460, 2014.","title":"Adaptive resource allocation for wildlife protection against illegal poachers","authors":"Rong Yang, Benjamin J Ford, Milind Tambe, Andrew Lemieux","year":"2014","venue":"AA-MAS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yin and An, 2016] Yue Yin and Bo An. Efficient resource allocation for protecting coral reef ecosystems. In IJCAI, pages 531-537, 2016.","title":"Efficient resource allocation for protecting coral reef ecosystems","authors":"Yue Yin, Bo An","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/82"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}