When Security Games Go Green: Designing Defender Strategies to Prevent Poaching and Illegal Fishing

1 Introduction

Poaching and illegal over-fishing are critical international problems leading to destruction of ecosystems. For example, three out of nine tiger species have gone extinct in the past 100 years and others are now endangered due to poaching [Secretariat, 2013]. Law enforcement agencies in many countries are hence challenged with applying their limited resources to protecting endangered animals and fish stocks.

Building upon the success of applying SSGs to protect infrastructure including airports [Pita et al., 2008], ports [Shieh et al., 2012] and trains [Yin et al., 2012], researchers are now applying game theory to green security domains, e.g., protecting fisheries from over-fishing [Brown et al., 2014; Haskell et al., 2014] and protecting wildlife from poaching [Yang et al., 2014]. There are several key features in green security domains that introduce novel research challenges. First, the defender is faced with multiple adversaries who carry out repeated and frequent illegal activities (attacks), yielding a need to go beyond the one-shot SSG model. Second, in carrying out such frequent attacks, the attackers generally do not conduct extensive surveillance before performing an attack and spend less time and effort in each attack, and thus it becomes more important to model the attackers' bounded rationality and bounded surveillance. Third, there is more attack data available in green security domains than in infrastructure security domains, which makes it possible to learn the attackers' decision making model from data.

Previous work in green security domains [Yang et al., 2014; Haskell et al., 2014] models the problem as a game with multiple rounds and each round is a SSG [Yin et al., 2010] where the defender commits to a mixed strategy and the attackers respond to it. In addition, they address the bounded rationality of attackers using the SUQR model [Nguyen et al., 2013]. While such advances have allowed these works to be tested in the field, there are three key weaknesses in these efforts. First, the Stackelberg assumption in these works - that the defender's mixed strategy is fully observed by the attacker via extensive surveillance before each attack - can be unrealistic in green security domains as mentioned above. Indeed, the attacker may experience a delay in observing how the defender strategy may be changing over time, from round to round. Second, since the attacker may lag in observing the defender's strategy, it may be valuable for the defender to plan ahead; however these previous efforts do not engage in any planning and instead rely only on designing strategies for the current round. Third, while they do exploit the available attack data, they use Maximum Likelihood Estimation (MLE) to learn the parameters of the SUQR model for individual attackers which we show may lead to skewed results.

In this paper, we offer remedies for these limitations. First, we introduce a novel model called Green Security Games (GSGs). Generalizing the perfect Stackelberg assumption, GSGs assume that the attackers' understanding of the defender strategy may not be up-to-date and can be instead approximated as a convex combination of the defender strategies used in recent rounds. Previous models in green security domains, e.g., such as [Yang et al., 2014; Haskell et al., 2014] can be seen as a special case of GSGs, as they assume that the attackers always have up-to-date information, whereas GSGs allow for more generality and hence planning of defender strategies.

Second, we provide two algorithms that plan ahead - the generalization of the Stackelberg assumption introduces a need to plan ahead and take into account the effect of defender strategy on future attacker decisions. While the first algorithm plans a fixed number of steps ahead, the second one designs a short sequence of strategies for repeated execution.

Third, the paper also provides a novel framework that incorporates learning of parameters in the attackers' bounded rationality model into the planning algorithms where, instead of using MLE as in past work, we use insights from Bayesian updating. All proposed algorithms are fully implemented and we provide detailed empirical results.

2 Motivation and Defining GSGs

Our motivating example assumes a perfectly rational attacker purely for simplicity of exposition. In the rest of the paper, we consider attackers with bounded rationality.

Example 1. Consider a ranger protecting a large area with rhinos. The area is divided into two subareas N1 and N2 of the same importance. The ranger chooses a subarea to guard every day and she can stop any snaring by poachers

in the guarded area. The ranger has been using a uniform random strategy throughout last year. So for this January, she can choose to continue using the uniform strategy throughout the month, catching 50% of the snares. But now assume that the poachers change their strategy every two weeks based on the most recently observed ranger strategy. In this case, the ranger can catch 75% of the snares by always protecting N1 in the first two weeks of January, and then switching to always protecting N2: At the beginning of January, the poachers are indifferent between the two subareas due to their observation from last year. Thus, 50% of the snares will be placed in N1 and the ranger can catch these snares in the first half of January by only protecting N1. But after observing the change in ranger strategy, the poachers will switch to only putting the snares in N2. The poachers' behavior change can be expected by the ranger and the ranger can catch 100% of the snares by only protecting N2 starting from mid-January. (Of course the poachers must then be expected to adapt further).

This example conceptually shows that the defender can benefit from planning strategy changes in green security domains. We now define GSG as an abstraction of the problem in green security domains (borrowing some terminology from Stackelberg Security Games [Yin et al., 2010]).

Definition 1. A GSG is a T(< ∞) round repeated game between a defender and L GSG attackers and (i) The defender has K guards to protect N(≥ K) targets. (ii) Each round has multiple episodes and in every episode, each guard can protect one target and each attacker can attack one target. (iii) In round t, the defender chooses a mixed strategy at the beginning of the round, which is a probability distribution over all pure strategies, i.e., N choose K assignments from the guards to targets. In every episode, the guards are assigned to targets according to an assignment randomly sampled from the mixed strategy. (iv) Each target i ∈ [N] has payoff values P a i , Ra i , P d i , Rd i ("P" for "Penalty", "R" for "Reward", "a" for "attacker" and "d" for "defender"). If an attacker attacks target i which is protected by a guard, the attacker gets utility P a i , and the defender gets Rd i . If target i is not protected, the attacker gets utility Ra i , and the defender gets P d i . Rd i > P d i and Ra i > P a i . (v) The defender's utility in round t is the total expected utility calculated over all attackers.

Each round of the repeated game corresponds to a period of time, which can be a time interval (e.g., a month) after which the defender (e.g., warden) communicate with local guards to assign them a new strategy. We divide each round into multiple episodes for the players to take actions.

Consistent with previous work on green security games [Yang et al., 2014; Haskell et al., 2014], we divide the protected area into subareas or grid cells and treat each subarea or cell as a target. Different targets may have different importance to the defender and the attackers due to differences in resource richness and accessibility. We therefore associate each target i ∈ [N] with payoff values. A mixed defender strategy can be represented compactly by a coverage vector c = 〈ci〉 where 0 ≤ ci ≤ 1 is the probability that target i is covered by some guard and it satisfies ∑Ni=1 ci ≤ K [Kiekintveld et al., 2009; Korzhyk et al., 2010]. If an attacker attacks target i, the expected utility for the defender is U d i (c) = ciRd i + (1 - ci)P d i given defender strategy c. We denote the mixed defender strategy in round t as ct .

Definition 2. A GSG attacker is characterized by his memory length Γ, coefficients α0, ...αΓ and his parameter vector ω. In round t, A GSG attacker with memory length Γ responds to a convex combination of the defender strategy in recent Γ + 1 rounds, i.e., he responds to ηt = ∑Γτ=0 ατ ct-τ where ∑Γτ=0 ατ = 1 and ct = c0 if t ≤ 0. In every episode of round t, a GSG attacker follows the SUQR model and chooses a random target to attack based on his parameter vector ω in the SUQR model.

We aim to provide automated decision aid to defenders in green security domains who defend against human adversaries such as poachers who have no automated tools - hence we model the poachers as being boundedly rational and having bounded memory. We approximate a GSG attacker's belief of the defender's strategy in round t as a convex combination of the defender strategy in the current round and the last Γ rounds. This is because the attackers may not be capable of knowing the defender's exact strategy when attacking; naturally, they will consider the information they get from the past. Further, human beings have bounded memory, and the attackers may tend to rely on recent information instead of the whole history. The Stackelberg assumption in [Yang et al., 2014; Haskell et al., 2014] can be seen as a special case of this approximation with α0 = 1. In this paper, we assume all attackers have the same memory length Γ, coefficients ατ and these values are known to the defender. c0 is the defender strategy used before the game starts and is known to players.

To model the bounded rationality of the human attackers such as poachers, we use the SUQR model, which has performed the best so far against human subjects in security games [Nguyen et al., 2013]. In this model, an attacker's choice is based on key properties of each target, including the coverage probability, the reward and the penalty, represented by the parameter vector ω = (ω1, ω2, ω3). Given η as the attacker's belief (with ηi the belief of the coverage probability on target i), the probability that an attacker with parameter ω attacks target i is

Following the work of Yang et. al [2014], in this paper, we assume the group of attackers may have heterogeneous weighting coefficients, i.e., each attacker l ∈ [L] is associated with a parameter vector ωl = (ωl1 , ωl2 , ωl3 ).

A GSG defender strategy profile [c] is defined as a sequence of defender strategies with length T, i.e., [c] = 〈c1 , ..., cT 〉. The defender's expected utility in round t is Et ([c]) = ∑l ∑i qi(ωl , ηt )U d i (ct ). The objective of the defender is to find the strategy profile with the highest average expected utility over all rounds, i.e., to maximize E([c]) = ∑Tt=1 Et ([c])/T .

3 Planning in GSGs

The defender can potentially improve her average expected utility by carefully planning changes in her strategy from round to round in a GSG. In this section, we consider the case where the attackers' parameter vectors ω1 , ..., ωL, are known to the defender. For clarity of exposition, we will first focus on the case where α0 = 0 and Γ = 1. This is the special case when the attackers have one round memory and have no information about the defender strategy in the current round, i.e., the attackers respond to the defender strategy in the last round. We discuss the more general case in Section 5.

To maximize her average expected utility, the defender could optimize over all rounds simultaneously. However, this approach is computationally expensive when T is large: it needs to solve a non-convex optimization problem with NT variables (cti ) as the defender must consider attacker response, and the attacking probability has a non-convex form (see Equation 1). An alternative is the myopic strategy, i.e., the defender can always protect the targets with the highest expected utility in the current round. However, this myopic choice may lead to significant quality degradation as it ignores the impact of ct in the next round.

Therefore, we propose an algorithm named PlanAhead-M (or PA-M) that looks ahead a few steps (see Algorithm 1).

PA-M finds an optimal strategy for the current round as if it is the M th last round of the game. If M = 2, the defender chooses a strategy assuming she will play a myopic strategy in the next round and end the game. When there are less than M - 1 future rounds, the defender only needs to look ahead T-t steps (Line 2). PA-T corresponds to the optimal solution and PA-1 is the myopic strategy. Unless otherwise specified, we choose 1 < M < T . Function f-PlanAhead(ct-1 , ω,m) solves the following mathematical program (MP).

This is a non-convex problem when m > 0 and can be solved approximately with local search approaches.

Although we show in the experiment section that PA-2 can provide significant improvement over baseline approaches in most cases, there exist settings where PA-2 can perform arbitrarily badly when compared to the optimal solution. The intuition is that the defender might make a suboptimal choice in the current round with an expectation to get a high reward in the next round; however, when she moves to the next round, she plans for two rounds again, and as a result, she never gets a high reward until the last round.

Example 2. Consider a guard protecting two subareas with payoff values shown on the right (X ≫ 1). For simplicity of the example, as-

sume the defender can only choose pure strategies. There is one poacher with a large negative coefficient on coverage probability, i.e., the poacher will always snare in the subarea that is not protected in the last round. The initial defender strategy is protecting N1, meaning the attacker will snare in N2 in round 1. According to PA-2, the defender will protect N1 in round 1 and plan to protect N2 in round 2, expecting a total utility of 3 + X. However, in round 2, the defender chooses N1 again as she assumes the game ends after round 3. Thus, her average expected utility is 3(T-1)+X T ≈ 3. On the other hand, if the defender alternates between N1 and N2, she gets a total utility of X+2 for two consecutive rounds and her average utility is at least X2 ≫ 3.

PA-2 fails in such cases because it over-estimates the utility in the future. To remedy this, we generalize PA-M to PA-M-γ by introducing a discount factor 0 < γ ≤ 1 for future rounds when T - t < M - 1, i.e., substituting Equation 2 with

While PA-M-γ presents an effective way to design sequential defender strategies, we provide another algorithm called FixedSequence-M (FS-M) for GSGs (see Algorithm 2). FSM not only has provable theoretical guarantees, but may also

Algorithm 2 Fixed Sequence

Output: defender strategy profile [c]

1: (a1 , ..., aM ) = f-FixedSequence(ω,M). 2: for t=1 to T do 3: ct = a(t mod M)+1

ease the implementation in practice. The idea of FS-M is to find a short sequence of strategies with fixed length M and require the defender to execute this sequence repeatedly. If M = 2, the defender will alternate between two strategies and she can exploit the attackers' delayed response. It can be easier to communicate with local guards to implement FS-M in green security domains as the guards only need to alternate between several types of maneuvers. Function f-FixedSequence(ω,M) calculates the best fixed sequence of length M through the following MP.

Theorem 1 shows that the solution to this MP provides a good approximation of the optimal defender strategy profile.

Theorem 1. In a GSG with T rounds, α0 = 0 and Γ = 1, for any fixed length 1 < M ≤ T, there exists a cyclic defender strategy profile [s] with period M that is a (1- 1M ) Z-1 Z+1 approximation of the optimal strategy profile in terms of the normalized utility, where Z = ⌈ TM ⌉.

We leave the detailed proof to the online appendix 1 . According to Theorem 1, when a GSG has many rounds (T ≫ M), the cyclic sequence constructed by repeating a1 , ...aM is a 1 - 1/M approximation.

4 Learning and Planning in GSGs

In Section 3, we assume that the parameter vectors ω1 , ..., ωL in the attackers' bounded rationality model are known. Since the defender may not know these parameter values precisely at the beginning of the game in practice, we now aim to learn the attackers' average parameter distribution from attack data. Previous work in green security domains [Yang et al., 2014; Haskell et al., 2014] treats each data point, e.g., each snare or fishnet, as an independent attacker and applies MLE to select the most probable parameter vector. However, some of the assumptions made in previous work in proposing MLE may not always hold as MLE works well when a large number of data samples are used to estimate one set of parameters [Eliason, 1993]. Here we show that estimating ω from a single data point using MLE can lead to highly biased results.

Example 3. Consider a guard protecting two targets in round 1. The payoff structure and initial defender strategy are shown in Table 2 where X ≫ 1 and 0 < δ ≪ 1. An attacker with parameter vector ω = (-1, 0, 0) will choose N1 or N2 with the probability ≈ 0.5, as ω1 = -1 means he has

1 http://ijcai2015cs.yolasite.com/

Algorithm 3 Learn-BU (η, χ,{ˆω}, p)

Output: ¯ p - a probability distribution over {ˆω} = {ˆω1 , ..., ˆωS }.

• 1: for i=1 to N do
• 2: for s=1 to S do
• 3: ¯ pi(s) = p(s)qi(ˆωs ,η) ∑r p(r)qi(ˆωr,η)
• 4: for s=1 to S do
• 5: ¯ ∑i χi
• p(s) = ∑i χi ¯ pi(s)

a slight preference on targets with lower coverage probability (see Equation 1). If the attacker attacks N1, applying MLE will lead to an estimation of ω = (+∞, ·, ·), meaning the attacker will always choose the target with higher coverage probability. This is because the probability of attacking N1 is 1 given ω1 = +∞, which is higher than that of any other parameter value. Similarly, if the attacker attacks N2, an extreme parameter of (-∞, ·, ·) is derived fromMLE. These extreme parameters will mislead the defender in designing her strategy in the following round.

We therefore leverage insights from Bayesian Updating. For each data point, we estimate a probability distribution over parameter values instead of selecting the ω vector that maximizes the likelihood of the outcome. This approach is also different from maximum a posteriori probability (MAP) because MAP still provides single value estimates, whereas Bayesian Updating uses distributions to summarize data.

Algorithm 3 describes the learning algorithm for one round of the game. Rather than learning single parameter values, one from each attack, we learn a probability distribution. The input of the algorithm includes the number of attacks χi found on each target i ∈ [N], the attackers' belief of the defender strategy η, and the prior distribution p = 〈p1, ..., pS〉 over a discrete set of parameter values {ˆω} = {ˆω1 , ..., ˆωS }, each of which is a 3-element tuple. If an attacker attacks target i, we can calculate the posterior distribution of this attacker's parameter by applying Bayes' rule based on the prior distribution p (Line 3). We then calculate the average posterior distribution ¯ p over all attackers (Line 5).

Based on Algorithm 3, we now provide a novel framework that incorporates the learning algorithm into PA-M(-γ) as shown in Algorithm 4. The input p1 is the prior distribution about the attackers' parameters before the game starts. This prior distribution is for the general population of attackers and we need to learn the distribution of the L attackers we are facing in one game. The main idea of the algorithm is to use the average posterior distribution calculated in round t (denoted as ¯ pt ) as the prior distribution in round t+1 (denoted as pt+1 ), i.e.,pt+1 = ¯ pt . Given prior pt , Function f-PlanAhead in Line 2 is calculated through Equation 2 - 5 by substituting Equation 3 with Et = L ∑s ∑i pt (s)qi(¯ωs , ct-1 )U d i (ct ). Note that there was no probability term in Equation 3 because there we know exactly the parameter values of the attackers. After we collect data in round t, we apply Learn-BU (Algorithm 3)

Algorithm 4 BU-PA-M-γ(p1 )

Output: Defender strategy profile 〈c1 , ...cT 〉.

• 1: for t=1 to T do
• 2: ct = f-PlanAhead(ct-1 , ω,min{T - t,M -1})
• 3: ¯ pt = Learn-BU(ct-1 , χt , {ˆω}, pt )
• 4: pt+1 = ¯ pt

again and update the prior for next round (Line 3). This is a simplification of the more rigorous process which enumerates the matchings (exponentially many) between the data points and attackers and updates the distribution of each attacker separately when the attack data is anonymous (the guard may only find the snares placed on ground without knowing the identity of the poacher).

When incorporating Algorithm 3 into FS-M, we divide the game into several stages, each containing more than M rounds, and only update the parameter distribution at the end of each stage. As FS-Mmay not achieve its average expected utility if only a part of the sequence is executed, updating the parameter distribution in every round may lead to low utility.

5 General Case

Generalization from Γ = 1 and α0 = 0 to Γ > 1 and/or α0 ∈ [0, 1] can be achieved via generalizing ηt . PA-M(-γ) can be calculated by substituting Constraint 4 with ητ = ∑Mk=0 αkcτ-k , and FS-M can be calculated by changing Constraints 9-10 accordingly. Theorem 2 shows the theoretical bound of FS-M with Γ > 1 and the proof is similar to that of Theorem 1 (see online appendix 1 for details).

Theorem 2. In a GSG with T rounds, for any fixed length Γ < M ≤ T, there exists a cyclic defender strategy profile [s] with period M that is a (1 - ΓM ) Z-1 Z+1 approximation of the optimal strategy profile in terms of the normalized utility, where Z = ⌈ T-Γ+1 M ⌉.

6 Experimental Results

We test all the proposed algorithms on GSGs motivated by scenarios in green security domains such as defending against poaching and illegal fishing. Each round corresponds to 30 days and each poacher/fisherman will choose a target to place snares/fishnets every day. All algorithms are implemented in MATLAB with the fmincon function used for solving MPs and tested on 2.4GHz CPU with 128 GB memory. All key differences noted are statistically significant (p < 0.05).

6.1 Planning Algorithms

We compare proposed planning algorithms PA-M(-γ) and FS-M with baseline approaches FS-1 and PA-1. FS-1 is equivalent to calculating the defender strategy with a perfect Stackelberg assumption, which is used in previous work [Yang et al., 2014; Haskell et al., 2014], as the defender uses the same strategy in every round and the attackers' belief coincides with the defender strategy. PA-1 is the myopic strategy which tries to maximize the defender's expected utility in the current round. We assume c0 is the MAXIMIN strategy.

We first consider the special case (α0 = 0, Γ = 1) and test on 32 game instances of 5 attackers, 3 targets, 1 guard and 100 rounds with random reward and penalty chosen from [0, 10] and [-10, 0] respectively (denoted as Game Set 1). We run 100 restarts for each MP. Figure 2(a) shows that PA-M(- γ) and FS-M significantly outperform FS-1 and PA-1 in terms of the defender's average expected utility (AEU). This means using the perfect Stackelberg assumption would be detrimental to the defender if the attackers respond to last round's strategy. For PA-M, adding a discount factor γ may improve the solution. Figure 2(b) shows FS-M takes much less time than PA-M overall as FS-M only needs to solve one MP throughout a game while PA-M solves a MP for each round.

We also test on 32 games with 100 attackers, 10 targets, 4 guards and 100 rounds (denoted as Game Set 2) in the special case (see Figure 2(c)). We set a 1-hour runtime limit for the algorithms and again, FS-M and PA-M(-γ) significantly outperform FS-1 and PA-1 in solution quality.

We then test general cases on Game Set 2. Figure 2(d) shows the defender's AEU with varying α0 when Γ = 1. In the extreme case of α0 = 1, i.e., the attackers have perfect knowledge of the current defender strategy, the problem reduces to a repeated Stackelberg game and all approaches provide similar solution quality. However, when α0 < 0.5, FS-2 and PA-2 provide significant improvement over FS-1 and PA1, indicating the importance of planning.

We further test the robustness of FS-2 when there is slight deviation in α0 with Γ = 1 (see Figure 3). For example, the value of 5.891 in the 2nd row, 1st column of the table is the defender's AEU when the actual α0 = 0 and the defender assumes (estimates) it to be 0.125 when calculating her strategies. Cells in the diagonal show the case when the estimation is accurate. Cells in the last row show results for baseline algorithm FS-1. FS-1 uses the Stackelberg assumption and thus the estimated value makes no difference. When the actual value slightly deviates from the defender's estimate (cells adjacent to the diagonal ones in the same column), the solution quality does not change much if the actual α0 > 0.5. When the actual α0 < 0.5, FS-2 outperforms FS-1 significantly even given the slight deviation.

In Figure 2(e), we compare algorithms assuming Γ = 2, α1 = α2 = 0.5 and α0 = 0. As expected, PA-M with M > 1 and FS-M with M > 2 significantly outperforms FS-1 and PA-1. The improvement of FS-2 over FS-1 is negligible, as any fixed sequence of length 2 can be exploited by the attackers with memory length = 2.

Figure 2(f) shows the solution quality of PA-M when the defender assumes the attackers' memory length is 3 but the actual value of Γ varies from 1 to 4. When Γ is slightly overestimated (actual Γ = 1 or 2), PA-M still significantly outperforms the baseline algorithm FS-1 and PA-1. However, when Γ is under-estimated (actual Γ = 4), the attackers have longer memory than the defender's estimate and thus the attackers can exploit the defender's planning. This observation suggests that it is more robust to over-estimate the attackers' memory length when there is uncertainty in Γ. We defer to future work to learn ατ and Γ from attack data.

6.2 Learning and Planning Framework

When the parameter vectors {ωl } are unknown, we compare Algorithm 3 with the baseline learning algorithm that uses MLE (denoted as MLE) when incorporated into planning algorithms. In each game of Game Set 2, we randomly choose {ωl } for the 100 attackers from a three-dimensional normal distribution with mean µ = (-17.81, 0.72, 0.47) and covari-

). We use BU to de-

note the case when an accurate prior (µ and Σ) is given to the defender. Recall that the defender plays against 100 attackers throughout a game, and BU aims to learn the parameter distribution of these 100 attackers. BU′ represents the case when the prior distribution is a slightly deviated estimation (a normal distribution with random µ′ and Σ′ satisfying ‖µi - µ′i‖ ≤ 5 and ‖Σ′ii - Σii‖ ≤ 5). KnownPara represents the case when the exact values of {ωl } are known to the defender. We set a time limit of 30 minutes for the planning algorithms. Figure 2(g) - 2(h) show that BU and BU′ significantly outperform MLE. Indeed, the solution quality of BU and BU′ is close to that of KnownPara, indicating the effectiveness of the proposed learning algorithm. Also, BU and BU′ run much faster than MLE as MLE solves a convex optimization problem for each target in every round.

7 Conclusion and RelatedWork

So far, the field had been lacking an appropriate gametheoretic model for green security domains: this paper provides Green Security Games (GSG) to fill this gap. GSG's generalization of the Stackelberg assumption which is commonly used in previous work has led it provide two new planning algorithms as well as a new learning framework, providing a significant advance over previous work in green security domains [Yang et al., 2014; Haskell et al., 2014].

Additional related work includes criminological work on poaching and illegal fishing [Lemieux, 2014; Beirne and South, 2007], but a game-theoretic approach is completely missing in this line of research. Planning and learning in repeated games against opponents with bounded memory has been studied [Sabourian, 1998; Powers and Shoham, 2005; Chakraborty et al., 2013; de Cote and Jennings, 2010; Banerjee and Peng, 2005]. However, most of the work considers the case where each player chooses one action from his finite action set in each round of the game, while we focus on the problem motivated by real-world green security challenges where the players can choose a mixed strategy and implement it for multiple episodes in each round; thus previous approaches fail to apply in our domains. We further handle multiple boundedly rational attackers each with a different SUQR model, leading to a need to learn heterogeneous parameters in the SUQR model, which was not addressed in this prior work which assume a single fully rational attacker. Previous work on learning in repeated SSGs [Marecki et al., 2012; Letchford et al., 2009; Blum et al., 2014] has mainly focused on learning the payoffs of attackers assuming perfectly rational attackers. In contrast, we not only generalize the Stackelberg assumption to fit green security domains but also provide algorithms to learn the parameters in the attackers' bounded rationality model. By embedding models of bounded rationality in GSG, we complement previous work that focus on modeling human bounded rationality and bounded memory [Rubinstein, 1997; Cowan, 2005].

Acknowledgement

This research was supported by MURI Grant W911NF-11-10332 and by the United States Department of Homeland Security through the Center for Risk and Economic Analysis of Terrorism Events (CREATE) under grant number 2010-ST061-RE0001. A portion of this work has taken place in the Learning Agents Research Group (LARG) at the Artificial Intelligence Laboratory, The University of Texas at Austin. LARG research is supported in part by grants from the National Science Foundation (CNS-1330072, CNS-1305287), ONR (21C184-01), AFOSR (FA8750-14-1-0070, FA955014-1-0087), and Yujin Robot.