Integrating Demand Response and Renewable Energy In Wholesale Market

1 Introduction

To achieve the balance between environment and economics, RESs and active DR have been integrated into the modern power system which brings a series of uncertainties on the grid reliability and operational cost [Yu and Xue, 2016]. With the high penetration of RES and DR program, the uncertainty places a high risk on the power system reliability. Due to the intermittency of RES and the uncertainty of DR, market participants may not supply the expected energy quantity or demand reduction which would lead to a serious supply-demand imbalance. The power system operation and control could become more complicated to take a countermeasure to avoid the outage and cascade failure [Xie et al., 2011].

To address the uncertainty caused by the high penetration of intermittent renewable energy sources and power consumption, various stochastic optimization methods and robust optimization techniques are applied to energy management problems including unit commitment, economic dispatch, power system infrastructure planning, energy trading, DR, frequency control and plug-in electric vehicle (PEV) charging/discharging management. A multi-period framework based game theoretical model for characterizing behaviors among crowdfunders, solar energy supplier, and the power utility is studied by [Zheng et al., 2015]. A payment mechanism is designed to encourage RES for cooperatively participating the energy trading market through scoring rules in [Robu et al., 2012]. To reflect the real costs of power consumption, a prediction-of-use tariff is developed by a cooperative game in [Vinyals et al., 2014]. From a market perspective, [Kuate et al., 2013] consider a broker agent based energy trading where prices and demands in the day-ahead and the real-time markets are handled by a non-homogeneous hidden Markov model. By considering the uncertainty of demand response, a two-stage payment mechanism is designed to propose a risk-sharing strategy in the smart grid scenario by [Methenitis et al., 2016]. Moreover, demand side management provides a promising solution for improving energy efficiency. To incentivize demand response participation, a DR-VCG mechanism is introduced to provide flexible contract sets and prices in [Meir et al., 2017] while a rewardbidding mechanism is employed for a truthful and reliable demand response program by [Ma et al., 2017]. To manage PEV charging behavior, a data-driven charging strategy is proposed to capture the risk from the charging demand uncertainty in [Li et al., 2017].

However, to the best of our knowledge, most of the existing literature considers the integration problem of RES and DR either from the demand side of the customer level or from the supply side of the power utility level which brings a difficulty to design a reasonable feed-in tariff for RES injection and develop an attractive contract for DR participation. In fact, the power generation is more powerful than RES based microgrid owners and DR program participants in terms of pricing the electricity which indicates a Cournot game model may not truly reflect the interaction behavior among the power utility, RES based microgrid owners and DR program participants in aforementioned results [Anderson and Engers, 1992]. A Stackelberg game model [An et al., 2011] is able to characterize the sequential interaction of energy trading. Furthermore, many existing literature only harness the simple statistical information of historical data to model patterns of variation and present an approximation solution with certain probability distribution assumptions, which cannot fully capture the inherent nature of uncertainty in power consumption and generation. Particularly, RES based microgrid owners and DR participants also cannot predict the precise quantity of energy output and load reduction [Garg et al., 2013]. Although two-stage stochastic programming models could produce a conservative solution of energy management, a risk measurement method is required to quantify this risk for a better risk controlled strategy.

Contributions. This paper aims at developing a new integration framework of RES and DR program to improve the energy efficiency and the system reliability. First, we propose a novel two-stage Stackelberg-Cournot game model to capture the energy trading behaviors among the power utility, RES based microgrids and DR participants. In the proposed model, power utility is the leader which can help power utility obtain a better benefit while RES based microgrids and DR participants organized by an aggregator are followers. Second, a conditional value-at-risk (CVaR) measurement is applied to quantify the intermittency of RES and the uncertainty of DR for a risk controlled game model which could result in a better reliable energy trading strategy for both the forward and the spot markets. Third, we design an efficient computational algorithm to speed up the best response dynamics at the follower side.

2 Restructuring ElectricityMarket

We consider a single leader multi-follower Stackelberg game model to restructure the electricity market in which a power utility, and a set of N RES based microgrids and DR participants are involved to achieve a higher energy efficiency. In the model, the power utility, as the leader, will provide the majority of the electricity to clients through purchasing energy from power generators, whereas RES based microgrids and DR participants are followers to supply the rest of energy demand. To mitigate the supply uncertainty of RES and DR, a two-stage stochastic game model can be introduced where the forward market and the spot market are considered for the energy trading among the power utility, microgrids and DR participants.

2.1 Two-stage Stackelberg-Cournot Game

In the forward market, the aggregator can decide the forward market clearing price pFM ($/MWh) which relies on the total expected quantity of energy bids through the market clearing mechanism. The power utility will bid xFM (MWh) first. In the spot market, assume that in scenario ω, the power utility could adjust the energy output xω (MWh) according to the market requirement. Given a strategy of power utility, a microgrid or a DR participant n ∈ N can bid the expected quantity of energy contribution qFM n (MWh) in forward market while injecting energy or curtail energy consumption qω n (MWh) according to the practical capacity in the spot market.

They also can be organized by aggregators. By considering price dynamics and market competitive scenarios, the linear supply function is harnessed to characterize the relationship between energy supply and load demand [Xu et al., 2016]. Thus the forward market is defined by

where pFM,0 and DFM,0 are the forward market price and the total demand parameters for the forward market pricing scheme, respectively. The spot prices regarding a scenario ω are defined by the random vector pω : Ω → R on a probability space (Ω,F,P) as following

where pω,0 and Dω,0 are the spot market price and the total demand parameters for the spot market pricing. All these parameters can be estimated by deep learning techniques to improve the accuracy. If the market participants significantly improve the bidding quantity of energy resources, the market clearing price will be dramatically decreased in general.

The power utility as the leader aims to maximize the profit which is the revenue minus the cost. The payoff function of the power utility can be obtained as

where Cx(xω ) = axω +b(xω )2 is the cost mapping of power generation for xω , ∆xω = xω - xFM . The constraint set X is defined by

where X and Xω are the maximal power dispatch capacities in the forward and the spot markets, respectively.

Microgrids and DR participants also try to obtain the maximal benefit for participating the energy market once the leader takes an action. Meanwhile, they are playing a Cournot game with each other to determine the price in two markets. The payoff function of microgrids and DR participants can be defined as

where Cq (qω n ) is the cost mapping of microgrids or DR participants for contributing energy quantity qω n , ∆qω n = qω n -qFM n . Remark that the value of the cost mapping for RES based microgrids is constant which is contributed by the degradation of the system, i.e., Cq(qω n ) = c. The value of the cost mapping for DR participants is proportional to the energy curtailment which mainly leads to the discomfort and inconvenience, i.e, Cq(qω ) = dn(qω n -Qωn )2 . c and dn are parameters for mapping cost to two types of market participants. The constraint set Qn is defined by

where Qn and Qωn are the maximal energy supply capacities in the forward and the spot markets.

The Stochastic Stackelberg-Nash-Cournot Equilibrium problem can be formulated as

where x = (xFM , xω ) and qn = (qFM n , qω n ). fx(x) and g(qn; q-n) are defined in (3) and (5), respectively. q-n is the set of market participants except n, i.e., ∑m∈{N/n} qm.

3 Stackelberg-Cournot Game with Risk Control

In general, the more quantities of electricity or load reduction bid, the more profit received for market participants. If the energy quantity bidden by these unreliable suppliers in the forward market is less than in the spot market, a serious power shortfall may happen which would pose a heavy threat on the reliability of power system operation. In fact, different scenarios ω would have a different power supply capacity which should be taken into account. It is essential to develop a risk control strategy for safely integrating RES and DR into the power system. The energy supply by the power utility is more reliable than that by RES based microgrids and DR participants. We mainly focus on modelling the risk from RES based microgrids and DR participants. To measure the overbidding risk, the conditional value-at-risk (CVaR) measurement [Rockafellar and Uryasev, 2002] is applied to capture the degree of overbidding in the forward market according to the historical performance as following:

where R(Qωn , qFM n ) = max{0, qFM n -Qωn } is the loss function for measuring whether there is an overbid in the forward market. Given an energy quantity bid qFM n , if it is greater than the output capacity Qωn of the spot market, this will be identified as the overbidding risk. (·)+ is the projection operator on [0,+∞). θn is the confidence level which is used to characterize the uncertainty of the market participant n. The constraint set of the CVaR measurement is defined by Rn = {rn ≥ 0}. According to [Rockafellar and Uryasev, 2002], CVaR is convex with respect to rn which links the forward market strategy qFM n and the spot market capacity Qωn .

By substituting (9) into the payoff function (5) of followers in the two-stage game model, it gives

where yn = (qFM n , qω n , rn)⊤, Yn = Qn ∪Rn and γn is the risk control parameter. Thus, a Stackelberg-Cournot game with risk control for wholesale electricity market can be formulated by

4 SSCN Equilibrium Seeking Algorithm

The definition of Nash equilibrium implies that no player can further gain more benefits through a new strategy. A stochastic Stackelberg-Nash-Cournot Equilibrium of the game model (11) is defined as an N + 1 dimensional vector (x∗ , y∗ 1 (x∗ ), ..., y∗N (x∗ ); ω)⊤ if

• x∗ solves the Stackelberg problem:

• y∗n (x∗ ) = (y∗ 1 (x∗ ), ..., y∗N (x∗ ); ω)⊤ solve the Cournot problem:

4.1 SAA of SSCN Equilibrium

Due to the expectation evaluation of the payoff function with a random variable ω, it is difficult to obtain the analytic solution of the game model. To make the game model computationally tractable, sample average approximation technique is applied to approximately compute an SSCN equilibrium instead of the computation of the expectation for all market participants' stochastic payoff functions. We use the Monte Carlo approach to discretize stochastic payoff functions where an independent and identically distributed sample {ωi, i = 1, 2, ..., I} can be generated to approximate the expectation of payoff functions in the sense of sampling average approximation [Lin and Fukushima, 2010]. Hence, SAA based payoff functions can be presented by

Then, an SAA SSCN equilibrium is defined as an (N + 1)I dimensional vector (x∗,I , y∗,I 1 (x∗,I ), ..., y∗,I N (x∗,I ))⊤. x∗,I = (x∗,1 , · · ·, x∗,I )⊤ and y∗,I n (x∗,I ) = (y∗,1 n (x∗,I ), · · ·, y∗,1 N (x∗,I ))⊤. If the number of samples I go to infinity, the SAA SSCN equilibrium would be infinitesimally close to the true SSCN equilibrium.

Before we propose the SAA SSCN equilibrium seeking algorithm, the property of the SAA payoff functions should be investigated. First, the existence of SAA SSCN equilibrium has to be guaranteed in advance.

Theorem 1. The two-stage stochastic game model (11) admits an SAA SSCN equilibrium if 4pFM,0 DFM,0 ≥ ∑Ni=1 pωi ,0 IDωi,0 .

Proof. Note that all constraint sets X and Yn(n ∈ N) are convex with respect to the decision variables. The risk measurement CVaR(·) and the cost functions C(·) of each market participants are convex with respect to the decision variables. Therefore, we only need to concentrate on the original payoff functions (3) and (5) which are approximated by SAA technique. To verify the convexity of each payoff function, the objective is set to minimize the negative revenue function. The revenue function of the power utility is given by

The corresponding Hessian with respect to x is computed by

If 4pFM,0 DFM,0 ≥ ∑Ni=1 pωi ,0 IDωi,0 holds, the Hessian (18) is positive definite. Thus the revenue function (17) is convex which implies the SAA payoff function (15) of power utility is concave on X . Similarly, we can prove that if 4 pω,0 Dω,0 ≥ 1 I ∑Ni=1 pωi ,0 Dωi ,0 holds, the SAA payoff function (16) of the follower n is concave on Yn. Therefore, the two-stage stochastic game model (11) admits an SAA SSCN equilibrium.

4.2 Seeking Algorithm of SAA SSCN Equilibrium

Due to the fact that the leader and the followers of the Stackelberg-Cournot game model are coupled by the pricing scheme, the solution generated by the seeking algorithm has to satisfy both sides of the leader and the followers' problem.

Follower side. To obtain an SAA SSCN equilibrium, we first study the Cournot game problem in the follower side. In view of (16), the optimization problem of each follower n can be reformulated with a fixed x as

where si(i ∈ I) is the convex relaxation for computing CVaR. To calculate the Cournot Nash equilibrium of the followers' problem, a best response dynamics based algorithm can be proposed by a Nikaido-Isoda function. The ρ-regularized Nikaido-Isoda function Ψ : (Y × Y) → R of the maximization problem (19) for all players n in the

Cournot game is defined as Ψ(y, z) := ∑Nn=1

{g(zn, y-n)-

g(yn, y-n) + η2 ‖zn - yn‖2}, where Y = ∏Nn=1 Yn. A point y∗ ∈ Y is defined as a normalized Nash equilibrium point for the Cournot game in (19) if minz∈Y Ψ(y∗ , z) = 0 holds. As known, this classic best response results in a linear convergence rate of Nash equilibrium seeking. If there are a large numbers of followers, there is a computational issue in classic best response algorithm that requires a speed-up strategy at algorithm level [Mohsenian-Rad et al., 2010]. Assume that all followers can communicate with each other in a networked environment. To develop a fast distributed optimization scheme, we define a convex combination of the best response strategy z and the current strategy y(k) as follow

where τ is a parameter to reconstruct a new best response strategy y at iteration k + 1 which enables a better convergence rate. The convex combination still matches the fixed point mapping

whose the subgradient can be determined by ∂F(y∗ ) = ∇z⊤(y∗ ) - E at a Nash equilibrium y∗ . E is the identical matrix with a proper dimension. The accelerated best response dynamics for seeking Cournot Nash equilibrium can be proposed as Algorithm 1.

Theorem 2. If parameters in Algorithm 1 satisfy

where λM(·) and λm(·) are the maximal and the minimal eigenvalues of the matrix regarding the subgradient ∂F(y) at y∗ , then Algorithm 1 superlinearly converges to a Cournot Nash equilibrium y∗ .

Proof. In light of the best response algorithm, the update rule in (22) shows a linear convergence rate. In fact, each update Algorithm 1 Seeking Cournot NE By Accelerated Best Response Strategy

• 12: end procedure

Algorithm 2 Seeking SAA SSCN Equilibrium

1: procedure SSCNE-SEEKING 2: Leader's strategy x ∈ X 3: j = 1; 4: repeat 5: Given a fixed x, yn is obtained by Algorithm 1. 6: Update leader's strategy x(j) by solving (15). 7: j = j + 1; 8: until ‖ x(j) -x(j -1) ‖≤ ϵ 9: end procedure

of best response strategy coincides with the nonlinear mapping (23). To analysis the convergence rate, the subgradient of the nonlinear mapping is given by

w(z(y)) ≤ 0 is the constraint set in accordance with (19) and (20). ⊗ is Kronecker product. By applying a multi-step update to obtain the new best response strategy, the condition numbers of the gradient update are given by

As known, the Heavy-ball technique [Ghadimi et al., 2015] presents a superlinear convergence rate which implies Algorithm 1 also has a superlinear convergence rate.

Leader side. To solve leader's optimization problem, we concentrate on (15) which is a convex optimization problem. Leader's strategy depends on followers's optimal decision which is given by Algorithm 1. Once followers do not change their strategy, the solution of leader's optimization problem is an SAA SSCN equilibrium. Therefore, the seeking algorithm can be proposed as an alternating direction framework of Algorithm 2 which converges to an SAA SSCN equilibrium according to the definition.

5 Simulation

In this section, the proposed model is evaluated by the effectiveness whereas the algorithms are tested by the computational efficiency. Australia has a long history of using RES in the past decades. To initialize the experiment, the data set regarding RES is collected from Australia Energy Market Operation (AEMO) 1 in the summer from Jan. 1, 2017 to April. 30, 2017, where the historical electricity prices and local demands in West Victoria can be obtained. The data is processed as 1-hour timeframe. Assume that there is one power utility (PU) with conventional power generations in this area with the maximal energy supply capacity 500 MWh. Regarding RES, it is assumed that they are divided into two categories and managed by one solar energy aggregator (SEA) and one wind energy aggregator (WEA). For DR program, we assume that the load consumed by HVAC system in the building can be curtailed to response to DR events due to the fact that there are a large number of energy for thermal requirements [Li et al., 2016]. We use the 100 commercial buildings' thermal demand and assume they are organized by one DR aggregator (DRA) in this area. The forward market parameters pFM,0 , DFM,0 can be predicted according to the historical price and demand by deep neural networks proposed in [Ghaderi et al., 2017]. The spot market parameters pω,0 and Dω,0 are sampled randomly. All parameters are easily chosen to satisfy the condition in Theorem 1 for guaranteeing the convexity of the objective function. Take θn = 0.95, η = 0.1 and ϵ = 0.01.

Two-stage gamemodel with risk control. Taking the sampling number I = 30 and varying the risk weight parameter γ from 0 to 0.5, we can obtain the price and the energy bids for both markets. As shown in Figures 1(a) and 1(c), the expected profit of PU is increased with the expected energy bid from $24, 933.65 to $34, 757.54 while the expected profits of other participants are decreased. In Figure 1(b) and 1(d), own to the decrease of bids by REA and DR, the forward market clearing price rises up from 61.3 $/MWh to 71.4 $/MWh. Due to the larger variability of wind energy in this area which is illustrated by its CVaR, WEA bids much less energy than other followers in the forward market, amounting at 67.32 MWh with γ = 0.5. Followers' CVaR also shows a decline while Leader's CVaR remains at zero. This is because the conventional power generations is reliable. Overall, with the increase of the risk weight, RES and DR based followers tend to submit less energy bids due to the increasingly penalty which implies the risk controlled game model has a potential to avoid an aggressive bidding behavior of RES owners and DR participants in the energy market.

1 http://www.aemo.com.au/

By keeping the risk weight γ = 0.1, the game model can produce a 24-hour bidding strategy as shown in Figure 2 where all market participants bid at their maximal capacity due to the large demand in this case.

Stackelberg model vs. Cournot game. To show the advantage of proposed model, we compare with Cournot Nash game by which power utility is not a leader and cannot take an action in advance. By increasing demand parameters of the forward and the spot markets from 400 to 2000 and keeping other parameters unchanged, the comparative case study of the expected forward market clearing price and the expected profit can be obtained in Figures 3(a) and 3(b). As illustrated, the expected price and the expected profit grow up with the increase of demand parameters in general. Meanwhile, the expected price and the expected profit produced by the Stackelberg model are larger than those by Cournot game which actually reflects the real situation. The power utility can take more benefits as it is the leader.

Computational efficiency. In a networked scenario, computational efficiency of seeking a Nash equilibrium plays a significant role in implementation of the proposed model. The scalability of Algorithm 1 for solving followers' problem is investigated. By increasing the number of followers N from 3 to 24 and keeping the sampling number I = 30 for all experiments, the decision variables are 62N which means the scale of the optimization problem ranges from medium to large. We make a comparison in terms of iteration number and computational time between two algorithms, i.e. Algorithm 1 and best response (BR) dynamics in (22).

For two accuracy requirements ϵ = 0.01/0.001, the comparative study of computational efficiency is reported in Table 1. When ϵ = 0.01, the number of iterations goes up from 5 and 8 as the number of followers increases. Note that if there is no strict accuracy requirement, both the number of iterations and computational time can be decreased considerably. For example, given N = 24, the computational time of the accuracy requirement ϵ = 0.001 is more than doubled compared with accuracy requirement ϵ = 0.01. This numerical example demonstrates that our proposed algorithm yields a good performance in terms of scalability.

6 Conclusion

In this paper, a two-stage Stackelberg-Cournot game model has been proposed to integrate RES and DR into wholesale electricity market. The interaction among the power utility, RES based microgrids, and DR participants are characterized by an SSCN equilibrium for maximizing their expected profits in the forward market which is solved by SAA based algorithm. By considering of overbidding in the forward market, this aggressive bidding behavior is controlled by CVaR based risk measurement. Moreover, we develop an accelerated best response dynamics based algorithm for solving follower's optimization problem where the optimal parameters of acceleration and convergence analysis are presented by a heavy ball method. The simulations show that our proposed model can effectively avoid the power supply shortfall risk by overcommitting energy supply in forward market.

Acknowledgement

This research is supported in part by the Australian Research Council (ARC) under the Discovery Program grant DP170102303, DP160102114, and in part by the National Priorities Research Program from the Qatar National Research Fund (a member of Qatar Foundation) under Grant 9-166-1-031.