Hierarchical Electricity Time Series Forecasting for Integrating Consumption Patterns Analysis and Aggregation Consistency

1 Introduction

Electricity demand forecasting is a crucial problem for power grid [Li et al., 2012; Taylor, 2012]. Precise prediction can give effective guidance to enable reasonable decision making and management on power generation and supply, and benefits the economy and environmental protection issues. As the smart grid develops faster, massive number of individual electricity consumption data (such as smart electricity meter data) can be obtained conveniently [Zheng et al., 2013; Siano, 2014]. The available electricity consumption data with large scale and wide variety leads to new challenges to the communities of research and industry [Ramchurn et al., 2012].

In electricity demand forecasting, the system structure is usually built on the geographical hierarchy [Yang et al., 2015]. For instance, electricity demand in a city can be disaggregated into ones of districts through the administrative divisions, which are further disaggregated into ones of streets and blocks, etc.. The "aggregation consistency" is a critical point in hierarchical forecasting, that is the disaggregated time series should add up equally to the high level aggregated ones. Since the aggregation consistency is very unlikely ensured in the forecasting of the entire time series independently, the "bottom-up" approach is adopted. It forecasts all of the bottom level disaggregated series and then adds the forecasts results to form various aggregated series forecasts to achieve the aggregation consistency. However, when the disaggregated data tends to have low signal-to-noise ratio, the overall prediction accuracy especially for the disaggregated series forecasting is poor [Taieb et al., 2017]. In recent work, various optimal reconciliation approaches become the mainstream. Hyndman et al. (2011) propose a solution of using least squares reconciliation to deal with the hierarchical forecasting problem. They apply a regression model to optimally reconcile the forecasts of the disaggregated and aggregated series correspondingly. However, the aggregation consistency is often achieved at the expense of the forecasting accuracy, since the side effects of the reconciliation and adjustments may enlarge the prediction errors.

In this paper, we propose a novel approach for electricity demand hierarchical forecasting. Instead of dealing with the geographical hierarchy directly, we make time series clustering analysis for electricity consumption pattern detection. Individual household electricity consumption (power usage) usually follows a steady pattern of growth over time in some extent. In this work we build a new consumption pattern based time series hierarchy based on the clustering results. When the time series with similar patterns are grouped together, the signal-to-noise ratio of the aggregated time series will be risen significantly, and the prediction accuracy of the aggregated ones will be improved. Figure 1 gives more illustration on this phenomenon. Note that the forecast of the electricity usage is composed of two parts: the ground truth of the electricity usage (denoted by TA and TD for aggregated and disaggregated forecasts respectively) and the prediction error (denoted by EA and ED for aggregated and disaggregated forecasts respectively). With aggregation constraints, we have the equation: TA +EA = Σ(TD +ED), and as the ground truth of the electricity consumptions at different levels are always equal, it can be deduced that EA = ΣED. Therefore, if the EA is decreased then in overall the prediction error of the disaggregated forecasts ED will be decreased either with proper reconciliation methods. Based on the above motivation, we propose a new optimal reconciliation approach to improve the forecasting ability of the disaggregated time series. Finally the "bottom-up" approach is applied for adding up the disaggregated forecasts to form various high level aggregated forecasts, hence the aggregation consistency on the geographical hierarchy is realized.

In this paper, our main contributions are threefold:

1. We present a novel hierarchical electricity demand forecasting approach. Unlike the traditional "single path" solution, we exploit the idea of electricity consumption pattern analysis, the optimal reconciliation regression and the "bottom-up" approach altogether to deal with the hierarchical electricity time series forecasting problem.
2. In our approach we present a novel electricity consumption pattern based hierarchical forecasting method. It is based on the observation that the reconciliation error of the disaggregated time series is "correlated" to its membership degree of the corresponding cluster (the consumption pattern). Hence we propose a new optimization object function with a regularization term of penalizing the correlation of the two distributions.
3. We conduct experiments on two real-life datasets for performance evaluations. In our experiments, the state-ofthe-art methods and the strong baselines are compared with our method on hierarchical electricity demand forecasting extensively. The experimental results show that our method achieves the best forecasting accuracy while keeping geographic aggregation consistency. Specially, Mean Absolute Percentage Error (MAPE) of our approach is 2.50% lower than that of the state-of-the-art methods.

2 RelatedWork

Base forecasting (BASE) [Hyndman et al., 2011] estimates time series at all levels in a hierarchical structure independently. Some common models, such as ETS (Smoothing State Space model), are applied in the classical forecasting [Box and Jenkins, 1976; Hyndman et al., 2008; De Livera et al., 2011]. Due to the characteristic of independent forecasting, the predictions at different levels do not adversely affect each other. But the aggregation consistency is not taken into account in BASE. To satisfy the aggregation consistency, Bottom-up [Athanasopoulos et al., 2009] adds up the bottom-level forecasts to obtain the upper ones.

Recently, the optimal combined forecasting becomes the mainstream in hierarchical time series forecasting with aggregation consistency. It estimates initial forecasts at bottom level by using BASE, then these forecasts are reconciled according to the geographic aggregation constraints. The method of Ordinary Least Square (OLS) [Hyndman et al., 2011] regards the covariance matrix as an identity matrix in the adjustment model learning. Although OLS ensures aggregation consistency, the forecasting accuracy may be decreased due to the simple assumption of covariance matrix. HLS method [Athanasopoulos et al., 2015] obtains weights from the hierarchical structure. It assumes that forecast errors of time series at the same level are equivalent while that of series at different levels are not. Different fromHLS, other optimal combined forecasting methods adopt weight least square method [Wickramasuriya et al., 2015]. WLS [Hyndman et al., 2016] estimates the diagonal part of the covariance matrix by using the sample variance of the pre-forecasts. Because WLS estimates the covariance matrix based on the historical observations (pre-forecasts), the adjustments at the bottom level series may be affected negatively when the time series is not steady. MinT-Reg [Taieb et al., 2017] obtains the sparse adjustments by solving a regression problem with the elastic net penalty.

For the general hierarchical forecasting problem, the Bayesian framework is used to obtain the proportion changes in hierarchical domestic tourism forecasting [Park and Nassar, 2014]; An adaptable regression method combined with multiple scalar forecasts to obtain accurate weather predictions is proposed [Williams et al., 2016]. A data-driven decision support system [Huber et al., 2017] is presented to predict the hierarchical demand of fast moving consumer goods.

3 Our ProposedMethod

3.1 Consumption Pattern Analysis and Hierarchy Construction

We explore consumption patterns of electricity smart meter data. Consumption patterns are extracted from amounts of individual household time series by using the X-means [Pelleg et al., 2000] clustering algorithm. As a toy example, two consumption patterns are extracted from clustering eight electricity time series, which is shown in Figure 2.

The ground truth of the electricity consumptions at different levels are always equal, as illustrated in Section 1. If the forecast of aggregated time series is accurate, the average prediction accuracy of individual time series will be likely improved under the aggregation constraints and reconciliation. We observe that the pattern of electricity time series is more regular than the aggregated time series based on geographical hierarchy (Figure 1). The prediction error of pattern is hence less than that of the regional aggregated time series. Motivated by the idea, we therefore construct a consumption pattern based hierarchy (Figure 3), consisting of patterns (the high level series) and individual time series (the bottom level series).

3.2 Hierarchical Forecasting Based on Aggregation Consistency

In the consumption pattern based hierarchy, the bottom level series adds up consistently over levels. We reconcile the bottom level forecasts under the constraints of ¯ap,T+h = Sc,a¯bT+h, where ¯aT+h and ¯bT+h denote the mean aggregated h-period forecasts and disaggregated ones with T given historical observations respectively. Sc denotes an N × Nb summing matrix derived from the consumption pattern based hierarchical structure. It consists of an Na × Nb submatrix Sc,a and an Nb × Nb identity matrix. N = Na + Nb.

Our goal is to obtain the reconciled bottom level forecasts by minimizing the expectation of square forecast errors.

where

Yp,T+h denotes an N-length vector of real forecasts at time T+h, ˜bT+h denotes an Nb-length vector of reconciled bottom level forecasts, Q denotes an Nb × N transformation matrix, ˆbT+h and ˆ Yp,T+h denote initial forecasts at bottom level and all levels based on ETS model, respectively.

The optimal solution in Equation 2 can be obtained by the best linear unbiased reconciled forecasts [Wickramasuriya et al., 2015].

where Wh denotes the covariance matrix of the h-period ahead forecast errors and its estimator of one-ahead base forecast errors is written as:

where kh denotes a positive constant. To simple the computation, Wh is assumed to be an identity matrix.

According to generalized least squares (GLS) [Kariya and Kurata, 2004], the bottom level forecasts given by Equation 5 can also be estimated from the regression model in Equation 7.

where βT+h is the conditional mean of the reconciled bottom level forecasts ˜bT+h under the observations at all levels. ξh is the estimated error of initial forecasts, whose mean and variance are zero and covariance matrix ˆWh, respectively. The GLS estimation of the bottom level forecasts then are obtained by minimizing the squared Mahalanobis length of the residual vector ˆ Yp,T+h - Sc˜bT+h,

In this way, our problem is formulated as an optimal GLS regression problem of forecast error, which can be effectively solved by using gradient descent method [Snyman, 2005]. This method is denoted by CHF in this work.

Let ΘT+h be an Nb-length vector of adjustments of the bottom level time series, we have ˜bT+h = ˆbT+h + ΘT+h.

Then our new optimization object function is as follows:

where Θnorm,T+h = ( |θ1| √ 1 Nb ∑Nb i=1 (θi)2 , ..., |θNb | √ 1 Nb ∑Nb i=1 (θi)2 ) , θi denotes the i-th unknown value of adjustment of the bottom level time series, 1 ≤ i ≤ Nb, Vnorm = ( d1 √ 1 Nb ∑Nb i=1 (di)2 , ..., dNb √ 1 Nb ∑Nb i=1 (di)2 ), and di denotes the i-th distance between the bottom level time series and its corresponding clustering center (the consumption pattern), which

can be estimated by using clustering analysis.

3.3 The Regularization

The regularization term in Equation 9 is defined by L2 norm [Hastie et al., 2015] of the distance between Θnorm,T+h and Vnorm. λ > 0 is a penalty parameter. The constraint of ˆbT+h + ΘT+h ≥ 0 ensures the reconciled forecasts are greater than or equal to zero. This method is referred to as CHF-Reg in the rest of the paper. The regularization term is based on exploring the correlation between consumption pattern and individual electricity time series. Figure 4 shows that the distribution of the absolute value of ideal adjustment of real electricity individual time series is similar to that of the distances from the time series to the clustering center (the corresponding pattern).

A time series is more similar to the pattern means it is more stable and regular. That is, when the time series is closer to the clustering center, its forecast is more accurate and therefore requires smaller adjustment. In order to find the optimal adjustment, we hence introduce the regularization term to penalize the correlation between the distribution of the adjustment of series and that of its membership degree of the corresponding pattern. When the time series is closer to clustering center, the corresponding adjustment obtained by solving Equation 9 will be smaller.

3.4 High Level Aggregated Forecasting

In order to keep geographical aggregation consistency, the "bottom-up" method is applied as shown in Figure 3. Based on the geographical hierarchy, we obtain the regional forecasts at all levels by summing the optimally reconciled bottom level forecasts. Yregion,T+h = Sregion(ˆbT+h +Θ∗T+h ). Where Sregion denotes a summing matrix derived from the geographical hierarchical structure.

4 Experiments

4.1 Experimental Datasets

We use two real-life datasets on individual household electric power consumption. The first one is power grid data from State Grid Shanghai Municipal Electric Power Company. It contains 3078 time series, taken from January 1th, 2014 to February 19th, 2015, at a sampling rate of 24 hours. The second one is public electricity data from Energy Demand Research Project: Early Smart Meter Trials (EDRP) [Raw and Ross, 2011]. It contains 2501 time series at a sampling rate of 30 minutes.

4.2 Experimental Setup

In the experiments, we compute one-step-ahead rolling forecasts [Hyndman et al., 2016] with historical consumption observations for the two datasets. Specifically, we repeat 15 forecasting tasks on the smart grid data from January 1th, 2014 to February 4th, 2015, so as to generate electricity demand in half a month. While for EDRP data, we repeat 48 forecasting tasks, from May 9th 00:00, 2009 to May 20th 09:30, 2009, in order to predict electricity demand in the whole day.

For each forecasting task, we use Mean Absolute Percentage Error (MAPE) [Wijaya et al., 2015] and Root Mean Square Error (RMSE) [Yang et al., 2017] to measure forecasting errors. They indicate the relative forecast errors and the average forecasting errors of household electricity consumption at one time, respectively.

In the experiments, we compare our method with the strong baselines and the state-of-the-art methods, including BASE, Bottom-up [Athanasopoulos et al., 2009], HLS [Athanasopoulos et al., 2015], WLS [Hyndman et al., 2016] and MinT-Reg [Taieb et al., 2017]. In our CHF and CHF-Reg, 32 types of patterns are extracted by clustering amounts of time series whose dimension is reduced to 50 by using PCA [Hotelling, 1933]. The value of λ are 3×105 and 104 for smart grid and EDRP dataset respectively.

4.3 Experimental Results and Analysis

The prediction accuracy at each level and the average statistical results for the two real-life datasets are shown in Table 1 - 4, respectively. The last columns in these tables indicate whether the forecasting methods satisfy the geographical aggregation consistency.

From the tables, we can see in general that the upper level predictions have less errors than the bottom level forecasts. Because the upper level forecasts have higher signal-to-noise ratios due to the aggregation. We can also observe that BASE has the least forecast errors in some upper levels, but it cannot preserve the constraints of geographical aggregation. In contrast, CHF-Reg not only satisfies the constraints, but also achieves a higher forecasting accuracy than BASE in most cases.

In Table 1 and Table 2, the MAPE of BASE is 0.14% lower than that of Bottom-up, and is 1.17% lower than that of HLS. It suggests that Bottom-up and HLS achieve the aggregate consistency at the expense of overall prediction accuracy. In contrast, CHF-Reg forecasts achieve lower MAPE and RMSE of 6.19% and 4.714 respectively than HLS. This is because that CHF-Reg optimally reconciles forecasts of the time series at bottom level according to the distances between these series and their corresponding clustering centers, while HLS uniformly reconciles the forecasts at the same level. The RMSE of CHF-Reg forecasts are 27.215 lower than that of WLS. The reason is that CHF-Reg estimates the adjustments based on patterns extracted from historical observations. Although both MinT-Reg and CHF-Reg have lower MAPE values than other methods, CHF-Reg still shows the best prediction performance. Specially, one-day electricity consumption forecasts of one household based on CHF-Reg is averagely 0.502 kilowatt hour more precise than MinT-Reg. This is because CHF-Reg can estimate the appropriate adjustment values applied to the forecasts at all levels, which effectively mitigates the negative effect of estimation errors in the adjustments. In addition, CHF-Reg has better forecasting performance than CHF. It demonstrates the effectiveness of the introduced regularization in the object function.

Figure 5 and Figure 6 show the comparison of prediction accuracy for the six methods averaged over all time series at the bottom and upper levels, respectively. We can see

that CHF-Reg (solid line with circle markers) always has less forecasting errors at different horizontal coordinates than other methods.

In the forecasting results on the EDRP dataset in Table 3 and Table 4, it can also be found that CHF-Reg obtains the most precise forecasts than other methods that satisfy aggregation consistency. We can see that CHF-Reg forecasts show more obvious advantages on the first dataset than the second one. This is because the electricity time series data in the second dataset are more stable and hence their adjustments will be smaller. Overall, in term of both MAPE and RMSE, CHF-Reg has better predictive power and more robustness on two real datasets than strong baselines and the state-of-the-art methods.

ture work, we will explore new hierarchy building approaches and the optimization prediction with aggregation consistency in the electricity demand forecasting problem and other application fields.

Acknowledgements

This work was supported by the National High Technology Research and Development Program (863 Program) of China (2015AA050203), NSFC grant No. 61370157 and No. 61572135, and the Shanghai Innovation Action Project ( No.16DZ1100200, No.17DZ1203600).

5 Conclusion

Aggregation consistency is one of the critical points in hierarchical time series forecasting. In most of the previous work, it is considered that the prediction accuracy is usually affected by the geographical aggregation constraints. In this work we demonstrate that a flexible usage of the aggregation constraints could bring the improvement of the electricity demand forecasting. To deal with the problem, we build a new data hierarchy by electricity consumption patterns analysis. Then based on the new hierarchy and our proposed optimization regression method with regularization, the prediction accuracy of the aggregated and disaggregated times series are significantly improved on the real-life datasets. In fu-