Deep Sparse Coding based Recursive Disaggregation Model for Water Conservation

1 Introduction

According to the 2012 statistics reported by [WHO, 2012] and [UNICER, 2012], more than 780 million people live in a shortage of portable water and about 2.5 billion people are living in areas where it is difficult or even impossible to access safe sanitation facilities. It's estimated that 2.3 billion people will be living without access to basic water needs for drinking, cooking and sanitation in 2025 [Concern-Worldwide, 2008]. To ensure that the world's water resources can be sustained for future generations, water conservation becomes an increasingly important research topic.

Generally, there are two main approaches to achieve the goal of water conservation: deploying infrastructures for efficient use of water, or reducing water demand by changing consumption habits. The first strategy will inevitably incur a large investment to develop new techniques for intelligent use of water, while the second strategy is more economical and mainly depends on customers' consumption behaviours. Recent studies have shown that detailed feedbacks on consumption patterns would largely affect customers' behaviours and ultimately save consumption up to 20 percent [Fischer, 2008; Froehlich et al., 2010; Houde et al., 2013]. We intend to employ the advances of artificial intelligence to separate aggregated water consumption into component devices, and provide device level usage to users for water conservation.

Water disaggregation is an emerging research topic, which involves taking an aggregated water consumption, for example, the total smart meter readings of a house, and decomposing it into the usages of different water fixtures. Recently, significant research efforts have been attracted to detect the "open" and "close" operations of appliances based on high sample rate sensing data. Froehlich et al. proposed HydroSense, a pressure based sensor, to identify activities at individual water fixtures within a home and estimate the amount of fixture level water usage [Froehlich et al., 2009]. Furthermore, an extensive study of the HydroSense technology was conducted by Larson et al. through a comprehensive analysis of valve- and fixture-level events identification [Larson et al., 2012]. In 2011, Froehlich et al. performed a longitudinal study of pressure sensing and concentrated on inferring real-world water usage events in the home [Froehlich et al., 2011]. Srinivasan et al. designed flow and motion sensor signatures based clustering techniques to identify unique water fixtures [Srinivasan et al., 2011]. Parson et al. used Hidden Markov Model for energy disaggregation in an iterative manner [Parson et al., 2011]. However, all of these studies depend on high sample rate (1Hz∼1Khz) sensing data. It is not practical to widely deploy high sample rate smart meters in real world due to privacy and reliable data transmission concerns.

This motivates another major stream for water disaggregation research with the usage of low sample rate (about 1/900 Hz) sensing data. Kolter et al. proposed discriminative sparse coding for low sample rate energy signal disaggregation, and applied structured prediction to refine the basis functions to minimize the regularized disaggregation error [Kolter et al., 2010]. However, the accuracy of this model will decease when common basis functions are shared by many devices.

A FHMM-based unsupervised disaggregation framework is presented in [Kim et al., 2011] to incorporate additional features, such as time of day and dependency between appliances. However, it's impractical to collect such prior knowledge especially when handling data collected from a large scale area. Wang et al. incorporated the shape and activation characteristics into sparse coding based dictionary learning, and achieved acceptable performance [Wang et al., 2012]. But only three devices were analysed since it's impractical to manually examine the spans and shapes for all devices, especially when the number of devices is large.

In this paper, we look specifically at the task of low sample rate water consumption disaggregation. Since existing approaches are all lack of the mechanisms to learn the decomposition structure, we conduct the first study on the representation learning of the disaggregation process. Through designing a hierarchical recursive structure, we can control the disaggregation sequence. Considering the problems that the learnable parameters of discriminative sparse coding model will significantly increase with the number of devices, several critical issues will emerge, such as over-fitting due to excessive complexity and slow convergence caused by vast local optima. It's of significant importance to perform separation in a structural manner targeting for reducing the size of dictionaries. We disaggregate consumption of each device from total consumption step by step, and learn a much smaller discriminative dictionary to distinguish the current device from all other residual devices in each step. Suppose there is a total consumption of k devices, and there are N bases for each device. If we learn the dictionary for separation using discriminative sparse coding, there will be a dictionary of k ×N for distinguishing the k devices simultaneously. Generally, there are more than 10 identical water appliances in each household, i.e., k > 10. On the other hand, using the recursive structure, we only need to learn a dictionary of 2 ×N for the separation between the current device and the remainders. An effective algorithm is designed for the discovery of the optimal separation sequence with the objective of minimizing the recursive disaggregation error respectively for each layer. In summary, the primary contributions of this paper are as follows:

• Construction of the hierarchical decomposition structure: We investigate the advantages of representation learning for disaggregation, and introduce sequential set to formalize the recursive structure.
• Design of a Deep Sparse Coding based Recursive Disaggregation Model (DSCRDM): DSCRDM is proposed to incorporate the separation architecture into the sparse coding model, and the recursive disaggregation error is presented to perform optimization.
• Efficient and effective learning algorithm for the discovery of separation sequence: We propose the algorithm deep sparse coding by relaxing the exact constraints to learn the layer level discriminative and reconstruction dictionaries sequentially.
• Extensible experiments to validate the effectiveness of DSCRDM: We demonstrated that DSCRDM outperformed other models in both whole home and device

level measures in a large scale real-word disaggregation task.

2 Preliminaries

In this section, we introduce the notations and discriminative disaggregation sparse coding.

Notations and Concepts Assume there are k water fixtures, such as faucet and irrigation, in a household. For ∀i = 1, 2, · · · , k, there exists a consumption matrix Ci ∈ RT×M , where T is the number of intervals in one day, and M indicates the number of days. The j th column of Ci, denoted as Cij, represents the consumption of device i in the j th day, where j = 1, 2, · · · ,M. The lth element of Cij, indicated by Cijl, denotes the water usage of device i at the lth interval in the j th day, where l = 1, · · · , T. The aggregated water consumption over all devices is ¯C = ∑ki=1 Ci. During the course of training, water consumption of individual fixture is supposed to be available, i.e., C1,C2, · · · ,Ck, while at the testing phase, only the aggregated consumption ¯C could be used for disaggregation. Under the context of sparse coding for source separation [Schmidt and Olsson, 2006; Schmidt et al., 2007], water consumption of the ith fixture is approximated by Ci ≈ BiAi, where Bi ∈ RT×N denotes the dictionary for fixture i, and Ai ∈ RN×M is the activations of dictionary Bi. N is the number of basis functions in dictionary Bi, which can be activated by Ai for the reconstruction of Ci [Olshausen and Field, 1996].

Discriminative Disaggregation Sparse Coding Discriminative Disaggregation Sparse Coding (DDSC) was proposed in [Kolter et al., 2010] and designed for energy disaggregation. With the assumption that the disaggregation dictionary is not necessary the same as that for reconstruction, they define the augmented regularized disaggregation error as the objective function

Where ‖X‖F ≡ (∑p,q (X)pq )1/2 denotes the Frobenius norm, λ ∈ R+ is a regularization parameter, X1:k is the shorthand for [X1,X2, · · · ,Xk]. In Equation (1), B1:k represents the reconstruction bases, which are learned based on sparse coding model. ˜B1:k denotes the discriminative bases which intend to reduce the difference between ˆA1:k and A⋆1:k , where A⋆1:k is the activations learned by sparse coding. A structured prediction based algorithm was applied for optimization by iteratively updating ˆA1:k and ˜B1:k.

However, DDSC might fail to identify disaggregation dictionary when training bases ˜B1:k with a large value of k and limited label data. Moreover, when the number of fixtures increases, it's of high probability that consumption patterns of devices are similar with each other. Consequently, the disaggregation performance of DDSC will decrease significantly.

Since the size of ˜B1:k increases with k, the vast amount of learnable parameters will lead to many local optima and high time consumption for iterative update. In addition, due to high cost and invasion of privacy concerns, only a small quantity of label data can be collected and available for training. This might result in poor efficacy of DDSC since ˜B1:k is large.

3 Deep Sparse Coding based Recursive Disaggregation Model

In this section, we design a decomposition structure to overcome the limitations of DDSC and propose the algorithm deep sparse coding to perform disaggregation task.

3.1 Design of the Recursive Decomposition Structure

Representation of the structure or data has been considered as a significant factor for the success of machine learning [Bengio et al., 2012], which also plays an important role in the disaggregation task. For example, suppose there are three parallel devices: two similar Showers (denoted as SH1 and SH2) along with one Clothes Washer (denoted as CW). If we disaggregate these three activities simultaneously, it's difficult to discriminate the two similar Showers. On the other hand, if we consider a different representation: SH1 along with the combination of SH2 and CW, then we can first derive SH1 from the total consumption, and then separate SH2 from the combination. Since the consumption patterns of the combination of one Shower and one Clothes Washer is usually quite different from those of one Shower, it's rather easier to achieve a high disaggregation performance based on such a disaggregation structure.

This motivates us to conduct an in-depth study on the representation of decomposition structure, and design a recursive architecture to conduct the disaggregation task. An illustration is shown in Figure 1: The disaggregation process is conducted in a recursive manner to separate one device from the remainders at one layer until all devices are examined. Considering there are five water fixtures, Irrigation is firstly extracted while the remainders are combined together and regarded as an artificial device. Then with the usage of the 1st residual consumption, Humidifier is separated from the other devices. Similarly, Dishwasher, Bath, and Hot Tub could be identified. Based on the designed structure, it's clear that within each step, only 2 ×N basis functions are required to decompose the aggregated consumption into one single device and the remainders. Compared with DDSC, it is capable of fully utilizing the limited label data, since k - 1 combinations of the data are inherently incorporated in the disaggregation process. To formalize the representation of the disaggregation architecture, we introduce the definition sequential set, Definition 1. Suppose set S contains k elements, i.e., S = {1, 2, · · · , k}. Then its sequential set Si, (1 ≤ i ≤ k) is defined as: Si = {S(1) i , S(2) i }, where

The reason we call Si for 1 ≤ i ≤ k as the sequential set is that it inherently sorts set S as S(1) 1 , S(1) 2 , · · · , S(1) k . For example, Figure 1 shows one sequential set for the devices: S(1) 1 = sIrrigation, S(2) 1 = {sHumidifier, sDishwasher, sBath, sHot Tub}, S(1) 2 = sHumidifier, S(2) 2 = {sDishwasher, sBath, sHot Tub}, S(1) 3 = sDishwasher, S(2) 3 = {sBath, sHot Tub}, S(1) 4 = sBath, S(2) 4 = {sHot Tub}, S(1) 5 = sHot Tub, S(2) 5 = ∅, where s⋆ is the order number for the corresponding device. The inherent sorting result is Irrigation → Humidifier → Dishwasher → Bath → Hot Tub, which corresponds to the disaggregation architecture we need to learn.

3.2 Disaggregation Structure Learning via Deep Sparse Coding

Given a set of k devices, we intend to identify a sequential set to reveal the optimal disaggregation order, which can maximize the disaggregation performance. Mathematically, given S = {1, 2, · · · , k}, we have the following optimization problem,

subject to:

Where E denotes the recursive disaggregation error, CS(1) i denotes the consumption of device S(1) i , CS(2) i = ∑|S(2) i | l=1 C(S(2) i )l ((S(2) i )l is the lth element in S(2) i ) represents the ith residual consumption, ¯CSi =CS(1) i +CS(2) i is the total consumption at the ith layer; BSi = [BS(1) i , BS(2) i ] denotes the discriminative dictionary which is learned by minimizing the difference between ˆASi and ASi , BS(j) i is the reconstruction dictionary, AS(j) i is the activations learned by sparse coding, ˆAS(j) i is the estimated activations for discrimination,

This optimization problem is a Non-Convex Mixed-Integer Nonlinear Programming (Non-Convex MINLP) problem, and it's NP-hard in general [Garey and Johnson, 1979]. Specifically, the time complexity of this problem increases exponentially with the number of involved devices, which will cause a prohibitively high time complexity for a large-sized S.

This motivates us to design the algorithm deep sparse coding to iteratively solve this problem based on the assumption that the choice of S(1) i (1 ≤ i ≤ k) can be learned sequentially. We introduce the objective function of deep sparse coding, which targets for the optimization with each layer,

The constraints for the above objective are similar with those formulated by Equations (3), (4), (5), (6), (7), but here we only consider those factors which affect the current layer i. Through the relaxation, in terms of the number of involved devices, the time complexity of the problem decreases from O(k!) to O(k2 ). The details of deep sparse coding algorithm are shown in Algorithm 1. We use Procedure RDSC to identify the device that needs to be disaggregated at each layer.

Algorithm 1 Deep sparse coding

Input: Ci: consumption for device i; λ : regularization parameter; α : gradient step learning rate; device set S = {1, · · · , k}.

Output: Si: layer-wise optimal sequential set of S; BS(j) i : discriminative dictionary; BS(j) i : reconstruction dictionary.

• 1: Initialization.
• 2: foreach i in S
• 3: (Ai ,Bi) ← SC(Ci).
• 4: m← k. l ← 1.
• 5: RDSC ( l, S, C1:m, B1:m, A1:m).

Procedure RDSC( l, S, C1:m, B1:m, A1:m)

• 1: if (S == ∅) Then RETURN.
• 2: foreach i in S

5:

(Bi, BRi , ˆAi, ˆARi )←DDSC ( ¯C, [Bi,BRi ], [Ai,ARi ]).

6: S(1)

l ← argmin

i∈S

Procedure SC(Ci)

• 1: Initialize Ai, Bi with non-negative values.
• 2: Iterate until convergence:
• 3: Ai ← argmin A≥0 12 ‖Ci -BiA‖2F + λ ∑p,q (A)p,q.

• 5: RETURN (Ai, Bi).

Procedure DDSC( ¯C, [B1, B2], [A1, A2])

• 1: A⋆1:2 ←A1:2. B1:2 ←B1:2. B← [B1, B2].
• 2: Iterate until convergence:

• 6: RETURN ( B1, B2, ˆA1, ˆA2 ).

RDSC can fully utilize Procedure DDSC in different layers to discriminatively optimize the bases B by reducing the difference between reconstruction and discrimination activations. Consequently, we can learn the sequential set Si of S, which specifies the disaggregation order. Both discriminative and reconstruction bases can also be learned for disaggregation.

Estimation of Device Level Consumption Based on the structure and parameters learned by deep learning algorithm (Algorithm 1), we can estimate device level consumption. The details of the disaggregation process are shown in Algorithm 2.

Ei. S(2)

l ←S \ {S(1)

l }. Sl ← {S(1)

l , S(2)

l }.

Algorithm 2 Device Level Consumption Estimation

Input: ¯C: aggregated consumption; Si: sequential set of S; BS(j) i : discriminative dictionary; BS(j) i : reconstruction dictionary.

Output: ˆCi: estimated consumption for device i.

• 1: Initialization.
• 2: for i ← 1 to k

• 4: ˆCS(1) i ← BS(1) i ˆA1.
• 5: ¯C←BS(2) i ˆA2.

4 Experimental Evaluation

In this section, we conducted a large scale water disaggregation task based on a real-world data set.

4.1 Experiment Setup

Data Set: We conducted water disaggregation with a data set collected by Aquacraft [Mayer et al., 1999]. The data set contains 1,959,817 water use events recorded during a two-year study from 1,188 residents across 12 study sites (e.g., San Diego, Denver, Eugene, and Seattle) located in six distinct regions of North America. Since the widely deployed smart meters are constrained to report at a low sample rate (typically 1/900 Hz), for generalization purpose, we converted the event records into interval based time series with a 1/900 Hz sample rate. In the course of experiments, we performed training/testing with respect to each household: 70% of the data for training, 30% of the data for testing.

Benchmark Methods: Two baselines were considered for comparison. One is DDSC model proposed in [Kolter et al., 2010], which was designed for energy disaggregation. The other one is FHMM based disaggregation approach presented in [Kim et al., 2011; Kolter and Johnson, 2011], which was capable of disaggregating power data. Since the features required for CFHMM in [Kim et al., 2011] are not available in the scenario of water disaggregation, we naturally used FHMM instead of CFHMM.

Evaluation Metrics: Both device and whole home level evaluation metrics were used to measure the performance. The whole home disaggregation performance was evaluated with accuracy and Normalized Disaggregation Error (NDE),

Where ˆCi = Bi ˆAi is the estimated consumption of device i. Large accuracy indicates that the estimated consumption can cover the true data well; while large NDE means that the estimated consumption is significantly different from the ground truth. In addition, the metrics used to evaluate the device level disaggregation performance include precision, recall, and Fmeasure. Precision is the fraction of estimated consumption that is correctly separated, recall indicates the fraction of true consumption that is successfully classified, and F-measure is calculated by 2 × precision×recall precision+recall .

4.2 Performance Evaluation and Comparison

Device Level Disaggregation Performance: We assessed DSCRDM, DDSC, and FHMM with 12 domestic water fixtures. Based on Algorithm 1, we learned the recursive disaggregation structure and its corresponding disaggregation sequence for these 12 water devices was 11 → 4 → 6 → 2 → 9 → 7 → 5 → 10 → 3 → 8 → 12 → 1.

As shown in Table 1, DSCRDM significantly outperformed DDSC and FHMM. Specifically, DSCRDM achieved 48.53% in average F-measure (the mean value of all devices' F-measure), while DDSC and FHMM only respectively achieved 22.89% and 16.80%. DSCRDM was capable of attaining more than 80% precision for Humidifier, Treatment, Irrigation, and Swimming Pool, and achieving more than 70% recall for Faucet and Shower. Interestingly, DDSC outperformed DSCRM in recall with respect to Dishwasher, Humidifier, Treatment, Hot Tub, and Bath, but DDSC reached a relatively low precision for these devices. The low precision and high recall phenomenon was caused by the fact that DDSC failed to identify the discriminative dictionaries, and the estimated consumption was much larger than true consumption, causing good recall and bad precision. Similarly, FHMM was able to achieve 48.55% in recall but only 9.74% in precision.

Household Level Disaggregation Performance: The whole home level accuracy and normalized disaggregation error comparison between DSCRDM, DDSC, and FHMM are shown in Figure 2. DSCRDM significantly outperformed FHMM with respect to accuracy. However, it failed to be more successful than DDSC. Referring to Equation 9, we could conclude that the estimated consumption of DDSC is much larger than the true consumption, which is also in accordance with the low precision and high recall phenomenon mentioned above. Since the low precision and high recall phenomenon occurred in five water fixtures, it largely distorted the authenticity of accuracy. NDE achieved by DSCRDM was as small as 0.7417, which was much smaller than those attained by either DDSC or FHMM. Since NDE measures the difference between the estimated and actual consumption, we could infer that DSCRDM better estimated the overall water usage.

Water Conservation with Detailed Feedbacks We applied DSCRDM to estimate the device level consumption and provide water usage information to users for water conservation. Figure 3 shows that the Actual Consumption (Actual Con.) in one home for a short period from 7:15am to 12:15am, along with the Estimated Consumption (Estimated Con.) in the testing set by DSCRDM. In whole home level, DSCRDM was capable of correctly capturing most of the actual consumption in spite of the small estimation error. The figure also illustrates both true and estimated time series of four selected devices of this home. In many cases, the devices Clothes Washer, and Dishwasher were estimated accurately. There were also cases DSCRDM could not perform estimation precisely, such as the premature estimation of the spike for Shower, and the underestimation of the peak of Faucet. Nonetheless, despite some poor estimations, the estimated water consumption is still quite informative, which could effectively help consumers save a significant amount of water. We investigated the general disaggregation performance of DSCRDM over all devices with the usage pie charts. As shown in Figure 4, DSCRDM could correctly identify the usage percentage information for most devices, although it misclassified some of Irrigation as Faucet. Such high level usage information could inform users that Irrigation occupies more than 70% of the total, indicating that further actions need to be taken, such as using the most water-efficient drip irrigation.

5 Conclusion

Water disaggregation, which separates total consumption into individual device consumption, has significant contribution to water conservation. In this paper, we examine the importance of structure learning for disaggregation task and present a machine learning model to automatically discover the decomposition structure. The extensible experiments showed that our algorithm significantly outperformed the benchmark methods. The estimated consumption results by our proposed model were specifically inspected, and detailed feedbacks could be provided to help customers for conservation.