Autoencoder Regularized Network For Driving Style Representation Learning

1 Introduction

Studying human drivers' driving behaviors from automobile sensor data is an interesting research topic. Similar to biometrics such as gait, voice, and typing rhythm, each driver also has a signature pattern of driving, which is also called driving style [Lin et al., 2014]. There are many aspects that can be used to measure driving styles. In this paper, we focus on studying vehicle movement measures including speed change, turning, and their temporal combinations derived from GPS (Global Positioning System) sensor data that are collected in a short and regular time interval (e.g., 1 second). These measures can reflect drivers' fine-grained behavioral habits of steering and speed control.

Learning driving style representations from automobile sensor data has been intensively studied [Van Ly et al., 2013; Lin et al., 2014; Kuderer et al., 2015; Dong et al., 2016]. Compared with other automobile sensors, such as OBD (OnBoard Diagnostic) system, CAN (Controller Area Network) buses and cameras, GPS sensor data are often easier to collect, making them popular in large-scale research. Auto insurance companies have become highly interested in utilizing driving style information extracted from GPS data to solve their business problems [Laurie, 2011]. A good driving style representation can help answer questions such as if the driver identified on the policy is driving the proper car, how many drivers share a car, and if an additional driver is driving a car, etc. Insights to these questions can be critical for risk evaluation and can be applied to policy premiums in insurance programs such as pay-as-you-drive. Besides, a good driving style representation also helps to better modeling and understanding human drivers' behaviors, which are beneficial to improving the designs of driving assistance systems, drivercar interactions, and autonomous driving [Lin et al., 2014; Kuderer et al., 2015].

∗Corresponding author.

Existing approaches typically follow the supervised learning paradigm, where the inputs are vehicle movement features derived from GPS data and the labels are drivers' identities. The learning process is usually guided by minimizing a classification loss. The driving style representation (features) learnt in this way can work well in describing unseen trips of seen drivers. Nonetheless, the learnt representation is not guaranteed to be a good generalized representation on unseen drivers. When the number of drivers in the training set is small, the learnt model can hardly work well since unseen drivers' driving behaviors can be extremely diverse in practice. On the other hand, collecting data from a large number of drivers and ensuring a sufficiently large trip training set for each driver can be challenging. Besides, when the number of drivers becomes large (say, thousands), the classifier can become difficult to train.

To solve the problem, in this paper, we propose a novel deep neural network, Autoencoder Regularized Network (ARNet) for generalized driving style representation learning. Figure 1 illustrates the overall architecture. Different from existing deep neural networks, ARNet directly learns from GPS data and combines supervised and unsupervised feature learning in one architecture. The motivation is to use a specially designed autoencoder structure to regularize the discriminative feature learning in a classification network. In ARNet, supervised learning is combined with unsupervised feature reconstruction using a Recurrent Neural Network (RNN) [Elman, 1990; Chung et al., 2014] output as a shared hidden layer. An l1 regularized bottleneck layer (fc1 in Figure 1) of the autoencoder structure serves as the final driving style feature representation extraction layer. The feature learning is guided simultaneously by a classification loss defined on trip labels and a reconstruction loss defined on how well the driving style feature layer reconstructs the hidden RNN output. Notably, the autoencoder in ARNet aims to reconstruct the hidden-layer RNN feature that keep changing in training, but not to reconstruct the fixed network inputs as in typical autoencoder networks. Such a design can be viewed as a form of regularization to the hidden-layer RNN feature for classification: The feature should be discriminative, meanwhile, they should be reconstructible. The bottleneck layer of the autoencoder (fc1) thus can learn the basis of driving styles, which is expected to generalize better on unseen drivers. Reversely, the design can also be regarded as introducing supervisory information to unsupervised feature learning: Labels of limited training samples bring in prior knowledge to the unsupervised autoencoder, making the learnt basis feature more meaningful and discriminative. Furthermore, we also propose a trip encoding framework, namely trip2vec, which encodes a varied-length GPS trip into a fixed-length vector describing the trip-level driving style using the proposed ARNet as the base encoder.

We study two problems as benchmarks on a large real dataset. The first one is a new and challenging real-world problem raised from the auto insurance industry, called driver number estimation. The objective is to identify the true number of drivers from a set of anonymous trips. Importantly, these drivers are new and unseen to the training phase. Largescale experimental studies show that ARNet significantly outperforms alternative methods. On a wide range of tests (from 1 to 10 drivers), the average absolute error between the estimation and the ground truth is just 0.68 (less than one driver). In contrast, other candidate methods lead to errors much larger than 1. The second problem is the classical driver identification problem, measured by the classification accuracy on unseen trips of seen drivers. Experiments on a 50-class problem show that, ARNet reaches the highest classification accuracy by at least 3% improvement compared with several existing classification-based methods.

2 Autoencoder Regularized Network (ARNet) And Trip2vec Encoding

We first introduce ARNet that reads GPS data as inputs and learns a compact driving style feature representation. Then, we introduce trip2vec, a trip encoding framework, which extends the learnt driving style representation to trip-level.

2.1 GPS Data Transformation

A trip (i.e., GPS trajectory) can be defined by a variedlength sequence of tuples (u, v, t), where (u, v) denotes a geo-location and t denotes time. We follow the data transformation method proposed by [Dong et al., 2016] to construct neural network inputs from raw GPS data, which has been proved an effective way of extending deep learning to working on GPS data. A trip is first windowed into segments of a fixed length Ls with a shift Ls/2, each of which encodes five instantaneous car movement features, namely basic features, derived from neighboring GPS data: speed norm, difference of speed norm, acceleration norm, difference of acceleration norm, and angular speed. Then, each segment is further applied a sliding window of length Lf (Lf < Ls) with a shift Lf/2. Each window produces a frame encoding seven statistics of the basic features: mean, minimum, maximum, 25%, 50% and 75% quartiles, and standard deviation. As a result, a set of statistical feature matrices of 5 × 7 = 35 rows and 2 × Ls/Lf columns can be obtained from a given trip. A feature matrix describing one trip segment defines an input sample to neural networks. For instance, given GPS data sampled once per second as in the experiments, using Ls = 256s,Lf = 4s, we define feature matrices of size 35 × 128 as network inputs. The label of a sample inherits from the trip to which the corresponding segment belongs.

2.2 Autoencoder Regularized Network (ARNet)

The proposed ARNet architecture is depicted in Figure 1, which consists of three parts: a stacked RNN, an autoencoder for reconstruction, and a softmax for classification.

Stacked RNN

Let x denote the 35×128 input, i.e., a trip segment. A stacked RNN (gru1+gru2+dropout in Figure 1) reads x to extract higher-level features. As driving style is typically the temporal combination of driving actions, we regard x as a sequence of length 128 and each element of which as a 35-d vector. Here we employs a 2-layer stacked GRU (Gated Recurrent Unit) [Cho et al., 2014] architecture to exploit the sequential dependencies. GRU network has been proved an effective RNN design [Chung et al., 2014]. Our empirical studies show that GRU works slightly better than several popular RNN architectures on our problem, including LSTM [Hochreiter and Schmidhuber, 1997] and bi-directional RNN [Schuster and Paliwal, 1997], hence we adopt GRU in the design. The first GRU layer (gru1) reads 35 × 128 input x with unrolling itself 128 steps along the time axis, and outputs a sequence of a same length 128, each element of which is a vector. The size of the vector (i.e., dimension) equals to the number of hidden units in gru1. The second GRU layer (gru2) is appended to gru1. It also unrolls 128 steps, but outputs a vector instead of a sequence. The size of the vector equals to the number of hidden units in gru2. A dropout layer is applied to gru2 to reduce overfitting [Hinton et al., 2012]. This dropout layer plays as the shared hidden feature layer bridging the supervised and unsupervised learning. Let ˜x denote its output given an input x.

Autoencoder

A 3-layer autoencoder (dropout+fc1+fc2 in Figure 1) is employed for feature reconstruction. Notably, we use the autoencoder to reconstruct ˜x instead of x, which is critical for learning better generalized driving style representation. A fullyconnected bottleneck layer (fc1) is used to learn a compressed representation s of ˜x. ReLU nonlinearity f(z) = max(0, z) is used in fc1 to ensure s non-negative, which will be used in the trip2vec encoding (see Eq (6)). l1 sparsity regularization is applied on s (see Eq (1)). A fully-connected layer (fc2) is the output layer of the autoencoder, where f(z) = tanh(z) activations are used to approximate ˜x for reconstruction.

Softmax regression

A fully-connected layer (fc3 in Figure 1) appending to the dropout layer is defined for classification. A softmax regression is applied to produce a distribution over class labels. The number of classes (denoted by c) equals to the number of drivers in the training set.

Objective function and approximation

Given a training set {xi, yi} where i ∈ {1, . . . , n}, yi ∈ {1, . . . , c}, the overall objective function is defined as a combination of reconstruction and classification objectives. The reconstruction loss is defined as:

where ˜xi is the output of the RNN dropout layer, D ∈ R(n×k) is a "dictionary" of k vectors, si ∈ Rk is a "code vector" associated with the ˜xi. The first term of Jr is the reconstruction error, which intends to find a dictionary D and a new representation si to reconstruct ˜xi, the learnt feature fromRNN. l1 regularization is used to encourage si to be sparse. Eq (1) is a sparse coding objective [Olshausen and others, 1996]. The classification loss is defined as the standard cross-entropy:

where 1{·} is the indicator function and θ = {θ1, . . . , θc} are the softmax regression parameters. Overall, the combined objective function is defined as:

The motivation of Eq (1) is as follows. If excluding the autoencoder layers (fc1+fc2), the network becomes a stacked RNN and ˜x can be used as a feature representation of x. In this way, the learning of ˜x is guided only by supervisory information of trip labels (i.e., driver IDs). The dropout can help reduce overfitting, but considering the number of unseen drivers can be extremely large, given limited training data, the learning is still prone to overfit to the seen drivers in the training set. Therefore, ˜x can hardly be a good representation for unseen drivers. As a straightforward extension, we want to learn a representation which has the clustering characteristic and be more compact to have better generalization performance. Minimizing Jr can help to achieve this goal. Let us look at the objective of a classical clustering algorithm, K-means [Coates and Ng, 2012]:

which intends to find cluster centroids uk that minimize the distance between data points and the nearest centroid. Equally to Eq (4), K-means can be viewed as a way of reconstructing xi [Coates and Ng, 2012]:

Compare Eq (1) and Eq (5), they optimize the same type of reconstruction objective. The only difference is that Eq (1) allows more than one non-zero entry in each si, enabling a much more accurate representation of each xi while still requiring each si to be simple. Therefore, minimizing Jr can make the learnt representation have clustering characteristics and be more compact.

To achieve the goal of minimizing Eq (1), in ARNet, we use the layers dropout+fc1+fc2 to approximate so as to make the unified architecture easier to train. Known as the sparse coding objective, Eq (1) intends to learn a sparse reconstruction and an efficient coding for ˜xi, which shares the spirit of sparse autoencoders [Ng, 2011]. The dropout+fc1+fc2 is a typical autoencoder structure, and we use l1 regularization on the output of fc1 so that the coding for ˜xi is sparse.

We can read ARNet from two perspectives. On one hand, we can view Jr (Eq (1)) as regularization terms if regarding Jc (Eq (2)) as the main loss. The autoencoder structure plays as a regularizer to the classification feature ˜x, and should improve classification performance. On the other hand, we can view Jc as a term introducing prior knowledge to the unsupervised feature learning guided by Jr. The finally learnt feature s thus should work better than features learnt from purely unsupervised learning. Experimental studies in the next section will verify these from both sides.

2.3 Trip2vec: A Trip Encoding Framework

Once we have trained the ARNet, we can use the layers gru1+gru2+dropout+fc1 as a trip segment encoder. But still, it extracts driving style information only from trip data on segment-level, which can be impacted by local factors such as road shapes and traffic conditions, etc. Therefore, we need a more robust representation that describes trip-level driving styles. In order to do so, we propose a trip encoding framework, called trip2vec, which adopts the Bagof-Words (BoW) feature construction strategy [Fei-Fei and Perona, 2005] based on the trained trip segment encoder. Roughly speaking, we can treat a varied-length trip as an "article" and each segment as a "paragraph". The overall "topic" of an article can be derived from aggregating the paragraphlevel information. Similarly, based on the segment-level driving styles, we propose to define the trip-level driving style representation by the normalized sum of all the segment-level feature vectors. Figure 2 illustrates the trip2vec framework. Suppose a trip tr is divided into q segments, and the encoded segment features are {str i }, i ∈ {1, . . . , q}. The trip-level driving style feature representation is defined as:

where σtr = ∑qi str i is the vector sum, and σtr j denotes its j-th dimension (σtr j >= 0, ∀tr, j).

3 Experiments

We use a large real yet private dataset in experiments. The dataset is collected by an insurance company, containing over 500,000 GPS trips from over 2,500 drivers. Each driver has 200 trips that record the car's location every second.

3.1 Driver Number Estimation Problem

We first study the driver number estimation problem. The aim is to estimate the number of drivers from a set of anonymous trips. The driving style representation learning is based on labeled trips of a set of known drivers, but the testing trips are from unseen new drivers. This is to mimic the situations in real-world, where the auto insurance companies are interested to know how many drivers share a car given this car's recorded trips. However, the driver ID is unknown for the trips. More importantly, the potential drivers are most likely new, meaning their data are not available in model training, which makes the problem a challenging one. A precise estimation can help improve the risk modeling and pricing policies and to generate direct business values.

Experimental settings

For comparisons, we include two alternative architectures: a reconstruction-only network (RONet) and a classificationonly network (CONet), which are defined by removing one of the two losses from ARNet. For all the nets, we set 256 hidden units in gru1 and gru2, thus dropout output ˜x and fc2 output are both 256-d. We use dropout probability 0.5. We set 50 hidden units in fc1, thus the final driving style representation learnt by ARNet is a 50-d vector. We use λ=1e-5, ADADELTA optimizer [Zeiler, 2012] with learning rate 1.0, ρ=0.95 and ϵ=1e-8, and batch size 2560 in training these networks. The training is based on the trip data from the first 50 drivers in the dataset. For each driver, we use 80% trips as training data, and the rest 20% as classification validation data. Training ARNet and CONet stops until the validation accuracy is maximized (at epochs 33 and 116, respectively). Training RONet stops when the reconstruction loss Jr converges around 0.001 (at epoch 100). We use fc1 as the driving style feature layer for ARNet and RONet, and use the dropout layer for CONet. Trip features are computed using the proposed trip2vec framework for all the nets.

We also include a 57-d handcrafted trip feature representation proposed by [Dong et al., 2016] as another baseline, which demonstrated good classification performance working with GBDT (Gradient Boosting Decision Tree) [Friedman, 2001]. We denote it by TripGBDT feature. The 57 features include global and local driving behavior statistics. The global ones are trip-level statistics of speed, acceleration, and angular speed, total trip time duration, total trip length, trip average speed, and size of the minimal bounding rectangle describing trip geometry. The local ones are statistics of movement features calculated on different time scales with correlation to binned local road shapes. We will compare it with those trip features learnt by the deep architectures.

Based on the trip features, we employ Affinity Propagation (AP) [Frey and Dueck, 2007] to cluster the trips and estimate the number of drivers. Assuming different drivers should have different driving styles, each obtained cluster refers to the trips belong to one driver, and thus the number of clusters reflects the number of drivers. AP has the advantage of automatically determining the number of clusters. We employ the scikit-learn implementation of AP [Pedregosa et al., 2011] using the Euclidean affinity, where a preference parameter is needed. We tuned this parameter for each candidate, and chose preference values -5, -8.5, -12, and -3.5 for ARNet, RONet, CONet, and TripGBDT features, respectively. This is to reduce the effect of clustering algorithm and to fairly compare different feature representations. We use the default damping factor 0.5 in AP, since empirical studies showed that the results are insensitive to this parameter.

Design of testing

We sample from the unseen drivers (ID greater than 50) who are absent in training to construct testing sets. We build 10 test groups. Each corresponds to a fixed number of drivers, ranging from 1 to 10. For each test group, we randomly sample 25 times from the unseen drivers. As a result, each test group contains 25 trip sets, and each trip set refers to a random combination of drivers. We compute two metrics: (1) the absolute error between the true number of drivers and the estimates, and (2) the AMI (Adjusted Mutual Information) score [Vinh et al., 2010] measuring the clustering quality, which returns a value of 1 when the cluster partitions are perfectly matched with the true labels, while random partitions have an expected AMI around 0. For each test group, we report mean and standard deviation of these two metrics on the 25 runs. We also report the overall averaged mean performance across the 10 test groups for each candidate feature representation.

Results

Results are shown in Tables 1 and 2. The best entries in each test group (row) are bolded. We can see that as the number of driver grows, the problem becomes harder. Overall, clustering based on ARNet feature demonstrates the best performance among all. Table 1 shows that ARNet feature leads to the mean error less than one driver in 9 out of the 10 tests. It wins 5 out of the 10 tests with the least mean error, and places the second best in the 5 lost ones all by a small margin (0.12 at most). Its averaged mean error of all the tests is just 0.68. In contrast, the single-loss networks' features and the TripGBDT feature more often leads to larger errors. They all have the averaged mean error greater than 1. Table 2 shows that ARNet feature also leads to the best clustering quality. It wins 9 (including a tie) of the 10 tests with the highest mean AMI, only with lost the 9-th test by 0.01. Its averaged mean AMI 0.32 is the highest, while other candidates' are much worse. We depict the box plots of the results in Figures 3 and 4, where A, R, C and T stand for ARNet, RONet, CONet and TripGBDT features, respectively. From the comparisons, we can conclude that the ARNet results are often significantly better. The handcrafted TripGBDT feature performs the worst based on current data, implying the superiority of learning representations by deep networks.

We studied how the estimation error changes with the AP preference setting, as shown in Figure 5. We can see that the chosen thresholds lead to the best performance for each candidate, revealing that the advantage of ARNet feature is not due to the thresholding of clustering.

We further use t-SNE [Maaten and Hinton, 2008] to project the trip feature representations onto a 2-d space for visual comparisons. Again, we employ the scikit-learn [Pedregosa et al., 2011] t-SNE implementation with all parameters kept default. Figure 6 shows typical results on both seen and unseen drivers. We can see that the ARNet feature is robust: Trips of a same driver show relatively similar driving styles to each other, exhibiting clear clustering patterns, no matter the drivers are seen or unseen. In contrast, due to the absence of supervisory information, RONet learns "too generalized" feature that cannot differentiate drivers well, leading to poor clustering quality (small AMI scores in Table 2). The CONet feature is highly discriminative on seen trips and drivers. But on unseen drivers, it more easily split same-class trips faraway (driver ID 1267 in red). Similar to RONet result, the TripGBDT feature does not reflect clear clustering patterns, explaining its poor performance. In a word, ARNet learns a better generalized driving style representation, resulting in interpretable better performance on the problem.

3.2 Driver Identification Problem

We now study the classical driver identification problem to demonstrate that ARNet also helps improve the driver identification (i.e., classification) accuracy. Based on the same 50 drivers used in training on the driver number estimation problem, we train different candidate models and compare both segment-level and trip-level driver classification accuracies (50-class), where the predictions indicate to which driver the segment/trip belongs. A trip-level prediction is obtained by summing up the predictions on segments of this trip to give a vote weighted by confidence scores. For each driver, we use 80% trips for training and the rest 20% for testing. Note that here the testing trips are all from seen drivers, though the trips are excluded in training.

Candidate methods

In addition to ARNet and CONet, we include five supervised deep networks, NoPoolCNN, CNN, PretrainIRNN, IRNN and StackedIRNN studied in [Dong et al., 2016] and two GBDT methods, the TripGBDT and a GBDT reading the 35 × 128 input as a flattened vector. ARNet and CONet settings are kept unchanged as in the driver number estimation experiments. In ARNet, the fc3 output is used for prediction.

Results

Table 3 summarizes both segment and trip level accuracies. ARNet outperforms all the other candidates with the highest accuracies, segment 40.4%, trip top-1 58.2%, and trip top-5 78.3%. It improves the accuracies by roughly 3% compared with CONet, which performs the second best in terms of segment and trip top-1 performance. This verifies that ARNet that improves the supervised learning performance.

4 RelatedWork

Existing approaches on driving style learning usually follow the supervised learning paradigm, whether or not the input is GPS data. Many methods based on non-deep-learning classifiers and reinforcement learning were proposed, e.g., by [Letchner et al., 2006; Ziebart et al., 2008; Quintero M. et al., 2012; Quek and Ng, 2013; Van Ly et al., 2013; Kuderer et al., 2015]. Recently, [Dong et al., 2016] extended deep learning to GPS data and proposed several CNNs and RNNs that can learn interpretable driving style features. But still, these are typical supervised classification networks.

Though in literatures, there are plenty of neural networks combining supervised and unsupervised learning, e.g., [Lee and Lin, 1992; Karayiannis and Mi, 1997; Raina et al., 2007; Collobert and Weston, 2008], few attempts were made on employing autoencoder as a special regularizer to supervised learning as in ARNet. Especially, the autoencoder in ARNet is not for pre-training or applying unsupervised/supervised learning in turn.

ARNet can be viewed as a special case of multi-task learning (MTL) if regarding classification and reconstruction as two tasks. However, MTL typically learns a shared representation across tasks [Caruana, 1997; Argyriou et al., 2007], and in most (if not all) cases, the tasks are either all supervised or all unsupervised. The only MTL method combining unsupervised reconstruction with supervised learning that we are aware of is the Semi-supervised Autoencoder for Multitask Learning (SAML) [Zhuang et al., 2015]. In SAML, each task combines reconstruction and classification, and a shared autoencoder reconstructs the network input via shared feature layers used also for classification. Most differently, the autoencoder in ARNet reconstructs the shared hidden-layer feature instead of the network input. Also, the two tasks in ARNet (reconstruction and classification) use different (fc1 and fc3) but not a shared representation. ARNet's architecture can also be seen as a special case of the auxiliary mode of the Semi-Supervised Embedding deep learning framework [Weston et al., 2008]. However, [Weston et al., 2008] trains the auxiliary branch to only embed unlabeled data, thus if without unlabeled data, it becomes standard supervised learning. In contrast, the two branches of ARNet always process the same input data simultaneously. Besides, these existing methods are not designed for GPS data. To the best of our knowledge, ARNet is the first attempt on combining unsupervised autoencoder and supervised learning in a unified deep architecture for learning from GPS data.

5 Conclusion

In this paper, we study learning driving style representation from GPS data, and propose a novel deep architecture, Autoencoder Regularized Network (ARNet), which combines unsupervised and supervised feature learning by introducing an autoencoder as a special regularizer to supervised learning. ARNet can also be viewed as adding supervisory information to unsupervised feature learning of the autoencoder. In both ways, it improves the quality of learnt driving style representation. We further propose trip2vec, a trip encoding framework using ARNet as the base encoder to extract trip-level driving styles. Experiments on benchmark problems verify the advantages of ARNet over existing methods, especially on characterizing new drivers. Future work includes studying ARNet on other related problems such as to early detect abnormal driving status (e.g., drunk, fatigue, and drowsy) and those representation learning problems in other domains.