{"type":"ScholarlyArticle","title":"GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction","authors":[{"name":"Yuxuan Liang","slug":"yuxuan-liang","given":"Yuxuan","family":"Liang","affiliation":"School of Computer Science and Technology , Xidian University , Xi'an , China; Urban Computing Business Unit , JD Finance , Beijing , China"},{"name":"Songyu Ke","slug":"songyu-ke","given":"Songyu","family":"Ke","affiliation":"Zhiyuan College , Shanghai Jiao Tong University , Shanghai , China; Urban Computing Business Unit , JD Finance , Beijing , China"},{"name":"Junbo Zhang","slug":"junbo-zhang","given":"Junbo","family":"Zhang","affiliation":"Urban Computing Business Unit , JD Finance , Beijing , China; School of Information Science and Technology , Southwest Jiaotong University , Chengdu , China"},{"name":"Xiuwen Yi","slug":"xiuwen-yi","given":"Xiuwen","family":"Yi","affiliation":"School of Information Science and Technology , Southwest Jiaotong University , Chengdu , China; Urban Computing Business Unit , JD Finance , Beijing , China"},{"name":"Yu Zheng","slug":"yu-zheng","given":"Yu","family":"Zheng","affiliation":"Urban Computing Business Unit , JD Finance , Beijing , China; School of Computer Science and Technology , Xidian University , Xi'an , China; Zhiyuan College , Shanghai Jiao Tong University , Shanghai , China; School of Information Science and Technology , Southwest Jiaotong University , Chengdu , China"}],"abstract":"Numerous sensors have been deployed in different geospatial locations to continuously and cooperatively monitor the surrounding environment, such as the air quality. These sensors generate multiple geo-sensory time series, with spatial correlations between their readings. Forecasting geo-sensory time series is of great importance yet very challenging as it is affected by many complex factors, i.e., dynamic spatio-temporal correlations and external factors. In this paper, we predict the readings of a geo-sensor over several future hours by using a multi-level attention-based recurrent neural network that considers multiple sensors' readings, meteorological data, and spatial data. More specifically, our model consists of two major parts: 1) a multi-level attention mechanism to model the dynamic spatio-temporal dependencies. 2) a general fusion module to incorporate the external factors from different domains. Experiments on two types of real-world datasets, viz., air quality data and water quality data, demonstrate that our method outperforms nine baseline methods.","doi":"10.24963/ijcai.2018/476","url":"https://ijcai.org/proceedings/2018/476","pdf":"https://ijcai.org/proceedings/2018/0476.pdf","markdown":"https://ijcai.org/proceedings/2018/476.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/476","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning Applications","track":"Main track","number":476,"pages":{"first":3428,"last":3434},"keywords":["Machine Learning: Deep Learning","Machine Learning: Time-series;Data Streams","Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning","Machine Learning Applications: Environmental"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Ashish Agarwal, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467, 2016.","title":"Tensorflow: Large-scale machine learning on heterogeneous distributed systems","authors":"Abadi","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine transla- tion by jointly learning to align and translate. CoRR, abs/1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Box and Pierce, 1970] George EP Box and David A Pierce. Distribution of residual autocorrelations in autoregressive- integrated moving average time series models. Journal of the American statistical Association, 65(332):1509-1526, 1970.","title":"Distribution of residual autocorrelations in autoregressiveintegrated moving average time series models","authors":"Pierce ; George Ep Box, David A Box, Pierce","year":"1970","venue":"Journal of the American statistical Association","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cho et al., 2014a] Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the proper- ties of neural machine translation: Encoder-decoder ap- proaches. arXiv preprint arXiv:1409.1259, 2014.","title":"On the properties of neural machine translation: Encoder-decoder approaches","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cho et al., 2014b] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.","title":"Learning phrase representations using rnn encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Friedman, 2001] Jerome H Friedman. Greedy function ap- proximation: a gradient boosting machine. Annals of statistics, pages 1189-1232, 2001.","title":"Greedy function approximation: a gradient boosting machine","authors":"Jerome H Friedman","year":"2001","venue":"Annals of statistics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Koller and Friedman, 2009] Daphne Koller and Nir Fried- man. Probabilistic graphical models: principles and tech- niques. MIT press, 2009.","title":"Probabilistic graphical models: principles and techniques","authors":"Friedman Koller","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liang et al., 2017] Yuxuan Liang, Zhongyuan Jiang, and Yu Zheng. Inferring traffic cascading patterns. ACM SIGSPATIAL 2017, November 2017.","title":"Inferring traffic cascading patterns","authors":"Yuxuan Liang, Zhongyuan Jiang, Yu Zheng","year":"2017","venue":"ACM SIGSPATIAL 2017","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lin et al., 1996] Tsungnan Lin, Bill G Horne, Peter Tino, and C Lee Giles. Learning long-term dependencies in narx recurrent neural networks. IEEE Transactions on Neural Networks, 7(6):1329-1338, 1996.","title":"Learning long-term dependencies in narx recurrent neural networks","authors":"Lin","year":"1996","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2016a] Ye Liu, Yuxuan Liang, Shuming Liu, David S Rosenblum, and Yu Zheng. Predicting ur- ban water quality with ubiquitous data. arXiv preprint arXiv:1610.09462, 2016.","title":"Predicting urban water quality with ubiquitous data","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2016b] Ye Liu, Yu Zheng, Yuxuan Liang, Shum- ing Liu, and David S. Rosenblum. Urban water quality prediction based on multi-task multi-view learning. In Proceedings of the Twenty-Fifth International Joint Con- ference on Artificial Intelligence, IJCAI'16, pages 2576- 2582, 2016.","title":"Urban water quality prediction based on multi-task multi-view learning","authors":"Liu","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI'16","doi":null,"url":null,"internal_url":"/proceedings/2016/366"},{"position":13,"raw":"[Lv et al., 2015] Yisheng Lv, Yanjie Duan, Wenwen Kang, Zhengxi Li, and Fei-Yue Wang. Traffic flow prediction with big data: a deep learning approach. IEEE Trans- actions on Intelligent Transportation Systems, 16(2):865- 873, 2015.","title":"Traffic flow prediction with big data: a deep learning approach","authors":"Yisheng Lv, Yanjie Duan, Wenwen Kang, Zhengxi Li, Fei-Yue Wang","year":"2015","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Qin et al., 2017] Yao Qin, Dongjin Song, Haifeng Cheng, Wei Cheng, Guofei Jiang, and Garrison Cottrell. A dual- stage attention-based recurrent neural network for time se- ries prediction. arXiv preprint arXiv:1704.02971, 2017.","title":"A dualstage attention-based recurrent neural network for time series prediction","authors":"Dongjin Yao Qin, Haifeng Song, Wei Cheng, Guofei Cheng, Garrison Jiang, Cottrell","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Rossman et al., 1994] Lewis A Rossman, Robert M Clark, and Walter M Grayman. Modeling chlorine residuals in drinking-water distribution systems. Journal of environ- mental engineering, 120(4):803-820, 1994.","title":"Modeling chlorine residuals in drinking-water distribution systems","authors":"Robert M Lewis A Rossman, Clark, Walter M Grayman","year":"1994","venue":"Journal of environmental engineering","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rumelhart et al., 1986] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, 323(6088):533, 1986.","title":"Learning representations by back-propagating errors","authors":"Geoffrey E David E Rumelhart, Ronald J Hinton, Williams","year":"1986","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2005] Ying Wang, Guoshun Zhuang, Aohan Tang, Hui Yuan, Yele Sun, Shuang Chen, and Aihua Zheng. The ion chemistry and the source of pm2. 5 aerosol in beijing. Atmospheric Environment, 39(21):3771-3784, 2005.","title":"The ion chemistry and the source of pm2. 5 aerosol in beijing","authors":"Ying Wang, Guoshun Zhuang, Aohan Tang, Hui Yuan, Yele Sun, Shuang Chen, Aihua Zheng","year":"2005","venue":"Atmospheric Environment","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2016] Linlin Wang, Zhu Cao, Gerard de Melo, and Zhiyuan Liu. Relation classification via multi-level attention cnns. In ACL (1), 2016.","title":"Relation classification via multi-level attention cnns","authors":"Linlin Wang, Zhu Cao, Gerard De Melo, Zhiyuan Liu","year":"2016","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2018] Dong Wang, Junbo Zhang, Wei Cao, Jian Li, and Yu Zheng. When will you arrive? estimat- ing travel time based on deep neural networks. 2018.","title":"When will you arrive? estimating travel time based on deep neural networks","authors":"Dong Wang, Junbo Zhang, Wei Cao, Jian Li, Yu Zheng","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yu et al., 2017] Dongfei Yu, Jianlong Fu, Tao Mei, and Yong Rui. Multi-level attention networks for visual ques- tion answering. In Conf. on Computer Vision and Pattern Recognition, volume 1, page 8, 2017.","title":"Multi-level attention networks for visual question answering","authors":"Yu","year":"2017","venue":"Conf. on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2016] Junbo Zhang, Yu Zheng, Dekang Qi, Ruiyuan Li, and Xiuwen Yi. Dnn-based prediction model for spatio-temporal data. ACM SIGSPATIAL 2016, Octo- ber 2016.","title":"Dnn-based prediction model for spatio-temporal data","authors":"Junbo Zhang, Yu Zheng, Dekang Qi, Ruiyuan Li, Xiuwen Yi","year":"2016","venue":"ACM SIGSPATIAL 2016","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2017] Junbo Zhang, Yu Zheng, and Dekang Qi. Deep spatio-temporal residual networks for citywide crowd flows prediction. In AAAI, pages 1655-1661, 2017.","title":"Deep spatio-temporal residual networks for citywide crowd flows prediction","authors":"Junbo Zhang, Yu Zheng, Dekang Qi","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zheng et al., 2015] Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, and Tianrui Li. Forecasting fine-grained air quality based on big data. In Proceedings of the 21th ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining, pages 2267-2276. ACM, 2015.","title":"Forecasting fine-grained air quality based on big data","authors":"Yu Zheng, Xiuwen Yi, Ming Li, Ruiyuan Li, Zhangqing Shan, Eric Chang, Tianrui Li","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ziv, 2006] Vector Autoregressive Models for Multivariate Time Series, pages 385-429. Springer New York, New York, NY, 2006.","title":"Vector Autoregressive Models for Multivariate Time Series","authors":"Ziv","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}