{"type":"ScholarlyArticle","title":"ATTAIN: Attention-based Time-Aware LSTM Networks for Disease Progression Modeling","authors":[{"name":"Yuan Zhang","slug":"yuan-zhang","given":"Yuan","family":"Zhang","affiliation":"Computer Science , North Carolina State University"},{"name":"Xi Yang","slug":"xi-yang","given":"Xi","family":"Yang","affiliation":"Computer Science , North Carolina State University"},{"name":"Julie Ivy","slug":"julie-ivy","given":"Julie","family":"Ivy","affiliation":"Industrial and System Engineering , North Carolina State University"},{"name":"Min Chi","slug":"min-chi","given":"Min","family":"Chi","affiliation":"Computer Science , North Carolina State University"}],"abstract":"Modeling patient disease progression using Electronic Health Records (EHRs) is critical to assist clinical decision making. Long-Short Term Memory (LSTM) is an effective model to handle sequential data, such as EHRs, but it encounters two major limitations when applied to EHRs: it is unable to interpret the prediction results and it ignores the irregular time intervals between consecutive events. To tackle these limitations, we propose an attention-based time-aware LSTM Networks (ATTAIN), to improve the interpretability of LSTM and to identify the critical previous events for current diagnosis by modeling the inherent time irregularity. We validate ATTAIN on modeling the progression of an extremely challenging disease, septic shock, by using real-world EHRs. Our results demonstrate that the proposed framework outperforms the state-of-the-art models such as RETAIN and T-LSTM. Also, the generated interpretative time-aware attention weights shed some lights on the progression behaviors of septic shock.","doi":"10.24963/ijcai.2019/607","url":"https://ijcai.org/proceedings/2019/607","pdf":"https://ijcai.org/proceedings/2019/0607.pdf","markdown":"https://ijcai.org/proceedings/2019/607.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/607","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":607,"pages":{"first":4369,"last":4375},"keywords":["Multidisciplinary Topics and Applications: Biology and Medicine","Machine Learning Applications: Applications of Supervised Learning","Machine Learning: Time-series;Data Streams","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Baytas et al., 2017] Inci M Baytas, Cao Xiao, Xi Zhang, Fei Wang, Anil K Jain, and Jiayu Zhou. Paient subtyping via time-aware lstm networks. In SIGKDD. ACM, 2017.","title":"Paient subtyping via time-aware lstm networks","authors":"Inci M Baytas, Cao Xiao, Xi Zhang, Fei Wang, Anil K Jain, Jiayu Zhou","year":"2017","venue":"SIGKDD. ACM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Che et al., 2015] Zhengping Che, David Kale, Wenzhe Li, Mohammad Taha Bahadori, and Yan Liu. Deep computa- tional phenotyping. In SIGKDD. ACM, 2015.","title":"Deep computational phenotyping","authors":"Zhengping Che, David Kale, Wenzhe Li, Mohammad Taha Bahadori, Yan Liu","year":"2015","venue":"SIGKDD. ACM","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Che et al., 2017] Chao Che, Cao Xiao, Jian Liang, Bo Jin, Jiayu Zho, and Fei Wang. An rnn architecture with dy- namic temporal matching for personalized predictions of parkinson's disease. In SDM. SIAM, 2017.","title":"An rnn architecture with dynamic temporal matching for personalized predictions of parkinson's disease","authors":"Cao Chao Che, Jian Xiao, Bo Liang, Jiayu Jin, Fei Zho, Wang","year":"2017","venue":"SDM. SIAM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Che et al., 2018] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.","title":"Recurrent neural networks for multivariate time series with missing values","authors":"Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, Yan Liu","year":"2018","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choi et al., 2016a] Edward Choi, Mohammad Taha Ba- hadori, Andy Schuetz, Walter F Stewart, and Jimeng Sun. Doctor ai: Predicting clinical events via recurrent neural networks. In MLHC, pages 301-318, 2016.","title":"Doctor ai: Predicting clinical events via recurrent neural networks","authors":"Choi","year":"2016","venue":"MLHC","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi et al., 2016b] Edward Choi, Mohammad Taha Ba- hadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In NIPS, pages 3504-3512, 2016.","title":"Retain: An interpretable predictive model for healthcare using reverse time attention mechanism","authors":"Choi","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dellinger et al., 2008] R Phillip Dellinger, Mitchell M Levy, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2008. Intensive care medicine, 2008.","title":"Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock","authors":"Phillip Dellinger, Mitchell M Levy","year":"2008","venue":"Intensive care medicine","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Esteban et al., 2016] Cristóbal Esteban, Oliver Staeck, et al. Predicting clinical events by combining static and dynamic information using recurrent neural networks. In ICHI, pages 93-101. IEEE, 2016.","title":"Predicting clinical events by combining static and dynamic information using recurrent neural networks","authors":"Esteban","year":"2016","venue":"ICHI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ho et al., 2014] Joyce Ho, Cheng Lee, and Joydeep Ghosh. Septic shock prediction for patients with missing data. Management Information Systems, 5(1), April 2014.","title":"Septic shock prediction for patients with missing data","authors":"Joyce Ho, Cheng Lee, Joydeep Ghosh","year":"2014","venue":"Management Information Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jia et al., 2017] Xiaowei Jia, Ankush Khandelwal, Gu- ruprasad Nayak, James Gerber, Kimberly Carlson, Paul West, and Vipin Kumar. Incremental dual-memory lstm in land cover prediction. In SIGKDD. ACM, 2017.","title":"Incremental dual-memory lstm in land cover prediction","authors":"Xiaowei Jia, Ankush Khandelwal, Guruprasad Nayak, James Gerber, Kimberly Carlson, Paul West, Vipin Kumar","year":"2017","venue":"SIGKDD. ACM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jia et al., 2019] Xiaowei Jia, Sheng Li, Ankush Khandel- wal, Guruprasad Nayak, Anuj Karpatne, and Vipin Ku- mar. Spatial context-aware networks for mining temporal discriminative period in land cover detection. In Proceed- ings of the 2019 SIAM International Conference on Data Mining, pages 513-521. SIAM, 2019.","title":"Spatial context-aware networks for mining temporal discriminative period in land cover detection","authors":"Xiaowei Jia, Sheng Li, Ankush Khandelwal, Guruprasad Nayak, Anuj Karpatne, Vipin Kumar","year":"2019","venue":"Proceedings of the 2019 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kumar et al., 2006] Anand Kumar, Daniel Roberts, et al. Duration of hypotension before initiation of effective an- timicrobial therapy is the critical determinant of survival in human septic shock. Critical care medicine, 2006.","title":"Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock","authors":"Anand Kumar, Daniel Roberts","year":"2006","venue":"Critical care medicine","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2015] Hui Li, Xiaoyi Li, Xiaowei Jia, Murali Ra- manathan, and Aidong Zhang. Bone disease prediction and phenotype discovery using feature representation over electronic health records. In ACM-BCB. ACM, 2015.","title":"Bone disease prediction and phenotype discovery using feature representation over electronic health records","authors":"Hui Li, Xiaoyi Li, Xiaowei Jia, Murali Ramanathan, Aidong Zhang","year":"2015","venue":"ACM-BCB","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2018] Chen Lin, Yuan Zhang, Min Chi, et al. Early diagnosis and prediction of sepsis shock by combin- ing static and dynamic information using convolutional- lstm. In ICHI, pages 219-228. IEEE, 2018.","title":"Early diagnosis and prediction of sepsis shock by combining static and dynamic information using convolutionallstm","authors":"Chen Lin, Yuan Zhang, Min Chi","year":"2018","venue":"ICHI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lipton et al., 2015] Zachary C Lipton, David C Kale, et al. Learning to diagnose with lstm recurrent neural networks. arXiv preprint arXiv:1511.03677, 2015.","title":"Learning to diagnose with lstm recurrent neural networks","authors":"David C Zachary C Lipton, Kale","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2015] Chuanren Liu, Fei Wang, Jianying Hu, and Hui Xiong. Temporal phenotyping from longitudinal elec- tronic health records: A graph based framework. In SIGKDD, pages 705-714. ACM, 2015.","title":"Temporal phenotyping from longitudinal electronic health records: A graph based framework","authors":"Chuanren Liu, Fei Wang, Jianying Hu, Hui Xiong","year":"2015","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ma et al., 2017] Fenglong Ma, Radha Chitta, Jing Zhou, Quanzeng You, Tong Sun, and Jing Gao. Dipole: Diag- nosis prediction in healthcare via attention-based bidirec- tional recurrent neural networks. In SIGKDD, pages 1903- 1911. ACM, 2017.","title":"Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks","authors":"Ma","year":"2017","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ma et al., 2018] Tengfei Ma, Cao Xiao, and Fei Wang. Health-atm: A deep architecture for multifaceted patient health record representation and risk prediction. In SDM, pages 261-269. SIAM, 2018.","title":"Health-atm: A deep architecture for multifaceted patient health record representation and risk prediction","authors":"Ma","year":"2018","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Marlin et al., 2012] Benjamin M Marlin, David C Kale, Robinder G Khemani, et al. Unsupervised pattern discov- ery in electronic health care data using probabilistic clus- tering models. In IHI, pages 389-398. ACM, 2012.","title":"Unsupervised pattern discovery in electronic health care data using probabilistic clustering models","authors":"Benjamin M Marlin, David C Kale, G Robinder, Khemani","year":"2012","venue":"IHI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nachimuthu and Haug, 2012] Senthil K Nachimuthu and Peter J Haug. Early detection of sepsis in the emergency department using dynamic bayesian networks. In AMIA, 2012.","title":"Early detection of sepsis in the emergency department using dynamic bayesian networks","authors":"K Senthil, Peter J Nachimuthu, Haug","year":"2012","venue":"AMIA","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pham et al., 2016] Trang Pham, Truyen Tran, Dinh Phung, et al. Deepcare: A deep dynamic memory model for pre- dictive medicine. In PAKDD. Springer, 2016.","title":"Deepcare: A deep dynamic memory model for predictive medicine","authors":"Trang Pham, Truyen Tran, Dinh Phung","year":"2016","venue":"PAKDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sha and Wang, 2017] Ying Sha and May D Wang. Inter- pretable predictions of clinical outcomes with an attention- based recurrent neural network. In BCB, 2017.","title":"Interpretable predictions of clinical outcomes with an attentionbased recurrent neural network","authors":"Ying Sha, May D Wang","year":"2017","venue":"BCB","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Singer et al., 2016] Mervyn Singer, Clifford S Deutschman, et al. The third international consensus definitions for sep- sis and septic shock (sepsis-3). Jama, 315(8):801-810, 2016.","title":"The third international consensus definitions for sepsis and septic shock (sepsis-3)","authors":"Mervyn Singer, Clifford S Deutschman","year":"2016","venue":"Jama","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sundermeyer et al., 2012] Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. Lstm neural networks for language modeling. In ISCA, 2012.","title":"Lstm neural networks for language modeling","authors":"Martin Sundermeyer, Ralf Schlüter, Hermann Ney","year":"2012","venue":"ISCA","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2014] Xiang Wang, David Sontag, and Fei Wang. Unsupervised learning of disease progression mod- els. In SIGKDD, pages 85-94. ACM, 2014.","title":"Unsupervised learning of disease progression models","authors":"Xiang Wang, David Sontag, Fei Wang","year":"2014","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2018] Xi Yang, Yuan Zhang, and Min Chi. Time-aware subgroup matrix decomposition: Imputing missing data using forecasting events. In 2018 IEEE Inter- national Conference on Big Data (Big Data). IEEE, 2018.","title":"Time-aware subgroup matrix decomposition: Imputing missing data using forecasting events","authors":"Xi Yang, Yuan Zhang, Min Chi","year":"2018","venue":"IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2017] Yuan Zhang, Chen Lin, Min Chi, et al. Lstm for septic shock: Adding unreliable labels to reliable predictions. In Big Data, pages 1233-1242. IEEE, 2017.","title":"Lstm for septic shock: Adding unreliable labels to reliable predictions","authors":"Yuan Zhang, Chen Lin, Min Chi","year":"2017","venue":"Big Data","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou et al., 2013] Jiayu Zhou, Jimeng Sun, Yashu Liu, Jianying Hu, and Jieping Ye. Patient risk prediction model via top-k stability selection. In SDM, pages 55-63. SIAM, 2013.","title":"Patient risk prediction model via top-k stability selection","authors":"Jiayu Zhou, Jimeng Sun, Yashu Liu, Jianying Hu, Jieping Ye","year":"2013","venue":"SDM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}