{"type":"ScholarlyArticle","title":"Medical Concept Embedding with Multiple Ontological Representations","authors":[{"name":"Lihong Song","slug":"lihong-song","given":"Lihong","family":"Song","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong SAR , China"},{"name":"Chin Wang Cheong","slug":"chin-wang-cheong","given":"Chin Wang","family":"Cheong","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong SAR , China"},{"name":"Kejing Yin","slug":"kejing-yin","given":"Kejing","family":"Yin","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong SAR , China"},{"name":"William K. Cheung","slug":"william-k-cheung","given":"William K.","family":"Cheung","affiliation":"Department of Computer Science , Hong Kong Baptist University , Hong Kong SAR , China"},{"name":"Benjamin C. M. Fung","slug":"benjamin-c-m-fung","given":"Benjamin C. M.","family":"Fung","affiliation":"School of Information Studies , McGill University , Montreal , Canada"},{"name":"Jonathan Poon","slug":"jonathan-poon","given":"Jonathan","family":"Poon","affiliation":"Hong Kong Hospital Authority , Hong Kong SAR , China"}],"abstract":"Learning representations of medical concepts from the Electronic Health Records (EHR) has been shown effective for predictive analytics in healthcare. Incorporation of medical ontologies has also been explored to further enhance the accuracy and to ensure better alignment with the known medical knowledge. Most of the existing work assumes that medical concepts under the same ontological category should share similar representations, which however does not always hold. In particular, the categorizations in medical ontologies were established with various factors being considered. Medical concepts even under the same ontological category may not follow similar occurrence patterns in the EHR data, leading to contradicting objectives for the representation learning. In this paper, we propose a deep learning model called MMORE which alleviates this conflicting objective issue by allowing multiple representations to be inferred for each ontological category via an attention mechanism. We apply MMORE to diagnosis prediction and our experimental results show that the representations obtained by MMORE can achieve better predictive accuracy and result in clinically meaningful sub-categorization of the existing ontological categories.","doi":"10.24963/ijcai.2019/641","url":"https://ijcai.org/proceedings/2019/641","pdf":"https://ijcai.org/proceedings/2019/0641.pdf","markdown":"https://ijcai.org/proceedings/2019/641.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/641","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning Applications","track":"Main track","number":641,"pages":{"first":4613,"last":4619},"keywords":["Machine Learning Applications: Bio;Medicine","Machine Learning Applications: Applications of Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceed- ings of the 3rd International Conference on Learning Representations, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"Proceedings of the 3rd International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Choi et al., 2016a] Edward Choi, Mohammad Taha Ba- hadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James Bost, Javier Tejedor-Sojo, and Jimeng Sun. Multi-layer representation learning for medical con- cepts. In Proceedings of the 22nd ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, pages 1495-1504. ACM, 2016.","title":"Multi-layer representation learning for medical concepts","authors":"Choi","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Choi et al., 2016b] Edward Choi, Mohammad Taha Ba- hadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. RETAIN: An interpretable predictive model for healthcare using reverse time attention mecha- nism. In Advances in Neural Information Processing Sys- tems, pages 3504-3512, 2016.","title":"RETAIN: An interpretable predictive model for healthcare using reverse time attention mechanism","authors":"Choi","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Choi et al., 2016c] Youngduck Choi, Chill Yi-I Chiu, and David Sontag. Learning low-dimensional representations of medical concepts. AMIA Summits on Translational Sci- ence Proceedings, 2016:41, 2016.","title":"Learning low-dimensional representations of medical concepts","authors":"Choi","year":"2016","venue":"AMIA Summits on Translational Science Proceedings","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choi et al., 2017] Edward Choi, Mohammad Taha Ba- hadori, Le Song, Walter F Stewart, and Jimeng Sun. GRAM: Graph-based attention model for healthcare rep- resentation learning. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining, pages 787-795. ACM, 2017.","title":"GRAM: Graph-based attention model for healthcare representation learning","authors":"Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F Stewart, Jimeng Sun","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi et al., 2018] Edward Choi, Cao Xiao, Walter Stewart, and Jimeng Sun. MiME: Multilevel medical embedding of electronic health records for predictive healthcare. In Ad- vances in Neural Information Processing Systems, pages 4552-4562, 2018.","title":"MiME: Multilevel medical embedding of electronic health records for predictive healthcare","authors":"Edward Choi, Cao Xiao, Walter Stewart, Jimeng Sun","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2012] Eric H. Huang, Richard Socher, Christopher D. Manning, and Andrew Y. Ng. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 873-882. Association for Computational Linguistics, 2012.","title":"Improving word representations via global context and multiple word prototypes","authors":"Eric H Huang, Richard Socher, Christopher D Manning, Andrew Y Ng","year":"2012","venue":"Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Johnson et al., 2016] Alistair E. W. Johnson, Tom J. Pol- lard, Lu Shen, Li-wei H. Lehman, Mengling Feng, Mo- hammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. MIMIC-III, a freely accessible critical care database. Scientific Data, 3:160035, 2016.","title":"MIMIC-III, a freely accessible critical care database","authors":"E W Alistair, Tom J Johnson, Lu Pollard, Shen, H Li-Wei, Mengling Lehman, Mohammad Feng, Benjamin Ghassemi, Peter Moody, Leo Szolovits, Roger G Anthony Celi, Mark","year":"2016","venue":"Scientific Data","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim et al., 2017] Yejin Kim, Robert El-Kareh, Jimeng Sun, Hwanjo Yu, and Xiaoqian Jiang. Discriminative and dis- tinct phenotyping by constrained tensor factorization. Sci- entific Reports, 7(1):1114, 2017.","title":"Discriminative and distinct phenotyping by constrained tensor factorization","authors":"Yejin Kim, Robert El-Kareh, Jimeng Sun, Hwanjo Yu, Xiaoqian Jiang","year":"2017","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lee and Seung, 2001] Daniel D. Lee and H. Sebastian Se- ung. Algorithms for non-negative matrix factorization. In Advances in Neural Information Processing Systems, pages 556-562, 2001.","title":"Algorithms for non-negative matrix factorization","authors":"Daniel D Lee, H Sebastian Seung","year":"2001","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ma et al., 2017] Fenglong Ma, Radha Chitta, Jing Zhou, Quanzeng You, Tong Sun, and Jing Gao. Dipole: Diag- nosis prediction in healthcare via attention-based bidirec- tional recurrent neural networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining, pages 1903-1911. ACM, 2017.","title":"Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks","authors":"Ma","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ma et al., 2018] Fenglong Ma, Quanzeng You, Houping Xiao, Radha Chitta, Jing Zhou, and Jing Gao. KAME: Knowledge-based attention model for diagnosis prediction in healthcare. In Proceedings of the 27th ACM Interna- tional Conference on Information and Knowledge Man- agement, pages 743-752. ACM, 2018.","title":"KAME: Knowledge-based attention model for diagnosis prediction in healthcare","authors":"Ma","year":"2018","venue":"Proceedings of the 27th ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(Nov):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mikolov et al., 2013a] Tomas Mikolov, G.s Corrado, Kai Chen, and Jeffrey Dean. Efficient estimation of word rep- resentations in vector space. In Proceedings of the Inter- national Conference on Learning Representations, pages 1-12, 01 2013.","title":"Efficient estimation of word representations in vector space","authors":"Mikolov","year":"2013","venue":"Proceedings of the International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mikolov et al., 2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositional- ity. In Advances in Neural Information Processing Sys- tems, pages 3111-3119, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nguyen et al., 2016] Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, and Svetha Venkatesh. Deepr: a convo- lutional net for medical records. IEEE Journal of Biomed- ical and Health Informatics, 2016.","title":"Deepr: a convolutional net for medical records","authors":"Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, Svetha Venkatesh","year":"2016","venue":"IEEE Journal of Biomedical and Health Informatics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sha and Wang, 2017] Ying Sha and May D. Wang. Inter- pretable predictions of clinical outcomes with an attention- based recurrent neural network. In Proceedings of the 8th ACM International Conference on Bioinformatics, Com- putational Biology, and Health Informatics, pages 233- 240. ACM, 2017.","title":"Interpretable predictions of clinical outcomes with an attentionbased recurrent neural network","authors":"Ying Sha, May D Wang","year":"2017","venue":"Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shickel et al., 2018] Benjamin Shickel, Patrick James Tighe, Azra Bihorac, and Parisa Rashidi. Deep EHR: A survey of recent advances in deep learning techniques for electronic health record (EHR) analysis. IEEE Journal of Biomedical and Health Informatics, 22(5):1589-1604, 2018.","title":"Deep EHR: A survey of recent advances in deep learning techniques for electronic health record (EHR) analysis","authors":"Benjamin Shickel, Patrick James Tighe, Azra Bihorac, Parisa Rashidi","year":"2018","venue":"IEEE Journal of Biomedical and Health Informatics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Yin et al., 2018] Kejing Yin, William K. Cheung, Yang Liu, Benjamin CM Fung, and Jonathan Poon. Joint learning of phenotypes and diagnosis-medication correspondence via hidden interaction tensor factorization. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pages 3627-3633, 2018.","title":"Joint learning of phenotypes and diagnosis-medication correspondence via hidden interaction tensor factorization","authors":"Kejing Yin, William K Cheung, Yang Liu, Benjamin Cm Fung, Jonathan Poon","year":"2018","venue":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18","doi":null,"url":null,"internal_url":"/proceedings/2018/504"},{"position":20,"raw":"[Zeiler, 2012] Matthew D. Zeiler. ADADELTA: An adaptive learning rate method. ArXiv Preprint ArXiv:1212.5701, abs/1212.5701, 2012.","title":"ADADELTA: An adaptive learning rate method","authors":"D Matthew, Zeiler","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}