{"type":"ScholarlyArticle","title":"A Label Attention Model for ICD Coding from Clinical Text","authors":[{"name":"Thanh Vu","slug":"thanh-vu","given":"Thanh","family":"Vu","affiliation":"Australian e-Health Research Centre , CSIRO , Brisbane , Australia"},{"name":"Dat Quoc Nguyen","slug":"dat-quoc-nguyen","given":"Dat Quoc","family":"Nguyen","affiliation":"VinAI Research , Hanoi , Vietnam"},{"name":"Anthony Nguyen","slug":"anthony-nguyen","given":"Anthony","family":"Nguyen","affiliation":"Australian e-Health Research Centre , CSIRO , Brisbane , Australia"}],"abstract":"ICD coding is a process of assigning the International Classification of Disease diagnosis codes to clinical/medical notes documented by health professionals (e.g. clinicians). This process requires significant human resources, and thus is costly and prone to error. To handle the problem, machine learning has been utilized for automatic ICD coding. Previous state-of-the-art models were based on convolutional neural networks, using a single/several fixed window sizes. However, the lengths and interdependence between text fragments related to ICD codes in clinical text vary significantly, leading to the difficulty of deciding what the best window sizes are. In this paper, we propose a new label attention model for automatic ICD coding, which can handle both the various lengths and the interdependence of the ICD code related text fragments. Furthermore, as the majority of ICD codes are not frequently used, leading to the extremely imbalanced data issue, we additionally propose a hierarchical joint learning mechanism extending our label attention model to handle the issue, using the hierarchical relationships among the codes. Our label attention model achieves new state-of-the-art results on three benchmark MIMIC datasets, and the joint learning mechanism helps improve the performances for infrequent codes.","doi":"10.24963/ijcai.2020/461","url":"https://ijcai.org/proceedings/2020/461","pdf":"https://ijcai.org/proceedings/2020/0461.pdf","markdown":"https://ijcai.org/proceedings/2020/461.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/461","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning Applications","track":"Main track","number":461,"pages":{"first":3335,"last":3341},"keywords":["Machine Learning: Classification","Machine Learning: Deep Learning","Machine Learning Applications: Bio/Medicine","Natural Language Processing: Text Classification"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25123/file_25123.m3u8","length_sec":284,"provider":"hls"}],"references":[{"position":1,"raw":"[Baumel et al., 2018] Tal Baumel, Jumana Nassour-Kassis, et al. Multi-label classification of patient notes: case study on ICD code assignment. In Proceedings of the AAAI Workshop on Health Intelligence, pages 409-416, 2018.","title":"Multi-label classification of patient notes: case study on ICD code assignment","authors":"Baumel","year":"2018","venue":"Proceedings of the AAAI Workshop on Health Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chinchor, 1992] Nancy Chinchor. The statistical signifi- cance of the MUC-4 results. In Proceedings of the Con- ference on Message Understanding, pages 30-50, 1992.","title":"The statistical significance of the MUC-4 results","authors":"Nancy Chinchor","year":"1992","venue":"Proceedings of the Conference on Message Understanding","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cho et al., 2014] Kyunghyun Cho, Bart van Merriënboer, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of EMNLP, pages 1724-1734, 2014.","title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","authors":"Cho","year":"2014","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[de Lima et al., 1998] Luciano R. S. de Lima, Alberto H. F. Laender, and Berthier A. Ribeiro-Neto. A hierarchical ap- proach to the automatic categorization of medical docu- ments. In Proceedings of CIKM, pages 132-139, 1998.","title":"A hierarchical approach to the automatic categorization of medical documents","authors":"R S Luciano, Alberto H F De Lima, Berthier A Laender, Ribeiro-Neto","year":"1998","venue":"Proceedings of CIKM","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dror et al., 2018] Rotem Dror, Gili Baumer, Segev Shlo- mov, and Roi Reichart. The hitchhiker's guide to testing statistical significance in natural language processing. In Proceedings of ACL, pages 1383-1392, 2018.","title":"The hitchhiker's guide to testing statistical significance in natural language processing","authors":"Rotem Dror, Gili Baumer, Segev Shlomov, Roi Reichart","year":"2018","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Johnson et al., 2016] Alistair EW Johnson, Tom J Pollard, et al. MIMIC-III, a freely accessible critical care database. Scientific data, 3:160035, 2016.","title":"MIMIC-III, a freely accessible critical care database","authors":"E W Alistair, Tom J Johnson, Pollard","year":"2016","venue":"Scientific data","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Karimi et al., 2017] Sarvnaz Karimi, Xiang Dai, Hamed Hassanzadeh, and Anthony Nguyen. Automatic diagnosis coding of radiology reports: a comparison of deep learning and conventional classification methods. In Proceedings of BioNLP, pages 328-332, 2017.","title":"Automatic diagnosis coding of radiology reports: a comparison of deep learning and conventional classification methods","authors":"Karimi","year":"2017","venue":"Proceedings of BioNLP","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kim, 2014] Yoon Kim. Convolutional Neural Networks for Sentence Classification. In Proceedings of EMNLP, pages 1746-1751, 2014.","title":"Convolutional Neural Networks for Sentence Classification","authors":"Yoon Kim","year":"2014","venue":"Proceedings of EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceed- ings of ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of ICLR, 2017.","title":"Semi-Supervised Classification with Graph Convolutional Networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koopman et al., 2015] Bevan Koopman, Guido Zuccon, Anthony Nguyen, et al. Automatic ICD-10 classification of cancers from free-text death certificates. International journal of medical informatics, 84(11):956-965, 2015.","title":"Automatic ICD-10 classification of cancers from free-text death certificates","authors":"Bevan Koopman, Guido Zuccon, Anthony Nguyen","year":"2015","venue":"International journal of medical informatics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Larkey and Croft, 1996] Leah S Larkey and W Bruce Croft. Combining classifiers in text categorization. In Proceed- ings of SIGIR, volume 96, pages 289-297, 1996.","title":"Combining classifiers in text categorization","authors":"S Leah, W Bruce Larkey, Croft","year":"1996","venue":"Proceedings of SIGIR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee et al., 2011] Joon Lee, Daniel J Scott, et al. Open- access MIMIC-II database for intensive care research. In Proceedings of EMBC, pages 8315-8318, 2011.","title":"Openaccess MIMIC-II database for intensive care research","authors":"Joon Lee, Daniel J Scott","year":"2011","venue":"Proceedings of EMBC","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li and Yu, 2020] Fei Li and Hong Yu. ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network. In Proceedings of AAAI, 2020.","title":"ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network","authors":"Yu ; ] Fei Li, Hong Li, Yu","year":"2020","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2017] Zhouhan Lin, Minwei Feng, et al. A Struc- tured Self-Attentive Sentence Embedding. In Proceedings of ICLR, 2017.","title":"A Structured Self-Attentive Sentence Embedding","authors":"Lin","year":"2017","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Loshchilov and Hutter, 2019] Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. In Pro- ceedings of ICLR, 2019.","title":"Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization","authors":"Hutter Loshchilov","year":"2019","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Manning et al., 2008] Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. Introduction to Informa- tion Retrieval. Cambridge University Press, 2008.","title":"Introduction to Information Retrieval","authors":"D Christopher, Prabhakar Manning, Hinrich Raghavan, Schütze","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[McCallum, 1999] Andrew McCallum. Multi-label text clas- sification with a mixture model trained by EM. In The AAAI workshop on Text Learning, pages 1-7, 1999.","title":"Multi-label text classification with a mixture model trained by EM","authors":"Andrew Mccallum","year":"1999","venue":"The AAAI workshop on Text Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, et al. Distributed Representations of Words and Phrases and their Compositionality. In Proceedings of NIPS, pages 3111-3119. 2013.","title":"Distributed Representations of Words and Phrases and their Compositionality","authors":"Mikolov","year":"2013","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mullenbach et al., 2018] James Mullenbach, Sarah Wiegr- effe, et al. Explainable Prediction of Medical Codes from Clinical Text. In Proceedings of NAACL-HLT, pages 1101-1111, 2018.","title":"Explainable Prediction of Medical Codes from Clinical Text","authors":"James Mullenbach, Sarah Wiegreffe","year":"2018","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nguyen et al., 2018] Anthony N Nguyen, Donna Truran, et al. Computer-Assisted Diagnostic Coding: Effective- ness of an NLP-based approach using SNOMED CT to ICD-10 mappings. In Proceedings of AMIA, pages 807- 816, 2018.","title":"Computer-Assisted Diagnostic Coding: Effectiveness of an NLP-based approach using SNOMED CT to ICD-10 mappings","authors":"Donna Anthony N Nguyen, Truran","year":"2018","venue":"Proceedings of AMIA","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[O'malley et al., 2005] Kimberly J O'malley, Karon F Cook, et al. Measuring diagnoses: ICD code accuracy. Health services research, 40(5p2):1620-1639, 2005.","title":"Measuring diagnoses: ICD code accuracy","authors":"J O' Kimberly, Karon F Malley, Cook","year":"2005","venue":"Health services research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, et al. Py- Torch: An Imperative Style, High-Performance Deep Learning Library. In Proceedings of NIPS, pages 8024- 8035. 2019.","title":"Py-Torch: An Imperative Style, High-Performance Deep Learning Library","authors":"Adam Paszke, Sam Gross","year":"2019","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Perotte et al., 2013] Adler Perotte, Rimma Pivovarov, et al. Diagnosis code assignment: models and evaluation met- rics. JAMIA, 21(2):231-237, 2013.","title":"Diagnosis code assignment: models and evaluation metrics","authors":"Rimma Adler Perotte, Pivovarov","year":"2013","venue":"JAMIA","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Prakash et al., 2017] Aaditya Prakash, Siyuan Zhao, et al. Condensed memory networks for clinical diagnostic infer- encing. In Proceedings of AAAI, page 3274-3280, 2017.","title":"Condensed memory networks for clinical diagnostic inferencing","authors":"Aaditya Prakash, Siyuan Zhao","year":"2017","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shi et al., 2017] Haoran Shi, Pengtao Xie, et al. To- wards automated ICD coding using deep learning. arXiv preprint, arXiv:1711.04075, 2017.","title":"Towards automated ICD coding using deep learning","authors":"Haoran Shi, Pengtao Xie","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Song et al., 2019] Congzheng Song, Shanghang Zhang, et al. Generalized Zero-shot ICD Coding. arXiv preprint arXiv:1909.13154, 2019.","title":"Generalized Zero-shot ICD Coding","authors":"Song","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sukhbaatar et al., 2015] Sainbayar Sukhbaatar, arthur szlam, et al. End-To-End Memory Networks. In Proceedings of NIPS, pages 2440-2448. 2015.","title":"Sainbayar Sukhbaatar, arthur szlam, et al. End-To-End Memory Networks","authors":"Sukhbaatar","year":"2015","venue":"Proceedings of NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2018] Guoyin Wang, Chunyuan Li, et al. Joint embedding of words and labels for text classification. In Proceedings of ACL, pages 2321-2331, 2018.","title":"Joint embedding of words and labels for text classification","authors":"Wang","year":"2018","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xie et al., 2019] Xiancheng Xie, Yun Xiong, et al. EHR Coding with Multi-scale Feature Attention and Structured Knowledge Graph Propagation. In Proceedings of CIKM, pages 649-658, 2019.","title":"EHR Coding with Multi-scale Feature Attention and Structured Knowledge Graph Propagation","authors":"Xie","year":"2019","venue":"Proceedings of CIKM","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Yang et al., 2016] Zichao Yang, Diyi Yang, et al. Hierar- chical attention networks for document classification. In Proceedings of NAACL-HLT, pages 1480-1489, 2016.","title":"Hierarchical attention networks for document classification","authors":"Yang","year":"2016","venue":"Proceedings of NAACL-HLT","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}