{"type":"ScholarlyArticle","title":"Generalized Zero-Shot Text Classification for ICD Coding","authors":[{"name":"Congzheng Song","slug":"congzheng-song","given":"Congzheng","family":"Song","affiliation":"Cornell University"},{"name":"Shanghang Zhang","slug":"shanghang-zhang","given":"Shanghang","family":"Zhang","affiliation":"University of California , Berkeley"},{"name":"Najmeh Sadoughi","slug":"najmeh-sadoughi","given":"Najmeh","family":"Sadoughi","affiliation":"Petuum Inc."},{"name":"Pengtao Xie","slug":"pengtao-xie","given":"Pengtao","family":"Xie","affiliation":"Petuum Inc."},{"name":"Eric Xing","slug":"eric-xing","given":"Eric","family":"Xing","affiliation":"Petuum Inc."}],"abstract":"The International Classification of Diseases (ICD) is a list of classification codes for the diagnoses. Automatic ICD coding is a multi-label text classification problem with noisy clinical document inputs and long-tailed label distribution, making it difficult for fine-grained classification on both frequent and zero-shot codes at the same time, i.e. generalized zero-shot ICD coding. In this paper, we propose a latent feature generation framework to improve the prediction on unseen codes without compromising the performance on seen codes. Our framework generates semantically meaningful features for zero-shot codes by exploiting ICD code hierarchical structure and reconstructing the code-relevant keywords with a novel cycle architecture. To the best of our knowledge, this is the first adversarial generative model for generalized zero-shot learning on multi-label text classification. Extensive experiments demonstrate the effectiveness of our approach. On the public MIMIC-III dataset, our methods improve the F1 score from nearly 0 to 20.91% for the zero-shot codes, and increase the AUC score by 3% (absolute improvement) from previous state of the art. Code is available at https://github.com/csong27/gzsl_text.","doi":"10.24963/ijcai.2020/556","url":"https://ijcai.org/proceedings/2020/556","pdf":"https://ijcai.org/proceedings/2020/0556.pdf","markdown":"https://ijcai.org/proceedings/2020/556.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/556","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Natural Language Processing","track":"Main track","number":556,"pages":{"first":4018,"last":4024},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Natural Language Processing: NLP Applications and Tools","Machine Learning Applications: Bio/Medicine","Natural Language Processing: Text Classification"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25083/file_25083.m3u8","length_sec":298,"provider":"hls"}],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.","title":"Wasserstein gan","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2019] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distribution-aware margin loss. In NeurIPS, 2019.","title":"Learning imbalanced datasets with label-distribution-aware margin loss","authors":"Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, Tengyu Ma","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chao et al., 2016] Wei-Lun Chao, Soravit Changpinyo, Bo- qing Gong, and Fei Sha. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In ECCV, 2016.","title":"An empirical study and analysis of generalized zero-shot learning for object recognition in the wild","authors":"Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, Fei Sha","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chung et al., 2014] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evalua- tion of gated recurrent neural networks on sequence mod- eling. arXiv preprint arXiv:1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Devlin et al., 2018] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805, 2018.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Felix et al., 2018] Rafael Felix, Vijay BG Kumar, Ian Reid, and Gustavo Carneiro. Multi-modal cycle-consistent gen- eralized zero-shot learning. In ECCV, 2018.","title":"Multi-modal cycle-consistent generalized zero-shot learning","authors":"Rafael Felix, B G Vijay, Ian Kumar, Gustavo Reid, Carneiro","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NeurIPS, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gulrajani et al., 2017] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In NeurIPS, 2017.","title":"Improved training of wasserstein gans","authors":"Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Aaron C Vincent Dumoulin, Courville","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[ICD-9 Data, 2006] The web's free icd-9-cm medical cod- ing reference. http://www.icd9data.com, 2006. Accessed: 2019-06-01.","title":"The web's free icd-9-cm medical coding reference","authors":null,"year":"2006","venue":null,"doi":null,"url":"http://www.icd9data.com","internal_url":null},{"position":11,"raw":"[ICD-9 Guidelines, 2011] International classification of dis- eases,ninth revision, clinical modification (ICD-9-CM). https://www.cdc.gov/nchs/icd/icd9cm.htm, 2011. Ac- cessed: 2019-06-01.","title":"International classification of diseases,ninth revision, clinical modification (ICD-9-CM)","authors":null,"year":"2011","venue":null,"doi":null,"url":"https://www.cdc.gov/nchs/icd/icd9cm.htm,2011.Ac-cessed:2019-06-01","internal_url":null},{"position":12,"raw":"[Johnson et al., 2016] Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo An- thony Celi, and Roger G Mark. Mimic-iii, a freely ac- cessible critical care database. Scientific data, 2016.","title":"Mimic-iii, a freely accessible critical care database","authors":"E W Alistair, Tom J Johnson, Lu Pollard, H Shen, Mengling Lehman Li-Wei, Mohammad Feng, Benjamin Ghassemi, Peter Moody, Leo Szolovits, Roger G Anthony Celi, Mark","year":"2016","venue":"Scientific data","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2015] Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493, 2015.","title":"Gated graph sequence neural networks","authors":"Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard Zemel","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2019] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large-scale long-tailed recognition in an open world. In CVPR, 2019.","title":"Large-scale long-tailed recognition in an open world","authors":"Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, Stella X Yu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mullenbach et al., 2018] James Mullenbach, Sarah Wiegr- effe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. Ex- plainable prediction of medical codes from clinical text. In NAACL, 2018.","title":"Explainable prediction of medical codes from clinical text","authors":"James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, Jacob Eisenstein","year":"2018","venue":"NAACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ni et al., 2019] Jian Ni, Shanghang Zhang, and Haiyong Xie. Dual adversarial semantics-consistent network for generalized zero-shot learning. In NeurIPS, 2019.","title":"Dual adversarial semantics-consistent network for generalized zero-shot learning","authors":"Jian Ni, Shanghang Zhang, Haiyong Xie","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pushp and Srivastava, 2017] Pushpankar Kumar Pushp and Muktabh Mayank Srivastava. Train once, test anywhere: Zero-shot learning for text classification. arXiv preprint arXiv:1712.05972, 2017.","title":"Pushpankar Kumar Pushp and Muktabh Mayank Srivastava. Train once, test anywhere: Zero-shot learning for text classification","authors":"Srivastava Pushp","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rios and Kavuluru, 2018] Anthony Rios and Ramakanth Kavuluru. Few-shot and zero-shot multi-label learning for structured label spaces. In EMNLP, 2018.","title":"Few-shot and zero-shot multi-label learning for structured label spaces","authors":"Anthony Rios, Ramakanth Kavuluru","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi et al., 2017] Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, and Eric P Xing. Towards automated icd cod- ing using deep learning. arXiv preprint arXiv:1711.04075, 2017.","title":"Towards automated icd coding using deep learning","authors":"Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, Eric P Xing","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Stanfill et al., 2010] Mary H Stanfill, Margaret Williams, Susan H Fenton, Robert A Jenders, and William R Hersh. A systematic literature review of automated clinical cod- ing and classification systems. JAMIA, 2010.","title":"A systematic literature review of automated clinical coding and classification systems","authors":"Mary H Stanfill, Margaret Williams, Susan H Fenton, Robert A Jenders, William R Hersh","year":"2010","venue":"JAMIA","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xian et al., 2018] Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. In CVPR, 2018.","title":"Feature generating networks for zero-shot learning","authors":"Xian","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xie and Xing, 2018] Pengtao Xie and Eric Xing. A neural architecture for automated icd coding. In ACL, 2018.","title":"Pengtao Xie and Eric Xing. A neural architecture for automated icd coding","authors":"Xing Xie","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2019a] Jingqing Zhang, Piyawat Lertvit- tayakumjorn, and Yike Guo. Integrating semantic knowledge to tackle zero-shot text classification. In ACL, 2019.","title":"Integrating semantic knowledge to tackle zero-shot text classification","authors":"Zhang","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2019b] Yijia Zhang, Qingyu Chen, Zhihao Yang, Hongfei Lin, and Zhiyong Lu. Biowordvec, im- proving biomedical word embeddings with subword infor- mation and mesh. Scientific data, 2019.","title":"Biowordvec, improving biomedical word embeddings with subword information and mesh","authors":"Zhang","year":"2019","venue":"Scientific data","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}