{"type":"ScholarlyArticle","title":"Metric Learning on Healthcare Data with Incomplete Modalities","authors":[{"name":"Qiuling Suo","slug":"qiuling-suo","given":"Qiuling","family":"Suo","affiliation":"Department of Computer Science and Engineering , SUNY at Buffalo , NY , USA"},{"name":"Weida Zhong","slug":"weida-zhong","given":"Weida","family":"Zhong","affiliation":"Department of Computer Science and Engineering , SUNY at Buffalo , NY , USA"},{"name":"Fenglong Ma","slug":"fenglong-ma","given":"Fenglong","family":"Ma","affiliation":"Department of Computer Science and Engineering , SUNY at Buffalo , NY , USA"},{"name":"Ye Yuan","slug":"ye-yuan","given":"Ye","family":"Yuan","affiliation":"College of Information and Communication Engineering , Beijing University of Technology , China"},{"name":"Jing Gao","slug":"jing-gao","given":"Jing","family":"Gao","affiliation":"Department of Computer Science and Engineering , SUNY at Buffalo , NY , USA"},{"name":"Aidong Zhang","slug":"aidong-zhang","given":"Aidong","family":"Zhang","affiliation":"Department of Computer Science , University of Virginia , VA , USA"}],"abstract":"Utilizing multiple modalities to learn a good distance metric is of vital importance for various clinical applications. However, it is common that modalities are incomplete for some patients due to various technical and practical reasons in healthcare datasets. Existing metric learning methods cannot directly learn the distance metric on such data with missing modalities. Nevertheless, the incomplete data contains valuable information to characterize patient similarity and modality relationships, and they should not be ignored during the learning process. To tackle the aforementioned challenges, we propose a metric learning framework to perform missing modality completion and multi-modal metric learning simultaneously. Employing the generative adversarial networks, we incorporate both complete and incomplete data to learn the mapping relationship between modalities. After completing the missing modalities, we use the nonlinear representations extracted by the discriminator to learn the distance metric among patients. Through jointly training the adversarial generation part and metric learning, the similarity among patients can be learned on data with missing modalities. Experimental results show that the proposed framework learns more accurate distance metric on real-world healthcare datasets with incomplete modalities, comparing with the state-of-the-art approaches. Meanwhile, the quality of the generated modalities can be preserved.","doi":"10.24963/ijcai.2019/490","url":"https://ijcai.org/proceedings/2019/490","pdf":"https://ijcai.org/proceedings/2019/0490.pdf","markdown":"https://ijcai.org/proceedings/2019/490.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/490","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":490,"pages":{"first":3534,"last":3540},"keywords":["Machine Learning Applications: Bio;Medicine","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2018] Lei Cai, Zhengyang Wang, Hongyang Gao, Dinggang Shen, and Shuiwang Ji. Deep adversarial learn- ing for multi-modality missing data completion. In Pro- ceedings of SIGKDD, pages 1158-1166, 2018.","title":"Deep adversarial learning for multi-modality missing data completion","authors":"Lei Cai, Zhengyang Wang, Hongyang Gao, Dinggang Shen, Shuiwang Ji","year":"2018","venue":"Proceedings of SIGKDD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Donahue et al., 2016] Jeff Donahue, Philipp Krahenbuhl, and Trevor Darrell. Adversarial feature learning. arXiv preprint arXiv:1605.09782, 2016.","title":"Adversarial feature learning","authors":"Jeff Donahue, Philipp Krahenbuhl, Trevor Darrell","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Du et al., 2018] Changying Du, Changde Du, Xingyu Xie, Chen Zhang, and Hao Wang. Multi-view adversari- ally learned inference for cross-domain joint distribution matching. In Proceedings of SIGKDD, pages 1348-1357, 2018.","title":"Multi-view adversarially learned inference for cross-domain joint distribution matching","authors":"Du","year":"2018","venue":"Proceedings of SIGKDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dumoulin et al., 2016] Vincent Dumoulin, Ishmael Belg- hazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Mar- tin Arjovsky, and Aaron Courville. Adversarially learned inference. arXiv preprint arXiv:1606.00704, 2016.","title":"Adversarially learned inference","authors":"Ishmael Vincent Dumoulin, Ben Belghazi, Olivier Poole, Alex Mastropietro, Martin Lamb, Aaron Arjovsky, Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gan et al., 2017] Zhe Gan, Liqun Chen, Weiyao Wang, Yuchen Pu, Yizhe Zhang, Hao Liu, Chunyuan Li, and Lawrence Carin. Triangle generative adversarial networks. In Proceedings of NeurIPS, pages 5247-5256, 2017.","title":"Triangle generative adversarial networks","authors":"Zhe Gan, Liqun Chen, Weiyao Wang, Yuchen Pu, Yizhe Zhang, Hao Liu, Chunyuan Li, Lawrence Carin","year":"2017","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Proceedings of NeurIPS, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hoffman et al., 2017] Judy Hoffman, Eric Tzeng, Tae- sung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell. Cycada: Cycle- consistent adversarial domain adaptation. arXiv preprint arXiv:1711.03213, 2017.","title":"Cycada: Cycleconsistent adversarial domain adaptation","authors":"Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, Trevor Darrell","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2014] Junlin Hu, Jiwen Lu, Junsong Yuan, and Yap-Peng Tan. Large margin multi-metric learning for face and kinship verification in the wild. In Proceedings of ACCV, pages 252-267, 2014.","title":"Large margin multi-metric learning for face and kinship verification in the wild","authors":"Junlin Hu, Jiwen Lu, Junsong Yuan, Yap-Peng Tan","year":"2014","venue":"Proceedings of ACCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huai et al., 2018] Mengdi Huai, Chenglin Miao, Qiuling Suo, Yaliang Li, Jing Gao, and Aidong Zhang. c. In Pro- ceedings of SDM, pages 270-278, 2018.","title":"c","authors":"Huai","year":"2018","venue":"Proceedings of SDM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Isola et al., 2017] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with con- ditional adversarial networks. pages 1125-1134, 2017.","title":"Image-to-image translation with conditional adversarial networks","authors":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A Efros","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2014] Rongjian Li, Wenlu Zhang, Heung-Il Suk, Li Wang, Jiang Li, Dinggang Shen, and Shuiwang Ji. Deep learning based imaging data completion for im- proved brain disease diagnosis. In Proceedings of MIC- CAI, pages 305-312, 2014.","title":"Deep learning based imaging data completion for improved brain disease diagnosis","authors":"Li","year":"2014","venue":"Proceedings of MIC-CAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2018] Yan Li, Tao Yang, Jiayu Zhou, and Jieping Ye. Multi-task learning based survival analysis for pre- dicting alzheimer's disease progression with multi-source block-wise missing data. In SDM, pages 288-296, 2018.","title":"Multi-task learning based survival analysis for predicting alzheimer's disease progression with multi-source block-wise missing data","authors":"Yan Li, Tao Yang, Jiayu Zhou, Jieping Ye","year":"2018","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ni et al., 2017] Jiazhi Ni, Jie Liu, Chenxin Zhang, Dan Ye, and Zhirou Ma. Fine-grained patient similarity measur- ing using deep metric learning. In Proceedings of CIKM, pages 1189-1198, 2017. [Odena et al., 2017] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxil- iary classifier gans. In Proceedings of ICML, pages 2642- 2651, 2017.","title":"Fine-grained patient similarity measuring using deep metric learning","authors":"Jiazhi Ni, Jie Liu, Chenxin Zhang, Dan Ye, Zhirou Ma, ; Odena","year":"2017","venue":"Proceedings of CIKM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ronneberger et al., 2015] Olaf Ronneberger, Philipp Fis- cher, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of MICCAI, pages 234-241, 2015.","title":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation","authors":"Ronneberger","year":"2015","venue":"Proceedings of MICCAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Russo et al., 2018] Paolo Russo, Fabio M Carlucci, Tatiana Tommasi, and Barbara Caputo. From source to target and back: symmetric bi-directional adaptive gan. In Proceed- ings of CVPR, pages 8099-8108, 2018.","title":"From source to target and back: symmetric bi-directional adaptive gan","authors":"Paolo Russo, Fabio M Carlucci, Tatiana Tommasi, Barbara Caputo","year":"2018","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shang et al., 2017] Chao Shang, Aaron Palmer, Jiangwen Sun, Ko-Shin Chen, Jin Lu, and Jinbo Bi. Vigan: Miss- ing view imputation with generative adversarial networks. In Proceedings of BigData, pages 766-775, 2017.","title":"Vigan: Missing view imputation with generative adversarial networks","authors":"Chao Shang, Aaron Palmer, Jiangwen Sun, Ko-Shin, Jin Chen, Jinbo Lu, Bi","year":"2017","venue":"Proceedings of BigData","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Suo et al., 2018] Qiuling Suo, Weida Zhong, Fenglong Ma, Yuan Ye, Mengdi Huai, and Aidong Zhang. Multi-task sparse metric learning for monitoring patient similarity progression. In Proceedings of ICDM, pages 477-486, 2018.","title":"Multi-task sparse metric learning for monitoring patient similarity progression","authors":"Suo","year":"2018","venue":"Proceedings of ICDM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2018] Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, and Yun Fu. Partial multi- view clustering via consistent gan. In Proceedings of ICDM, pages 1290-1295, 2018.","title":"Partial multiview clustering via consistent gan","authors":"Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, Yun Fu","year":"2018","venue":"Proceedings of ICDM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xie and Xing, 2013] Pengtao Xie and Eric Xing. Multi- modal distance metric learning. In Proceedings of IJCAI, 2013.","title":"Pengtao Xie and Eric Xing. Multimodal distance metric learning","authors":"Xing Xie","year":"2013","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2018] Yang Yang, De-Chuan Zhan, Xiang- Rong Sheng, and Yuan Jiang. Semi-supervised multi- modal learning with incomplete modalities. In Proceed- ings of IJCAI, pages 2998-3004, 2018.","title":"Semi-supervised multimodal learning with incomplete modalities","authors":"Yang Yang, Xiang-Rong De-Chuan Zhan, Yuan Sheng, Jiang","year":"2018","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yuan et al., 2018] Ye Yuan, Xun Guangxu, Fenglong Ma, Yaqing Wang, Nan Du, Kebin Jia, Lu Su, and Aidong Zhang. Muvan: A multi-view attention network for mul- tivariate temporal data. In Proceedings of ICDM, pages 717-726, 2018.","title":"Muvan: A multi-view attention network for multivariate temporal data","authors":"Ye Yuan, Xun Guangxu, Fenglong Ma, Yaqing Wang, Nan Du, Kebin Jia, Lu Su, Aidong Zhang","year":"2018","venue":"Proceedings of ICDM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhan et al., 2016] Mengting Zhan, Shilei Cao, Buyue Qian, Shiyu Chang, and Jishang Wei. Low-rank sparse feature selection for patient similarity learning. In Proceedings of ICDM, pages 1335-1340, 2016.","title":"Low-rank sparse feature selection for patient similarity learning","authors":"Zhan","year":"2016","venue":"Proceedings of ICDM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2017] Heng Zhang, Vishal M Patel, and Rama Chellappa. Hierarchical multimodal metric learning for multimodal classification. In Proceedings of CVPR, pages 3057-3065, 2017.","title":"Hierarchical multimodal metric learning for multimodal classification","authors":"Heng Zhang, M Vishal, Rama Patel, Chellappa","year":"2017","venue":"Proceedings of CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2018] Changqing Zhang, Yeqinq Liu, Yue Liu, Qinghua Hu, Xinwang Liu, and Pengfei Zhu. Fish-mml: Fisher-hsic multi-view metric learning. In Proceedings of IJCAI, pages 3054-3060, 2018.","title":"Fish-mml: Fisher-hsic multi-view metric learning","authors":"Changqing Zhang, Yeqinq Liu, Yue Liu, Qinghua Hu, Xinwang Liu, Pengfei Zhu","year":"2018","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/424"},{"position":25,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. pages 2223- 2232, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Zhu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}