{"type":"ScholarlyArticle","title":"Hierarchical Apprenticeship Learning for Disease Progression Modeling","authors":[{"name":"Xi Yang","slug":"xi-yang","given":"Xi","family":"Yang","affiliation":"IBM Research"},{"name":"Ge Gao","slug":"ge-gao","given":"Ge","family":"Gao","affiliation":""},{"name":"Min Chi","slug":"min-chi","given":"Min","family":"Chi","affiliation":"North Carolina State University"}],"abstract":"Disease progression modeling (DPM) plays an essential role in characterizing patients' historical pathways and predicting their future risks. Apprenticeship learning (AL) aims to induce decision-making policies by observing and imitating expert behaviors. In this paper, we investigate the incorporation of AL-derived patterns into DPM, utilizing a Time-aware Hierarchical EM Energy-based Subsequence (THEMES) AL approach. To the best of our knowledge, this is the first study incorporating AL-derived progressive and interventional patterns for DPM. We evaluate the efficacy of this approach in a challenging task of septic shock early prediction, and our results demonstrate that integrating the AL-derived patterns significantly enhances the performance of DPM.","doi":"10.24963/ijcai.2023/265","url":"https://ijcai.org/proceedings/2023/265","pdf":"https://ijcai.org/proceedings/2023/0265.pdf","markdown":"https://ijcai.org/proceedings/2023/265.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/265","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Data Mining","track":"Main Track","number":265,"pages":{"first":2388,"last":2396},"keywords":["Multidisciplinary Topics and Applications: MDA: Health and medicine","Data Mining: DM: Applications","Data Mining: DM: Mining spatial and/or temporal data"],"videos":[],"references":[{"position":1,"raw":"[Abbeel and Ng, 2004] Pieter Abbeel and Andrew Y Ng. Apprenticeship learning via inverse reinforcement learn- ing. In Proceedings of the 21st international conference on Machine learning, page 1. ACM, 2004.","title":"Apprenticeship learning via inverse reinforcement learning","authors":"Abbeel, ; Ng, Pieter Abbeel, Andrew Y Ng","year":"2004","venue":"Proceedings of the 21st international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Azizsoltani et al., 2019] Hamoon Azizsoltani, Yeojin Kim, Markel Sanz Ausin, Tiffany Barnes, and Min Chi. Unob- served is not equal to non-existent: Using gaussian pro- cesses to infer immediate rewards across contexts. In In Proceedings of the 28th IJCAI, 2019.","title":"Unobserved is not equal to non-existent: Using gaussian processes to infer immediate rewards across contexts","authors":"Hamoon Azizsoltani, Yeojin Kim, Markel Sanz Ausin, Tiffany Barnes, Min Chi","year":"2019","venue":"Proceedings of the 28th IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/273"},{"position":3,"raw":"[Babes et al., 2011] Monica Babes, Vukosi Marivate, Kaushik Subramanian, and Michael L Littman. Appren- ticeship learning about multiple intentions. In Proceedings of the 28th ICML, pages 897-904, 2011.","title":"Apprenticeship learning about multiple intentions","authors":"Monica Babes, Kaushik Vukosi Marivate, Subramanian, Michael L Littman","year":"2011","venue":"Proceedings of the 28th ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Baytas et al., 2017] Inci M Baytas, Cao Xiao, Xi Zhang, Fei Wang, Anil K Jain, and Jiayu Zhou. Paient subtyping via time-aware lstm networks. In SIGKDD. ACM, 2017.","title":"Paient subtyping via time-aware lstm networks","authors":"Inci M Baytas, Cao Xiao, Xi Zhang, Fei Wang, Anil K Jain, Jiayu Zhou","year":"2017","venue":"SIGKDD. ACM","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Boyd et al., 2011] Stephen Boyd, Neal Parikh, and Eric Chu. Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Pub- lishers Inc, 2011.","title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","authors":"Stephen Boyd, Neal Parikh, Eric Chu","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016.","title":"Openai gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chan and van der Schaar, 2021] Alex J Chan and Mihaela van der Schaar. Scalable bayesian inverse reinforcement learning. arXiv preprint arXiv:2102.06483, 2021.","title":"Scalable bayesian inverse reinforcement learning","authors":"Chan, Van Der Schaar ; Alex, J Chan, Mihaela Van Der Schaar","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Choi et al., 2016] Edward Choi, Mohammad Taha Ba- hadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In NIPS, pages 3504-3512, 2016.","title":"Retain: An interpretable predictive model for healthcare using reverse time attention mechanism","authors":"Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, Walter Stewart","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Clifford et al., 2016] Kalin M Clifford, Eliza A Dy- Boarman, Krystal K Haase, Kristen Maxvill, Steven E Pass, and Carlos A Alvarez. Challenges with diagnosing and managing sepsis in older adults. Expert review of anti- infective therapy, 14(2):231-241, 2016.","title":"Challenges with diagnosing and managing sepsis in older adults","authors":"M Kalin, Eliza A Clifford, Krystal K Dy-Boarman, Kristen Haase, Steven E Maxvill, Carlos A Pass, Alvarez","year":"2016","venue":"Expert review of antiinfective therapy","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Cook and Bies, 2016] Sarah F Cook and Robert R Bies. Disease progression modeling: key concepts and recent developments. Current pharmacology reports, 2016.","title":"Disease progression modeling: key concepts and recent developments","authors":"F Sarah, Robert R Cook, Bies","year":"2016","venue":"Current pharmacology reports","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dimitrakakis and Rothkopf, 2011] Christos Dimitrakakis and Constantin A Rothkopf. Bayesian multitask inverse reinforcement learning. In European workshop on reinforcement learning, pages 273-284. Springer, 2011.","title":"Bayesian multitask inverse reinforcement learning","authors":"Christos Dimitrakakis, Constantin A Rothkopf","year":"2011","venue":"European workshop on reinforcement learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dulac-Arnold et al., 2020] Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. An empirical investigation of the challenges of real-world reinforcement learning. arXiv preprint arXiv:2003.11881, 2020.","title":"An empirical investigation of the challenges of real-world reinforcement learning","authors":"Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, Todd Hester","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Esteban et al., 2016] Cristóbal Esteban, Oliver Staeck, Stephan Baier, Yinchong Yang, and Volker Tresp. Predict- ing clinical events by combining static and dynamic in- formation using recurrent neural networks. In 2016 IEEE ICHI, pages 93-101. Ieee, 2016.","title":"Predicting clinical events by combining static and dynamic information using recurrent neural networks","authors":"Esteban","year":"2016","venue":"IEEE ICHI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Faruqui et al., 2021] Syed Hasib Akhter Faruqui, Adel Alaeddini, Jing Wang, Carlos A Jaramillo, and Mary Jo Pugh. A functional model for structure learning and pa- rameter estimation in continuous time bayesian network: An application in identifying patterns of multiple chronic conditions. IEEE Access, 2021.","title":"A functional model for structure learning and parameter estimation in continuous time bayesian network: An application in identifying patterns of multiple chronic conditions","authors":"Syed Hasib, Akhter Faruqui, Adel Alaeddini, Jing Wang, Carlos A Jaramillo, Mary Jo Pugh","year":"2021","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Finn et al., 2016] Chelsea Finn, Sergey Levine, and Pieter Abbeel. Guided cost learning: Deep inverse optimal con- trol via policy optimization. In International conference on machine learning. PMLR, 2016.","title":"Guided cost learning: Deep inverse optimal control via policy optimization","authors":"Finn","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Friedman et al., 2001] Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics New York, 2001.","title":"The elements of statistical learning","authors":"Jerome Friedman, Trevor Hastie, Robert Tibshirani","year":"2001","venue":"Springer series in statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Friedman et al., 2008] Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Sparse inverse covariance estima- tion with the graphical lasso. Biostatistics, 2008.","title":"Sparse inverse covariance estimation with the graphical lasso","authors":"Jerome Friedman, Trevor Hastie, Robert Tibshirani","year":"2008","venue":"Biostatistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gao et al., 2022] Ge Gao, Qitong Gao, Xi Yang, Miroslav Pajic, and Min Chi. A reinforcement learning-informed pattern mining framework for multivariate time series clas- sification. In IJCAI, 2022.","title":"A reinforcement learning-informed pattern mining framework for multivariate time series classification","authors":"Ge Gao, Qitong Gao, Xi Yang, Miroslav Pajic, Min Chi","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/415"},{"position":19,"raw":"[Gauer, 2013] Robert Gauer. Early recognition and manage- ment of sepsis in adults: the first six hours. American fam- ily physician, 88(1):44-53, 2013.","title":"Early recognition and management of sepsis in adults: the first six hours","authors":"Robert Gauer, Gauer","year":"2013","venue":"American family physician","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Giannoula et al., 2018] Alexia Giannoula, Alba Gutierrez- Sacristán, Álex Bravo, Ferran Sanz, and Laura I Furlong. Identifying temporal patterns in patient disease trajectories using dynamic time warping: a population-based study. Scientific reports, 8(1):1-14, 2018.","title":"Identifying temporal patterns in patient disease trajectories using dynamic time warping: a population-based study","authors":"Alexia Giannoula, Alba Gutierrez-Sacristán, Álex Bravo, Ferran Sanz, Laura I Furlong","year":"2018","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Goh et al., 2021] Kim Huat Goh, Le Wang, Adrian Yong Kwang Yeow, et al. Artificial intelligence in sep- sis early prediction and diagnosis using unstructured data in healthcare. Nature communications, 12(1):1-10, 2021.","title":"Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare","authors":"Kim Huat Goh, Le Wang, Adrian Yong Kwang, Yeow","year":"2021","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Grathwohl et al., 2019] Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud, Moham- mad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03263, 2019.","title":"Your classifier is secretly an energy based model and you should treat it like one","authors":"Grathwohl","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Hallac et al., 2017] David Hallac, Sagar Vare, Stephen Boyd, and Jure Leskovec. Toeplitz inverse covariance- based clustering of multivariate time series data. In SIGKDD, pages 215-223, 2017.","title":"Toeplitz inverse covariancebased clustering of multivariate time series data","authors":"David Hallac, Sagar Vare, Stephen Boyd, Jure Leskovec","year":"2017","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Hausman et al., 2017] Karol Hausman, Yevgen Chebotar, Stefan Schaal, Gaurav Sukhatme, and Joseph J Lim. Multi-modal imitation learning from unstructured demon- strations using generative adversarial nets. Advances in neural information processing systems, 30, 2017.","title":"Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets","authors":"Karol Hausman, Yevgen Chebotar, Stefan Schaal, Gaurav Sukhatme, Joseph J Lim","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ho and Ermon, 2016] Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016.","title":"Generative adversarial imitation learning","authors":"Jonathan Ho, Stefano Ermon","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Jarrett et al., 2020] Daniel Jarrett, Ioana Bica, and Mihaela van der Schaar. Strictly batch imitation learning by energy- based distribution matching. Advances in Neural Informa- tion Processing Systems, 2020.","title":"Strictly batch imitation learning by energybased distribution matching","authors":"Daniel Jarrett, Ioana Bica, Mihaela Van Der Schaar","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Komorowski et al., 2018] Matthieu Komorowski, Leo A Celi, et al. The artificial intelligence clinician learns opti- mal treatment strategies for sepsis in intensive care. Nature medicine, 24(11):1716-1720, 2018.","title":"The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care","authors":"Matthieu Komorowski, Leo A Celi","year":"2018","venue":"Nature medicine","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Kostrikov et al., 2018] Ilya Kostrikov, Kumar Krishna Agrawal, Debidatta Dwibedi, Sergey Levine, and Jonathan Tompson. Discriminator-actor-critic: Address- ing sample inefficiency and reward bias in adversarial imitation learning. ICLR, 2018.","title":"Discriminator-actor-critic: Addressing sample inefficiency and reward bias in adversarial imitation learning","authors":"Ilya Kostrikov, Krishna Kumar, Debidatta Agrawal, Sergey Dwibedi, Jonathan Levine, Tompson","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Krishnan et al., 2016] Sanjay Krishnan, Animesh Garg, Richard Liaw, Lauren Miller, Florian T Pokorny, and Ken Goldberg. Hirl: Hierarchical inverse reinforcement learn- ing for long-horizon tasks with delayed rewards. arXiv preprint arXiv:1604.06508, 2016.","title":"Hirl: Hierarchical inverse reinforcement learning for long-horizon tasks with delayed rewards","authors":"Sanjay Krishnan, Animesh Garg, Richard Liaw, Lauren Miller, Florian T Pokorny, Ken Goldberg","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Kumar et al., 2006] Anand Kumar, Daniel Roberts, et al. Duration of hypotension before initiation of effective an- timicrobial therapy is the critical determinant of survival in human septic shock. Critical care medicine, 2006.","title":"Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock","authors":"Anand Kumar, Daniel Roberts","year":"2006","venue":"Critical care medicine","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Kwon et al., 2020] Bum Chul Kwon, Peter Achenbach, Jes- sica L Dunne, et al. Modeling disease progression trajecto- ries from longitudinal observational data. In AMIA Annual Symposium Proceedings, 2020.","title":"Modeling disease progression trajectories from longitudinal observational data","authors":"Kwon","year":"2020","venue":"AMIA Annual Symposium Proceedings","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Levine et al., 2020] Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tu- torial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020.","title":"Offline reinforcement learning: Tutorial, review, and perspectives on open problems","authors":"Sergey Levine, Aviral Kumar, George Tucker, Justin Fu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Li et al., 2018] Jia Li, Yu Rong, Helen Meng, et al. Tatc: Predicting alzheimer's disease with actigraphy data. In Proceedings of the 24th ACM SIGKDD, 2018.","title":"Tatc: Predicting alzheimer's disease with actigraphy data","authors":"Jia Li, Yu Rong, Helen Meng","year":"2018","venue":"Proceedings of the 24th ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Lipton et al., 2015] Zachary C Lipton, David C Kale, Charles Elkan, and Randall Wetzel. Learning to diag- nose with lstm recurrent neural networks. arXiv preprint arXiv:1511.03677, 2015.","title":"Learning to diagnose with lstm recurrent neural networks","authors":"David C Zachary C Lipton, Charles Kale, Randall Elkan, Wetzel","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Martin et al., 2003] Greg S Martin, David M Mannino, Stephanie Eaton, and Marc Moss. The epidemiology of sepsis in the united states from 1979 through 2000. New England Journal of Medicine, 348(16):1546-1554, 2003.","title":"The epidemiology of sepsis in the united states from 1979 through 2000","authors":"David M Greg S Martin, Stephanie Mannino, Marc Eaton, Moss","year":"2003","venue":"New England Journal of Medicine","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ng et al., 2000] Andrew Y Ng, Stuart Russell, et al. Algo- rithms for inverse reinforcement learning. In ICML, 2000.","title":"Algorithms for inverse reinforcement learning","authors":"Y Andrew, Stuart Ng, Russell","year":"2000","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Raza et al., 2012] Saleha Raza, Sajjad Haider, and Mary- Anne Williams. Teaching coordinated strategies to soccer robots via imitation. In 2012 IEEE ROBIO, 2012.","title":"Teaching coordinated strategies to soccer robots via imitation","authors":"Saleha Raza, Sajjad Haider, Mary-Anne Williams","year":"2012","venue":"IEEE ROBIO","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Ross et al., 2011] Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and struc- tured prediction to no-regret online learning. In Proceed- ings of the 14th international conference on artificial in- telligence and statistics, pages 627-635, 2011.","title":"A reduction of imitation learning and structured prediction to no-regret online learning","authors":"Stéphane Ross, Geoffrey Gordon, Drew Bagnell","year":"2011","venue":"Proceedings of the 14th international conference on artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Saqib et al., 2018] Mohammed Saqib, Ying Sha, and May D Wang. Early prediction of sepsis in emr records using traditional ml techniques and deep learning lstm net- works. In 40th IEEE EMBC, pages 4038-4041, 2018.","title":"Early prediction of sepsis in emr records using traditional ml techniques and deep learning lstm networks","authors":"Mohammed Saqib, Ying Sha, May D Wang","year":"2018","venue":"40th IEEE EMBC","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Severson et al., 2020] Kristen A Severson, Lana M Chahine, Luba Smolensky, Kenney Ng, Jianying Hu, and Soumya Ghosh. Personalized input-output hidden markov models for disease progression modeling. In Machine Learning for Healthcare Conference. PMLR, 2020.","title":"Personalized input-output hidden markov models for disease progression modeling","authors":"Kristen A Severson, Lana M Chahine, Luba Smolensky, Kenney Ng, Jianying Hu, Soumya Ghosh","year":"2020","venue":"Machine Learning for Healthcare Conference","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Singer et al., 2016] Mervyn Singer, Clifford S Deutschman, Christopher Warren Seymour, et al. The third international consensus definitions for sepsis and septic shock (sepsis- 3). Jama, 315(8):801-810, 2016.","title":"The third international consensus definitions for sepsis and septic shock (sepsis-3)","authors":"Mervyn Singer, Christopher Warren Clifford S Deutschman, Seymour","year":"2016","venue":"Jama","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Tintinalli et al., 1985] Judith E Tintinalli, Gabor D Kelen, J Stephan Stapczynski, et al. Emergency medicine: a com- prehensive study guide. Mcgraw-hill New York, 1985.","title":"Emergency medicine: a comprehensive study guide","authors":"Judith E Tintinalli, Stephan Gabor D Kelen, Stapczynski","year":"1985","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Viterbi, 1967] Andrew Viterbi. Error bounds for convolu- tional codes and an asymptotically optimum decoding al- gorithm. IEEE transactions on Information Theory, 1967.","title":"Error bounds for convolutional codes and an asymptotically optimum decoding algorithm","authors":"Andrew Viterbi","year":"1967","venue":"IEEE transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2018] Lu Wang, Wei Zhang, Xiaofeng He, and Hongyuan Zha. Supervised reinforcement learning with recurrent neural network for dynamic treatment recom- mendation. In Proceedings of 24th SIGKDD, 2018.","title":"Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation","authors":"Lu Wang, Wei Zhang, Xiaofeng He, Hongyuan Zha","year":"2018","venue":"Proceedings of 24th SIGKDD","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wang et al., 2021] Lu Wang, Wenchao Yu, Wei Cheng, Bo Zong, and Haifeng Chen. Hierarchical imitation learn- ing with contextual bandits for dynamic treatment regimes. In RL4RealLife Workshop in the 38 th ICML, 2021.","title":"Hierarchical imitation learning with contextual bandits for dynamic treatment regimes","authors":"Lu Wang, Wenchao Yu, Wei Cheng, Bo Zong, Haifeng Chen","year":"2021","venue":"RL4RealLife Workshop in the 38 th ICML","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Wang et al., 2022] Lu Wang, Ruiming Tang, Xiaofeng He, and Xiuqiang He. Hierarchical imitation learning via sub- goal representation learning for dynamic treatment recom- mendation. In Proceedings 15th ACM ICWSDM, 2022.","title":"Hierarchical imitation learning via subgoal representation learning for dynamic treatment recommendation","authors":"Lu Wang, Ruiming Tang, Xiaofeng He, Xiuqiang He","year":"2022","venue":"Proceedings 15th ACM ICWSDM","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Welling and Teh, 2011] Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynam- ics. In Proceedings of the 28th ICML, 2011.","title":"Bayesian learning via stochastic gradient langevin dynamics","authors":"Teh ; ] Max Welling, Yee W Welling, Teh","year":"2011","venue":"Proceedings of the 28th ICML","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yang et al., 2020] Xi Yang, Guojing Zhou, Michelle Taub, Roger Azevedo, and Min Chi. Student subtyping via em- inverse reinforcement learning. International Educational Data Mining Society, 2020.","title":"Student subtyping via eminverse reinforcement learning","authors":"Xi Yang, Guojing Zhou, Michelle Taub, Roger Azevedo, Min Chi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Yang et al., 2021] Xi Yang, Yuan Zhang, and Min Chi. Multi-series time-aware sequence partitioning for disease progression modeling. In IJCAI, 2021.","title":"Multi-series time-aware sequence partitioning for disease progression modeling","authors":"Xi Yang, Yuan Zhang, Min Chi","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/493"},{"position":50,"raw":"[Yang et al., 2023] Xi Yang, Ge Gao, and Min Chi. An offline time-aware apprenticeship learning frame- work for evolving reward functions. arXiv preprint arXiv:2305.09070, 2023.","title":"An offline time-aware apprenticeship learning framework for evolving reward functions","authors":"Xi Yang, Ge Gao, Min Chi","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Yu et al., 2021] Chao Yu, Jiming Liu, Shamim Nemati, and Guosheng Yin. Reinforcement learning in healthcare: A survey. ACM Computing Surveys (CSUR), 2021.","title":"Reinforcement learning in healthcare: A survey","authors":"Chao Yu, Jiming Liu, Shamim Nemati, Guosheng Yin","year":"2021","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhang et al., 2017] Yuan Zhang, Chen Lin, Min Chi, Julie Ivy, Muge Capan, and Jeanne M Huddleston. Lstm for septic shock: Adding unreliable labels to reliable predic- tions. In 2017 IEEE Big Data, pages 1233-1242, 2017.","title":"Lstm for septic shock: Adding unreliable labels to reliable predictions","authors":"Yuan Zhang, Chen Lin, Min Chi, Julie Ivy, Muge Capan, Jeanne M Huddleston","year":"2017","venue":"IEEE Big Data","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhang et al., 2019] Yuan Zhang, Xi Yang, Ivy Julie, and Min Chi. Attain: Attention-based time-aware lstm net- works for disease progression modeling. In In Proceedings of the 28th IJCAI, 2019.","title":"Attain: Attention-based time-aware lstm networks for disease progression modeling","authors":"Yuan Zhang, Xi Yang, Ivy Julie, Min Chi","year":"2019","venue":"Proceedings of the 28th IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/607"},{"position":54,"raw":"[Ziebart et al., 2008] Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, and Anind K Dey. Maximum entropy inverse reinforcement learning. In AAAI, volume 8, pages 1433-1438. Chicago, IL, USA, 2008.","title":"Maximum entropy inverse reinforcement learning","authors":"Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}