{"type":"ScholarlyArticle","title":"On Whom Should I Perform this Lab Test Next? An Active Feature Elicitation Approach","authors":[{"name":"Sriraam Natarajan","slug":"sriraam-natarajan","given":"Sriraam","family":"Natarajan","affiliation":"University of Texas at Dallas"},{"name":"Srijita Das","slug":"srijita-das","given":"Srijita","family":"Das","affiliation":"Indiana University Bloomington"},{"name":"Nandini Ramanan","slug":"nandini-ramanan","given":"Nandini","family":"Ramanan","affiliation":"Indiana University Bloomington"},{"name":"Gautam Kunapuli","slug":"gautam-kunapuli","given":"Gautam","family":"Kunapuli","affiliation":"University of Texas at Dallas"},{"name":"Predrag Radivojac","slug":"predrag-radivojac","given":"Predrag","family":"Radivojac","affiliation":"Indiana University Bloomington"}],"abstract":"We consider the problem of actively feature elicitation in which given a few examples with all the features (say the full EHR) and a few examples with some of the features (say demographics), the goal is to identify the set of examples on whom more information  (say the lab tests) needs to be collected. The observation is that some set of features may be more expensive, personal or cumbersome to collect. We propose an active learning approach which identifies examples that are dissimilar to the ones with the full set of data and acquire the complete set of features for these examples. Motivated by real clinical tasks, our extensive evaluation on three clinical tasks demonstrate the effectiveness of this approach.","doi":"10.24963/ijcai.2018/486","url":"https://ijcai.org/proceedings/2018/486","pdf":"https://ijcai.org/proceedings/2018/0486.pdf","markdown":"https://ijcai.org/proceedings/2018/486.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/486","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning Applications","track":"Main track","number":486,"pages":{"first":3498,"last":3505},"keywords":["Machine Learning Applications: Applications of Supervised Learning","Machine Learning Applications: Bio;Medicine","Machine Learning: Active Learning"],"videos":[],"references":[{"position":1,"raw":"[Bilgic and Getoor, 2007] M. Bilgic and L. Getoor. VOILA: efficient feature-value acquisition for classification. NCAI, pages 1225-1230, 2007.","title":"VOILA: efficient feature-value acquisition for classification","authors":"M Bilgic, L Getoor","year":"2007","venue":"NCAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boutilier et al., 2009] C. Boutilier, K. Regan, and P. Viappi- ani. Online feature elicitation in interactive optimization. ICML, pages 73-80, 2009.","title":"Online feature elicitation in interactive optimization","authors":"C Boutilier, K Regan, P Viappiani","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cichocki and Amari, 2010] A. Cichocki and S. Amari. Families of alpha-beta-and gamma-divergences: flexible and robust measures of similarities. Entropy, 12(6):1532- 1568, 2010.","title":"Families of alpha-beta-and gamma-divergences: flexible and robust measures of similarities","authors":"A Cichocki, S Amari","year":"2010","venue":"Entropy","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cieslak et al., 2012] D. A. Cieslak, T. R. Hoens, et al. Hellinger distance decision trees are robust and skew- insensitive. Data Min Knowl Discov, 24(1):136-158, 2012.","title":"Hellinger distance decision trees are robust and skewinsensitive","authors":"D A Cieslak, T R Hoens","year":"2012","venue":"Data Min Knowl Discov","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Csiszár, 1967] I. Csiszár. Information measures of differ- ence of probability distributions and indirect observations. Studia Sci Math Hungar, 2:299-318, 1967.","title":"Information measures of difference of probability distributions and indirect observations","authors":"I Csiszár","year":"1967","venue":"Studia Sci Math Hungar","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Davis and Goadrich, 2006] J. Davis and M. Goadrich. The relationship between precision-recall and ROC curves. ICML, pages 233-240, 2006.","title":"The relationship between precision-recall and ROC curves","authors":"J Davis, M Goadrich","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Deza and Deza, 2013] M. M. Deza and E. Deza. Encyclo- pedia of distances. Springer, 2013.","title":"Encyclopedia of distances","authors":"M M Deza, E Deza","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Druck et al., 2009] G. Druck, B. Settles, and A. McCallum. Active learning by labeling features. EMNLP, pages 81- 90, 2009.","title":"Active learning by labeling features","authors":"G Druck, B Settles, A Mccallum","year":"2009","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Friedman, 2001] J. H. Friedman. Greedy function ap- proximation: a gradient boosting machine. Ann Stat, 29(5):1189-1232, 2001.","title":"Greedy function approximation: a gradient boosting machine","authors":"J H Friedman","year":"2001","venue":"Ann Stat","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hofmann and Buhmann, 1998] T. Hofmann and J. M. Buh- mann. Active data clustering. NIPS, pages 528-534, 1998.","title":"Active data clustering","authors":"T Hofmann, J M Buhmann","year":"1998","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Judah et al., 2014] K. Judah, A. Fern, et al. Imitation learn- ing with demonstrations and shaping rewards. AAAI, pages 1890-1896, 2014.","title":"Imitation learning with demonstrations and shaping rewards","authors":"K Judah, A Fern","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kanani and Melville, 2008] P. Kanani and P. Melville. Prediction-time active feature-value acquisition for cost- effective customer targeting. Workshop on Cost Sensitive Learning at NIPS, 2008.","title":"Prediction-time active feature-value acquisition for costeffective customer targeting","authors":"P Kanani, P Melville","year":"2008","venue":"Workshop on Cost Sensitive Learning at NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kedem et al., 2012] D. Kedem, S. Tyree, et al. Non-linear metric learning. NIPS, pages 2573-2581. 2012.","title":"Non-linear metric learning","authors":"D Kedem, S Tyree","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krishnapuram et al., 2005] B. Krishnapuram, D. Williams, et al. Active learning of features and labels. Workshop on Learning with Multiple Views at ICML, 2005.","title":"Active learning of features and labels","authors":"B Krishnapuram, D Williams","year":"2005","venue":"Workshop on Learning with Multiple Views at ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kunapuli and Shavlik, 2012] G. Kunapuli and J. Shavlik. Mirror descent for metric learning: a unified approach. ECML PKDD, pages 859-874, 2012.","title":"Mirror descent for metric learning: a unified approach","authors":"G Kunapuli, J Shavlik","year":"2012","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lewis and Catlett, 1994] D. D. Lewis and J. Catlett. Het- erogeneous uncertainty sampling for supervised learning. ICML, pages 148-156, 1994.","title":"Heterogeneous uncertainty sampling for supervised learning","authors":"D D Lewis, J Catlett","year":"1994","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lewis and Gale, 1994] D. D. Lewis and W. A. Gale. A sequential algorithm for training text classifiers. SIGIR, pages 3-12, 1994.","title":"A sequential algorithm for training text classifiers","authors":"D D Lewis, W A Gale","year":"1994","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lizotte et al., 2003] D. J. Lizotte, O. Madani, and R. Greiner. Budgeted learning of naive-bayes classi- fiers. UAI, pages 378-385, 2003.","title":"Budgeted learning of naive-bayes classifiers","authors":"D J Lizotte, O Madani, R Greiner","year":"2003","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lopes et al., 2009] M. Lopes, F. Melo, and L. Montesano. Active learning for reward estimation in inverse reinforce- ment learning. ECML PKDD, pages 31-46, 2009.","title":"Active learning for reward estimation in inverse reinforcement learning","authors":"M Lopes, F Melo, L Montesano","year":"2009","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[MacLeod et al., 2016] H. MacLeod, S. Yang, et al. Identify- ing rare diseases from behavioural data: a machine learn- ing approach. CHASE, pages 130-139, 2016.","title":"Identifying rare diseases from behavioural data: a machine learning approach","authors":"H Macleod, S Yang","year":"2016","venue":"CHASE","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Marek et al., 2011] K. Marek, D. Jennings, et al. The Parkinson Progression Marker Initiative (PPMI). Prog Neurobiol, 95(4):629-635, 2011.","title":"The Parkinson Progression Marker Initiative (PPMI)","authors":"K Marek, D Jennings","year":"2011","venue":"Prog Neurobiol","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Melville et al., 2004] P. Melville, M. Saar-Tsechansky, et al. Active feature-value acquisition for classifier induc- tion. ICDM, pages 483-486, 2004.","title":"Active feature-value acquisition for classifier induction","authors":"P Melville, M Saar-Tsechansky","year":"2004","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Melville et al., 2005] P. Melville, M. Saar-Tsechansky, et al. An expected utility approach to active feature-value acquisition. ICDM, pages 745-748, 2005.","title":"An expected utility approach to active feature-value acquisition","authors":"P Melville, M Saar-Tsechansky","year":"2005","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Natarajan et al., 2017] S. Natarajan, A. Prabhakar, et al. Boosting for postpartum depression prediction. CHASE, pages 232-240, 2017.","title":"Boosting for postpartum depression prediction","authors":"S Natarajan, A Prabhakar","year":"2017","venue":"CHASE","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Odom and Natarajan, 2016] P. Odom and S. Natarajan. Ac- tive advice seeking for inverse reinforcement learning. AAMAS, pages 512-520, 2016.","title":"Active advice seeking for inverse reinforcement learning","authors":"P Odom, S Natarajan","year":"2016","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Raghavan et al., 2006] H. Raghavan, O. Madani, and R. Jones. Active learning with feedback on features and instances. J Mach Learn Res, 7(Aug):1655-1686, 2006.","title":"Active learning with feedback on features and instances","authors":"H Raghavan, O Madani, R Jones","year":"2006","venue":"J Mach Learn Res","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Saar-Tsechansky et al., 2009] M. Saar-Tsechansky, P. Melville, and F. Provost. Active feature-value acquisition. Manag Sci, 55(4), 2009.","title":"Active feature-value acquisition","authors":"M Saar-Tsechansky, P Melville, F Provost","year":"2009","venue":"Manag Sci","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Settles, 2012] B. Settles. Active learning. Synthesis Lec- tures on Artificial Intelligence and Machine Learning, 6(1), 2012.","title":"Active learning","authors":"B Settles","year":"2012","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Smith et al., 1988] J. W. Smith, J. E. Everhart, et al. Using the ADAP learning algorithm to forecast the onset of dia- betes mellitus. Proc Annu Symp Comput Appl Med Care, pages 261-265, 1988.","title":"Using the ADAP learning algorithm to forecast the onset of diabetes mellitus","authors":"J W Smith, J E Everhart","year":"1988","venue":"Proc Annu Symp Comput Appl Med Care","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Thahir et al., 2012] M. Thahir, T. Sharma, and M. K. Gana- pathiraju. An efficient heuristic method for active feature acquisition and its application to protein-protein interac- tion prediction. BMC Proc, 6(Suppl 7):S2, 2012.","title":"An efficient heuristic method for active feature acquisition and its application to protein-protein interaction prediction","authors":"M Thahir, T Sharma, M K Ganapathiraju","year":"2012","venue":"BMC Proc","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Tong and Koller, 2000] S. Tong and D. Koller. Active learn- ing for parameter estimation in Bayesian networks. NIPS, pages 647-653, 2000.","title":"Active learning for parameter estimation in Bayesian networks","authors":"S Tong, D Koller","year":"2000","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tong and Koller, 2001a] S. Tong and D. Koller. Active learning for structure in Bayesian networks. IJCAI, pages 863-869, 2001.","title":"Active learning for structure in Bayesian networks","authors":"Koller Tong, ] S Tong, D Koller","year":"2001","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2001/118"},{"position":33,"raw":"[Tong and Koller, 2001b] S. Tong and D. Koller. Support vector machine active learning with applications to text classification. J Mach Learn Res, 2(Nov):45-66, 2001.","title":"Support vector machine active learning with applications to text classification","authors":"Tong, ] S Koller ; B, D Tong, Koller","year":"2001","venue":"J Mach Learn Res","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zheng and Padmanabhan, 2002] Z. Zheng and B. Padman- abhan. On active learning for data acquisition. ICDM, pages 562-569, 2002.","title":"On active learning for data acquisition","authors":"Padmanabhan Zheng, ] Z Zheng, B Padmanabhan","year":"2002","venue":"ICDM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}