{"type":"ScholarlyArticle","title":"Learning with Previously Unseen Features","authors":[{"name":"Yuan Shi","slug":"yuan-shi","given":"Yuan","family":"Shi","affiliation":"Computer Science Department University of Southern California"},{"name":"Craig A. Knoblock","slug":"craig-a-knoblock","given":"Craig A.","family":"Knoblock","affiliation":"Information Sciences Institute University of Southern California"}],"abstract":"We study the problem of improving a machine learning model by identifying and using features that are not in the training set. This is applicable to machine learning systems deployed in an open environment. For example, a prediction model built on a set of sensors may be improved when it has access to new and relevant sensors at test time. To effectively use new features, we propose a novel approach that learns a model over both the original and new features, with the goal of making the joint distribution of features and predicted labels similar to that in the training set. Our approach can naturally leverage labels associated with these new features when they are accessible. We present an efficient optimization algorithm for learning the model parameters and empirically evaluate the approach on several regression and classification tasks. Experimental results show that our approach can achieve on average 11.2% improvement over baselines.","doi":"10.24963/ijcai.2017/379","url":"https://ijcai.org/proceedings/2017/379","pdf":"https://ijcai.org/proceedings/2017/0379.pdf","markdown":"https://ijcai.org/proceedings/2017/379.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/379","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":379,"pages":{"first":2722,"last":2729},"keywords":["Machine Learning: Semi-Supervised Learning","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[],"references":[{"position":1,"raw":"[Argyriou et al., 2008] Andreas Argyriou, Andreas Maurer, and Massimiliano Pontil. An algorithm for transfer learn- ing in a heterogeneous environment. In Joint European Conference on Machine Learning and Knowledge Discov- ery in Databases, pages 71-85. Springer, 2008.","title":"An algorithm for transfer learning in a heterogeneous environment","authors":"Andreas Argyriou, Andreas Maurer, Massimiliano Pontil","year":"2008","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Batista et al., 2002] Gustavo EAPA Batista, Maria Carolina Monard, et al. A study of k-nearest neighbour as an impu- tation method. HIS, 87(251-260):48, 2002.","title":"A study of k-nearest neighbour as an imputation method","authors":"Gustavo Eapa Batista, Maria Carolina Monard","year":"2002","venue":"HIS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bishop, 2006] Christopher M Bishop. Pattern recognition. Machine Learning, 128:1-58, 2006.","title":"Pattern recognition","authors":"M Christopher, Bishop","year":"2006","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blitzer et al., 2006] John Blitzer, Ryan McDonald, and Fer- nando Pereira. Domain adaptation with structural corre- spondence learning. In Proceedings of the 2006 confer- ence on empirical methods in natural language process- ing, pages 120-128. Association for Computational Lin- guistics, 2006.","title":"Domain adaptation with structural correspondence learning","authors":"John Blitzer, Ryan Mcdonald, Fernando Pereira","year":"2006","venue":"Proceedings of the 2006 conference on empirical methods in natural language processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dai et al., 2008] Wenyuan Dai, Yuqiang Chen, Gui-Rong Xue, Qiang Yang, and Yong Yu. Translated learning: Transfer learning across different feature spaces. In Ad- vances in neural information processing systems, pages 353-360, 2008.","title":"Translated learning: Transfer learning across different feature spaces","authors":"Wenyuan Dai, Yuqiang Chen, Gui-Rong Xue, Qiang Yang, Yong Yu","year":"2008","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Daume III and Marcu, 2006] Hal Daume III and Daniel Marcu. Domain adaptation for statistical classifiers. Jour- nal of Artificial Intelligence Research, 26:101-126, 2006.","title":"Hal Daume III and Daniel Marcu. Domain adaptation for statistical classifiers","authors":"Iii Daume, Marcu","year":"2006","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dereszynski and Dietterich, 2011] Ethan W Dereszynski and Thomas G Dietterich. Spatiotemporal models for data-anomaly detection in dynamic environmental moni- toring campaigns. ACM Transactions on Sensor Networks (TOSN), 8(1):3, 2011.","title":"Spatiotemporal models for data-anomaly detection in dynamic environmental monitoring campaigns","authors":"Dietterich Dereszynski, Ethan W Dereszynski, Thomas G Dietterich","year":"2011","venue":"ACM Transactions on Sensor Networks (TOSN)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duan et al., 2012] Lixin Duan, Dong Xu, and Ivor W Tsang. Learning with augmented features for heteroge- neous domain adaptation. In Proceedings of the 29th In- ternational Conference on Machine Learning (ICML-12), pages 711-718, 2012.","title":"Learning with augmented features for heterogeneous domain adaptation","authors":"Lixin Duan, Dong Xu, Ivor W Tsang","year":"2012","venue":"Proceedings of the 29th International Conference on Machine Learning (ICML-12)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Farhadi et al., 2009] Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing objects by their attributes. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 1778-1785. IEEE, 2009.","title":"Describing objects by their attributes","authors":"Ali Farhadi, Ian Endres, Derek Hoiem, David Forsyth","year":"2009","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grzymala-Busse and Hu, 2000] Jerzy W Grzymala-Busse and Ming Hu. A comparison of several approaches to missing attribute values in data mining. In International Conference on Rough Sets and Current Trends in Comput- ing, pages 378-385. Springer, 2000.","title":"A comparison of several approaches to missing attribute values in data mining","authors":"Grzymala-Busse Hu, Jerzy W Grzymala-Busse, Ming Hu","year":"2000","venue":"International Conference on Rough Sets and Current Trends in Computing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Harel and Mannor, 2010] Maayan Harel and Shie Mannor. Learning from multiple outlooks. 2010.","title":"Learning from multiple outlooks","authors":"Maayan Harel, Shie Mannor","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hou and Zhou, 2016] Chenping Hou and Zhi-Hua Zhou. One-pass learning with incremental and decremental fea- tures. arXiv preprint arXiv:1605.09082, 2016.","title":"One-pass learning with incremental and decremental features","authors":"Zhou Hou, Chenping Hou, Zhi-Hua Zhou","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hull, 1994] Jonathan J. Hull. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-554, 1994.","title":"A database for handwritten text recognition research","authors":"Jonathan J Hull","year":"1994","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kulis et al., 2011] Brian Kulis, Kate Saenko, and Trevor Darrell. What you saw is not what you get: Domain adap- tation using asymmetric kernel transforms. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Con- ference on, pages 1785-1792. IEEE, 2011.","title":"What you saw is not what you get: Domain adaptation using asymmetric kernel transforms","authors":"Brian Kulis, Kate Saenko, Trevor Darrell","year":"2011","venue":"Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lakshminarayan et al., 1996] Kamakshi Lakshminarayan, Steven A Harp, Robert P Goldman, Tariq Samad, et al. Imputation of missing data using machine learning tech- niques. In KDD, pages 140-145, 1996.","title":"Imputation of missing data using machine learning techniques","authors":"Kamakshi Lakshminarayan, Steven A Harp, Robert P Goldman, Tariq Samad","year":"1996","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Larochelle et al., 2008] Hugo Larochelle, Dumitru Erhan, and Yoshua Bengio. Zero-data learning of new tasks. In AAAI, volume 1, page 3, 2008.","title":"Zero-data learning of new tasks","authors":"Larochelle","year":"2008","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pan and Yang, 2010] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowl- edge and data engineering, 22(10):1345-1359, 2010.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2010","venue":"IEEE Transactions on knowledge and data engineering","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Quionero-Candela et al., 2009] Joaquin Quionero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning. The MIT Press, 2009.","title":"Dataset shift in machine learning","authors":"Quionero-Candela","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shi et al., 2010] Xiaoxiao Shi, Qi Liu, Wei Fan, S Yu Philip, and Ruixin Zhu. Transfer learning on heterogenous fea- ture spaces via spectral transformation. In 2010 IEEE in- ternational conference on data mining, pages 1049-1054. IEEE, 2010.","title":"Transfer learning on heterogenous feature spaces via spectral transformation","authors":"Xiaoxiao Shi, Qi Liu, Wei Fan, Yu Philip, Ruixin Zhu","year":"2010","venue":"IEEE international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Socher et al., 2013] Richard Socher, Milind Ganjoo, Christopher D Manning, and Andrew Ng. Zero-shot learning through cross-modal transfer. In Advances in neural information processing systems, pages 935-943, 2013.","title":"Zero-shot learning through cross-modal transfer","authors":"Richard Socher, Milind Ganjoo, Christopher D Manning, Andrew Ng","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vapnik and Vashist, 2009] Vladimir Vapnik and Akshay Vashist. A new learning paradigm: Learning using priv- ileged information. Neural Networks, 22(5):544-557, 2009.","title":"Vladimir Vapnik and Akshay Vashist. A new learning paradigm: Learning using privileged information","authors":"Vashist Vapnik","year":"2009","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang and Mahadevan, 2011] Chang Wang and Sridhar Ma- hadevan. Heterogeneous domain adaptation using mani- fold alignment. In IJCAI Proceedings-International Joint Conference on Artificial Intelligence, volume 22, page 1541, 2011.","title":"Heterogeneous domain adaptation using manifold alignment","authors":"Chang Wang, Sridhar Mahadevan","year":"2011","venue":"IJCAI Proceedings-International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2011/259"},{"position":23,"raw":"[Wei and Pal, 2011] Bin Wei and Christopher J Pal. Hetero- geneous transfer learning with rbms. In AAAI, 2011.","title":"Heterogeneous transfer learning with rbms","authors":"Pal ; ] Bin Wei, Christopher J Wei, Pal","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yeh et al., 2014] Yi-Ren Yeh, Chun-Hao Huang, and Yu- Chiang Frank Wang. Heterogeneous domain adaptation and classification by exploiting the correlation subspace. IEEE Transactions on Image Processing, 23(5):2009- 2018, 2014.","title":"Heterogeneous domain adaptation and classification by exploiting the correlation subspace","authors":"Yi-Ren Yeh, Chun-Hao Huang, Yu-Chiang Frank, Wang","year":"2014","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao and Hoi, 2010] Peilin Zhao and Steven C Hoi. Otl: A framework of online transfer learning. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), pages 1231-1238, 2010.","title":"Otl: A framework of online transfer learning","authors":"Peilin Zhao, Steven C Hoi","year":"2010","venue":"Proceedings of the 27th International Conference on Machine Learning (ICML-10)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2014] Joey Tianyi Zhou, Sinno Jialin Pan, Ivor W Tsang, and Yan Yan. Hybrid heterogeneous trans- fer learning through deep learning. In AAAI, pages 2213- 2220, 2014.","title":"Hybrid heterogeneous transfer learning through deep learning","authors":"Joey Tianyi Zhou, Sinno Jialin Pan, Ivor W Tsang, Yan Yan","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhu, 2005] Xiaojin Zhu. Semi-supervised learning litera- ture survey. 2005.","title":"Semi-supervised learning literature survey","authors":"Xiaojin Zhu","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}