{"type":"ScholarlyArticle","title":"AccGenSVM: Selectively Transferring from Previous Hypotheses","authors":[{"name":"Diana Benavides-Prado","slug":"diana-benavides-prado","given":"Diana","family":"Benavides-Prado","affiliation":"Dept. of Computer Science The University of Auckland"},{"name":"Yun Sing Koh","slug":"yun-sing-koh","given":"Yun Sing","family":"Koh","affiliation":"Dept. of Computer Science The University of Auckland"},{"name":"Patricia Riddle","slug":"patricia-riddle","given":"Patricia","family":"Riddle","affiliation":"Dept. of Computer Science The University of Auckland"}],"abstract":"In our research, we consider transfer learning scenarios where a target learner does not have access to the source data, but instead to hypotheses or models induced from it. This is called the Hypothesis Transfer Learning (HTL) problem. Previous approaches concentrated on transferring source hypotheses as a whole. We introduce a novel method for selectively transferring elements from previous hypotheses learned with Support Vector Machines. The representation of an SVM hypothesis as a set of support vectors allows us to treat this information as privileged to aid learning during a new task. Given a possibly large number of source hypotheses, our approach selects the source support vectors that more closely resemble the target data, and transfers their learned coefficients as constraints on the coefficients to be learned. This strategy increases the importance of relevant target data points based on their similarity to source support vectors, while learning from the target data. Our method shows important improvements on the convergence rate on three classification datasets of varying sizes, decreasing the number of iterations by up to 56% on average compared to learning with no transfer and up to 92% compared to regular HTL, while maintaining similar accuracy levels.","doi":"10.24963/ijcai.2017/199","url":"https://ijcai.org/proceedings/2017/199","pdf":"https://ijcai.org/proceedings/2017/0199.pdf","markdown":"https://ijcai.org/proceedings/2017/199.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/199","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":199,"pages":{"first":1440,"last":1446},"keywords":["Machine Learning: Classification","Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Aytar and Zisserman, 2011] Yusuf Aytar and Andrew Zis- serman. Tabula rasa: Model transfer for object category detection. In 2011 ICCV, pages 2252-2259. IEEE, 2011.","title":"Yusuf Aytar and Andrew Zisserman. Tabula rasa: Model transfer for object category detection","authors":"Zisserman Aytar","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beygelzimer et al., 2013] A Beygelzimer, S Kakadet, J Langford, S Arya, D Mount, and S Li. FNN: Fast nearest neighbor search algorithms and applications. R package version 1.1, 2013.","title":"FNN: Fast nearest neighbor search algorithms and applications","authors":"Beygelzimer, J Kakadet, S Langford, D Arya, Mount, Li","year":"2013","venue":"R package version","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bottou and Lin, 2007] Léon Bottou and Chih-Jen Lin. Sup- port vector machine solvers. Large scale kernel machines, pages 301-320, 2007.","title":"Support vector machine solvers","authors":"Lin ; Bottou, Léon Bottou, Chih-Jen Lin","year":"2007","venue":"Large scale kernel machines","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chang and Lin, 2011] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM TIST, 2(3):27, 2011.","title":"LIBSVM: A library for support vector machines","authors":"Chang, Lin ; ] Chih-Chung Chang, Chih-Jen Lin","year":"2011","venue":"ACM TIST","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cristianini and Shawe-Taylor, 2000] Nello Cristianini and John Shawe-Taylor. An introduction to support vector ma- chines and other kernel-based learning methods. Cam- bridge University Press, 2000.","title":"An introduction to support vector machines and other kernel-based learning methods","authors":"Shawe-Taylor ; Nello Cristianini, John Cristianini, Shawe-Taylor","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dai et al., 2007] Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu. Boosting for transfer learning. In ICML, pages 193-200. ACM, 2007.","title":"Boosting for transfer learning","authors":"Wenyuan Dai, Qiang Yang, Gui-Rong Xue, Yong Yu","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Daumé III, 2009] Hal Daumé III. Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815, 2009.","title":"Hal Daumé III. Frustratingly easy domain adaptation","authors":"Iii Daumé","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duan et al., 2009] Lixin Duan, Ivor W Tsang, Dong Xu, and Tat-Seng Chua. Domain adaptation from multiple sources via auxiliary classifiers. In ICML, pages 289-296. ACM, 2009.","title":"Domain adaptation from multiple sources via auxiliary classifiers","authors":"Lixin Duan, Ivor W Tsang, Dong Xu, Tat-Seng Chua","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Duan et al., 2012] Lixin Duan, Ivor W Tsang, and Dong Xu. Domain transfer multiple kernel learning. IEEE TPAMI, 34(3):465-479, 2012.","title":"Domain transfer multiple kernel learning","authors":"Lixin Duan, Ivor W Tsang, Dong Xu","year":"2012","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gong et al., 2012] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In IEEE CVPR, pages 2066-2073, 2012.","title":"Geodesic flow kernel for unsupervised domain adaptation","authors":"Gong","year":"2012","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Griffin et al., 2007] Gregory Griffin, Alex Holub, and Pietro Perona. Caltech-256 object category dataset. Technical report, 2007.","title":"Caltech-256 object category dataset","authors":"Gregory Griffin, Alex Holub, Pietro Perona","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hausser and Strimmer, 2014] Jean Hausser and Korbinian Strimmer. entropy: Estimation of Entropy, Mutual Infor- mation and Related Quantities, 2014. R package v.1.2.1.","title":"entropy: Estimation of Entropy, Mutual Information and Related Quantities","authors":"Strimmer Hausser, Jean Hausser, Korbinian Strimmer","year":"2014","venue":"R package","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hoffman et al., 2012] Judy Hoffman, Brian Kulis, Trevor Darrell, and Kate Saenko. Discovering latent domains for multisource domain adaptation. In European Conference on Computer Vision, pages 702-715. Springer, 2012.","title":"Discovering latent domains for multisource domain adaptation","authors":"Hoffman","year":"2012","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kuzborskij and Orabona, 2013] Ilja Kuzborskij and Francesco Orabona. Stability and hypothesis transfer learning. In ICML, pages 942-950, 2013.","title":"Ilja Kuzborskij and Francesco Orabona. Stability and hypothesis transfer learning","authors":"Orabona Kuzborskij","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kuzborskij et al., 2015] Ilja Kuzborskij, Francesco Orabona, and Barbara Caputo. Transfer learning through greedy subset selection. In International Conference on Image Analysis and Processing, pages 3-14. Springer, 2015.","title":"Transfer learning through greedy subset selection","authors":"Ilja Kuzborskij, Francesco Orabona, Barbara Caputo","year":"2015","venue":"International Conference on Image Analysis and Processing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lapin et al., 2014] Maksim Lapin, Matthias Hein, and Bernt Schiele. Learning using privileged information: SVM+ and weighted SVM. Neural Networks, 53:95-108, 2014.","title":"Learning using privileged information: SVM+ and weighted SVM","authors":"Lapin","year":"2014","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mozafari and Jamzad, 2016] Azadeh Sadat Mozafari and Mansour Jamzad. A SVM-based model-transferring method for heterogeneous domain adaptation. Pattern Recognition, 56:142-158, 2016.","title":"A SVM-based model-transferring method for heterogeneous domain adaptation","authors":"Azadeh Sadat, Mozafari, Mansour Jamzad","year":"2016","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Oneto et al., 2015] Luca Oneto, Alessandro Ghio, Sandro Ridella, and Davide Anguita. Shrinkage learning to im- prove SVM with hints. In IJCNN, pages 1-9. IEEE, 2015.","title":"Shrinkage learning to improve SVM with hints","authors":"Luca Oneto, Alessandro Ghio, Sandro Ridella, Davide Anguita","year":"2015","venue":"IJCNN","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pan and Yang, 2010] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE TKDE, 22(10):1345- 1359, 2010.","title":"A survey on transfer learning","authors":"Jialin Sinno, Qiang Pan, Yang","year":"2010","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pechyony and Vapnik, 2011] Dmitry Pechyony and Vladimir Vapnik. Fast optimization algorithms for solving SVM+. Statistical Learning and Data Science, 4:3-24, 2011.","title":"Fast optimization algorithms for solving SVM+","authors":"Dmitry Pechyony, Vladimir Vapnik","year":"2011","venue":"Statistical Learning and Data Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schölkopf and Smola, 2002] Bernhard Schölkopf and Alexander J Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT Press, 2002.","title":"Learning with kernels: support vector machines, regularization, optimization, and beyond","authors":"Smola Schölkopf, Bernhard Schölkopf, Alexander J Smola","year":"2002","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schölkopf et al., 2001] Bernhard Schölkopf, Ralf Herbrich, and Alex J Smola. A generalized representer theorem. In International Conference on Computational Learning Theory, pages 416-426. Springer, 2001.","title":"A generalized representer theorem","authors":"Bernhard Schölkopf, Ralf Herbrich, Alex J Smola","year":"2001","venue":"International Conference on Computational Learning Theory","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tommasi and Tuytelaars, 2014] Tatiana Tommasi and Tinne Tuytelaars. A testbed for cross-dataset analysis. In European Conference on Computer Vision, pages 18-31. Springer, 2014.","title":"A testbed for cross-dataset analysis","authors":"Tatiana Tommasi, Tinne Tuytelaars","year":"2014","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tommasi et al., 2014] Tatiana Tommasi, Francesco Orabona, and Barbara Caputo. Learning categories from few examples with multi model knowledge transfer. IEEE TPAMI, 36(5):928-941, 2014.","title":"Learning categories from few examples with multi model knowledge transfer","authors":"Tatiana Tommasi, Francesco Orabona, Barbara Caputo","year":"2014","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Valerio et al., 2016] Lorenzo Valerio, Andrea Passarella, and Marco Conti. Hypothesis transfer learning for effi- cient data computing in smart cities environments. In IEEE SMARTCOMP, pages 1-8. IEEE, 2016.","title":"Hypothesis transfer learning for efficient data computing in smart cities environments","authors":"Lorenzo Valerio, Andrea Passarella, Marco Conti","year":"2016","venue":"IEEE SMARTCOMP","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vapnik and Izmailov, 2016] Vladimir Vapnik and Rauf Iz- mailov. Learning with intelligent teacher. In Symposium on Conformal and Probabilistic Prediction with Applica- tions, pages 3-19. Springer, 2016.","title":"Vladimir Vapnik and Rauf Izmailov. Learning with intelligent teacher","authors":"Izmailov Vapnik","year":"2016","venue":"Symposium on Conformal and Probabilistic Prediction with Applications","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang and Hebert, 2016] Yu-Xiong Wang and Martial Hebert. Learning by transferring from unsupervised universal sources. In AAAI, pages 2187-2193, 2016.","title":"Learning by transferring from unsupervised universal sources","authors":"Yu-Xiong Wang, Martial Hebert","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yang et al., 2007] Jun Yang, Rong Yan, and Alexander G Hauptmann. Cross-domain video concept detection us- ing adaptive SVMs. In ACM International Conference on Multimedia, pages 188-197. ACM, 2007.","title":"Cross-domain video concept detection using adaptive SVMs","authors":"Jun Yang, Rong Yan, Alexander G Hauptmann","year":"2007","venue":"ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2016] Joey Tianyi Zhou, Xinxing Xu, Sinno Jialin Pan, Ivor W Tsang, Zheng Qin, and Rick Siow Mong Goh. Transfer hashing with privileged information. arXiv preprint arXiv:1605.04034, 2016.","title":"Transfer hashing with privileged information","authors":"Joey Tianyi Zhou, Xinxing Xu, Sinno Jialin Pan, Ivor W Tsang, Zheng Qin, Rick Siow, Mong Goh","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/344"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}