{"type":"ScholarlyArticle","title":"Stacking With Auxiliary Features","authors":[{"name":"Nazneen Fatema Rajani","slug":"nazneen-fatema-rajani","given":"Nazneen Fatema","family":"Rajani","affiliation":"Department of Computer Science University of Texas at Austin"},{"name":"Raymond J. Mooney","slug":"raymond-j-mooney","given":"Raymond J.","family":"Mooney","affiliation":"Department of Computer Science University of Texas at Austin"}],"abstract":"Ensembling methods are well known for improving prediction accuracy. However, they are limited in the sense that they cannot effectively discriminate among component models. In this paper, we propose stacking with auxiliary features that learns to fuse additional relevant information from multiple component systems as well as input instances to improve performance. We use two types of auxiliary features -- instance features and provenance features. The instance features enable the stacker to discriminate across input instances and the provenance features enable the stacker to discriminate across component systems. When combined together, our algorithm learns to rely on systems that not just agree on an output but also the provenance of this output in conjunction with the properties of the input instance. We demonstrate the success of our approach on three very different and challenging natural language and vision problems: Slot Filling, Entity Discovery and Linking, and ImageNet Object Detection. We obtain new state-of-the-art results on the first two tasks and significant improvements on the ImageNet task, thus verifying the power and generality of our approach.","doi":"10.24963/ijcai.2017/367","url":"https://ijcai.org/proceedings/2017/367","pdf":"https://ijcai.org/proceedings/2017/0367.pdf","markdown":"https://ijcai.org/proceedings/2017/367.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/367","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":367,"pages":{"first":2634,"last":2640},"keywords":["Natural Language Processing: Information Extraction","Machine Learning: Ensemble Methods"],"videos":[],"references":[{"position":1,"raw":"[Bollacker et al., 2008] K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, 2008.","title":"Freebase: a collaboratively created graph database for structuring human knowledge","authors":"K Bollacker, C Evans, P Paritosh, T Sturge, J Taylor","year":"2008","venue":"Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chang et al., 2016] H. Chang, M. Abdurrahman, A. Liu, J. Tian-Zheng Wei, A. Traylor, A. Nagesh, N. Monath, P. Verga, E. Strubell, and A. McCallum. Extracting Mul- tilingual Relations under Limited Resources: TAC 2016 Cold-Start KB construction and Slot-Filling using Compo- sitional Universal Schema. In Proceedings of TAC, 2016.","title":"Extracting Multilingual Relations under Limited Resources: TAC 2016 Cold-Start KB construction and Slot-Filling using Compositional Universal Schema","authors":"H Chang, M Abdurrahman, A Liu, J Tian-Zheng, A Wei, A Traylor, N Nagesh, P Monath, E Verga, A Strubell, Mccallum","year":"2016","venue":"Proceedings of TAC","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Dietterich, 2000] T. Dietterich. Ensemble methods in ma- chine learning. In First International Workshop on Multi- ple Classifier Systems, Lecture Notes in Computer Science, pages 1-15. Springer-Verlag, 2000.","title":"Ensemble methods in machine learning","authors":"T Dietterich","year":"2000","venue":"First International Workshop on Multiple Classifier Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Džeroski and Ženko, 2004] Saso Džeroski and Bernard Ženko. Is combining classifiers with stacking better than selecting the best one? Machine learning, 54(3):255-273, 2004.","title":"Is combining classifiers with stacking better than selecting the best one?","authors":"Saso Džeroski, Bernard Ženko","year":"2004","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Eigen et al., 2013] D. Eigen, M. Ranzato, and I. Sutskever. Learning factored representations in a deep mixture of ex- perts. arXiv preprint arXiv:1312.4314, 2013.","title":"Learning factored representations in a deep mixture of experts","authors":"D Eigen, M Ranzato, I Sutskever","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Felzenszwalb et al., 2010] P. Felzenszwalb, D. Girshick, R.and McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2010.","title":"Object detection with discriminatively trained part-based models","authors":"P Felzenszwalb, D Girshick, R Mcallester, D Ramanan","year":"2010","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ferguson et al., 2016] J. Ferguson, C. Lockard, N. Hawkins, S. Soderland, H. Hajishirzi, and D. Weld. University of washington tac-kbp 2016 system descrip- tion. In Proceedings of TAC, 2016.","title":"University of washington tac-kbp 2016 system description","authors":"J Ferguson, C Lockard, N Hawkins, S Soderland, H Hajishirzi, D Weld","year":"2016","venue":"Proceedings of TAC","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Francis-Landau et al., 2016] M. Francis-Landau, G. Dur- rett, and D. Klein. Capturing semantic similarity for entity linking with convolutional neural networks. In Proceed- ings of the NAACL, 2016.","title":"Capturing semantic similarity for entity linking with convolutional neural networks","authors":"M Francis-Landau, G Durrett, D Klein","year":"2016","venue":"Proceedings of the NAACL","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Girshick, 2015] Ross Girshick. Fast R-CNN. In Interna- tional Conference on Computer Vision, 2015.","title":"Fast R-CNN","authors":"Ross Girshick","year":"2015","venue":"International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2015] K. He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385, 2015.","title":"Deep residual learning for image recognition","authors":"K He, X Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jacobs et al., 1991] R.A Jacobs, M. Jordan, S. Nowlan, and G. Hinton. Adaptive mixtures of local experts. Neural computation, 1991.","title":"Adaptive mixtures of local experts","authors":"R Jacobs, M Jordan, S Nowlan, G Hinton","year":"1991","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ji et al., 2015] H. Ji, J. Nothman, B. Hachey, and R. Florian. Overview of TAC-KBP2015 Tri-lingual Entity Discovery and Linking. In Proceedings of TAC, 2015.","title":"Overview of TAC-KBP2015 Tri-lingual Entity Discovery and Linking","authors":"H Ji, J Nothman, B Hachey, R Florian","year":"2015","venue":"Proceedings of TAC","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ji et al., 2016] H. Ji, J. Nothman, and H. Trang Dang. Overview of TAC-KBP2016 Tri-lingual EDL and its im- pact on end-to-end Cold-Start KBP. In Proceedings of TAC, 2016.","title":"Overview of TAC-KBP2016 Tri-lingual EDL and its impact on end-to-end Cold-Start KBP","authors":"H Ji, J Nothman, H Trang Dang","year":"2016","venue":"Proceedings of TAC","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jia et al., 2014] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Y Jia, E Shelhamer, J Donahue, S Karayev, J Long, R Girshick, S Guadarrama, T Darrell","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kisiel et al., 2015] B. Kisiel, B. McDowell, M. Gardner, N. Nakashole, E. Platanios, A. Saparov, S. Srivastava, D. Wijaya, and T. Mitchell. CMUML system for KBP 2015 Cold Start Slot Filling. In Proceedings of the Eighth TAC, 2015.","title":"CMUML system for KBP 2015 Cold Start Slot Filling","authors":"B Kisiel, B Mcdowell, M Gardner, N Nakashole, E Platanios, A Saparov, S Srivastava, D Wijaya, T Mitchell","year":"2015","venue":"Proceedings of the Eighth TAC","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lowe, 2004] David G Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 2004.","title":"Distinctive image features from scale-invariant keypoints","authors":"David G Lowe","year":"2004","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mikolov et al., 2013] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"T Mikolov, I Sutskever, K Chen, G Corrado, J Dean","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mintz et al., 2009] M. Mintz, S. Bills, R. Snow, and D. Ju- rafsky. Distant supervision for relation extraction without labeled data. pages 1003-1011. ACL, 2009.","title":"Distant supervision for relation extraction without labeled data","authors":"M Mintz, S Bills, R Snow, D Jurafsky","year":"2009","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Peppoloni et al., 2014] L. Peppoloni, M. Satler, E. Luchetti, C. Avizzano, and P. Tripicchio. Stacked generalization for scene analysis and object recognition. In Proceedings of INES, 2014.","title":"Stacked generalization for scene analysis and object recognition","authors":"L Peppoloni, M Satler, E Luchetti, C Avizzano, P Tripicchio","year":"2014","venue":"Proceedings of INES","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ren et al., 2015] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with re- gion proposal networks. In Neural Information Processing Systems, 2015.","title":"Faster R-CNN: Towards real-time object detection with region proposal networks","authors":"S Ren, K He, R Girshick, J Sun","year":"2015","venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Russakovsky et al., 2015] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. Berg, and L. Fei-Fei. Ima- geNet Large Scale Visual Recognition Challenge. IJCV, 115(3), 2015.","title":"Ima-geNet Large Scale Visual Recognition Challenge","authors":"O Russakovsky, J Deng, H Su, J Krause, S Satheesh, S Ma, Z Huang, A Karpathy, A Khosla, M Bernstein, A Berg, L Fei-Fei","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sil et al., 2016] A. Sil, G. Dinu, and R. Florian. The IBM systems for trilingual entity discovery and linking at TAC 2016. In Proceedings of TAC, 2016.","title":"The IBM systems for trilingual entity discovery and linking at TAC 2016","authors":"A Sil, G Dinu, R Florian","year":"2016","venue":"Proceedings of TAC","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sill et al., 2009] J. Sill, G. Takács, L. Mackey, and D. Lin. Feature-weighted linear stacking. arXiv preprint arXiv:0911.0460, 2009.","title":"Feature-weighted linear stacking","authors":"J Sill, G Takács, L Mackey, D Lin","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Simonyan and Zisserman, 2015] K. Simonyan and A. Zis- serman. Very Deep Convolutional Networks for Large- scale Image Recognition. In Proceedings of ICLR, 2015.","title":"Very Deep Convolutional Networks for Largescale Image Recognition","authors":"K Simonyan, A Zisserman","year":"2015","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Uijlings et al., 2013] J. Uijlings, K. Van de Sande, T. Gev- ers, and A. Smeulders. Selective search for object recog- nition. IJCV, 2013.","title":"Selective search for object recognition","authors":"J Uijlings, K Van De Sande, T Gevers, A Smeulders","year":"2013","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Vilalta and Drissi, 2002] R. Vilalta and Y. Drissi. A Per- spective View and Survey of Meta-learning. Artificial In- telligence Review, 2002.","title":"A Perspective View and Survey of Meta-learning","authors":"R Vilalta, Y Drissi","year":"2002","venue":"Artificial Intelligence Review","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Viswanathan et al., 2015] V. Viswanathan, N. Rajani, Y. Bentor, and R. Mooney. Stacked ensembles of in- formation extractors for knowledge-base population. In Proceedings of ACL, 2015.","title":"Stacked ensembles of information extractors for knowledge-base population","authors":"V Viswanathan, N Rajani, Y Bentor, R Mooney","year":"2015","venue":"Proceedings of ACL","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wolpert, 1992] D. Wolpert. Stacked generalization. Neural Networks, 5, 1992.","title":"Stacked generalization","authors":"D Wolpert","year":"1992","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2016] Y. Zhang, A. Chaganty, A. Paranjape, D. Chen, J. Bolton, P. Qi, and C. Manning. Stanford at TAC KBP 2016: Sealing Pipeline Leaks and Understand- ing Chinese. In Proceedings of TAC, 2016.","title":"Stanford at TAC KBP 2016: Sealing Pipeline Leaks and Understanding Chinese","authors":"Y Zhang, A Chaganty, A Paranjape, D Chen, J Bolton, P Qi, C Manning","year":"2016","venue":"Proceedings of TAC","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}