{"type":"ScholarlyArticle","title":"Learning to Explain Entity Relationships by Pairwise Ranking with Convolutional Neural Networks","authors":[{"name":"Jizhou Huang","slug":"jizhou-huang","given":"Jizhou","family":"Huang","affiliation":"† Research Center for Social Computing and Information Retrieval , Harbin Institute of Technology , China"},{"name":"Wei Zhang","slug":"wei-zhang","given":"Wei","family":"Zhang","affiliation":"‡ Baidu Inc. , Beijing , China"},{"name":"Shiqi Zhao","slug":"shiqi-zhao","given":"Shiqi","family":"Zhao","affiliation":"‡ Baidu Inc. , Beijing , China"},{"name":"Shiqiang Ding","slug":"shiqiang-ding","given":"Shiqiang","family":"Ding","affiliation":"‡ Baidu Inc. , Beijing , China"},{"name":"Haifeng Wang","slug":"haifeng-wang","given":"Haifeng","family":"Wang","affiliation":"‡ Baidu Inc. , Beijing , China"}],"abstract":"Providing a plausible explanation for the relationship between two related entities is an important task in some applications of knowledge graphs, such as in search engines. However, most existing methods require a large number of manually labeled training data, which cannot be applied in large-scale knowledge graphs due to the expensive data annotation. In addition, these methods typically rely on costly handcrafted features. In this paper, we propose an effective pairwise ranking model by leveraging clickthrough data of a Web search engine to address these two problems. We first construct large-scale training data by leveraging the query-title pairs derived from clickthrough data of a Web search engine. Then, we build a pairwise ranking model which employs a convolutional neural network to automatically learn relevant features. The proposed model can be easily trained with backpropagation to perform the ranking task. The experiments show that our method significantly outperforms several strong baselines.","doi":"10.24963/ijcai.2017/561","url":"https://ijcai.org/proceedings/2017/561","pdf":"https://ijcai.org/proceedings/2017/0561.pdf","markdown":"https://ijcai.org/proceedings/2017/561.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/561","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Natural Language Processing","track":"Main track","number":561,"pages":{"first":4018,"last":4025},"keywords":["Natural Language Processing: NLP Applications and Tools","Natural Language Processing: Natural Language Processing"],"videos":[],"references":[{"position":1,"raw":"[Agichtein et al., 2006] E. Agichtein, E. Brill, S. Dumais, and R. Ragno. Learning user interaction models for pre- dicting web search result preferences. In SIGIR, 2006.","title":"Learning user interaction models for predicting web search result preferences","authors":"E Agichtein, E Brill, S Dumais, R Ragno","year":"2006","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banko et al., 2007] M. Banko, M. J Cafarella, S. Soderland, M. Broadhead, and O. Etzioni. Open information extrac- tion from the web. In IJCAI, 2007.","title":"Open information extraction from the web","authors":"M Banko, M Cafarella, S Soderland, M Broadhead, O Etzioni","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/429"},{"position":3,"raw":"[Blanco and Zaragoza, 2010] R. Blanco and H. Zaragoza. Finding support sentences for entities. In SIGIR, 2010.","title":"Finding support sentences for entities","authors":"R Blanco, H Zaragoza","year":"2010","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blanco et al., 2013] R. Blanco, B. B. Cambazoglu, P. Mika, and N. Torzec. Entity recommendations in web search. In ISWC, 2013.","title":"Entity recommendations in web search","authors":"R Blanco, B B Cambazoglu, P Mika, N Torzec","year":"2013","venue":"ISWC","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bordes et al., 2013] A. Bordes, N. Usunier, A. Garcia- Duran, J. Weston, and O. Yakhnenko. Translating embed- dings for modeling multi-relational data. In Advances in Neural Information Processing Systems, 2013.","title":"Translating embeddings for modeling multi-relational data","authors":"A Bordes, N Usunier, A Garcia-Duran, J Weston, O Yakhnenko","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Breiman, 2001] L. Breiman. Random forests. Machine learning, 45(1):5-32, 2001.","title":"Random forests","authors":"L Breiman","year":"2001","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Carletta, 1996] J. Carletta. Assessing agreement on classifi- cation tasks: The kappa statistic. Computational Linguis- tics, 1996.","title":"Assessing agreement on classification tasks: The kappa statistic","authors":"J Carletta","year":"1996","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chapelle et al., 2009] O. Chapelle, D. Metlzer, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded rel- evance. In CIKM, 2009.","title":"Expected reciprocal rank for graded relevance","authors":"O Chapelle, D Metlzer, Y Zhang, P Grinspan","year":"2009","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dou et al., 2008] Z. Dou, R. Song, X. Yuan, and J.-R. Wen. Are click-through data adequate for learning web search rankings? In CIKM, 2008.","title":"Are click-through data adequate for learning web search rankings?","authors":"Z Dou, R Song, X Yuan, J.-R Wen","year":"2008","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fader et al., 2011] A. Fader, S. Soderland, and O. Etzioni. Identifying relations for open information extraction. In EMNLP, 2011.","title":"Identifying relations for open information extraction","authors":"A Fader, S Soderland, O Etzioni","year":"2011","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Fang et al., 2011] L. Fang, A. D. Sarma, C. Yu, and P. Bo- hannon. REX: explaining relationships between entity pairs. Proceedings of the VLDB Endowment, 5(3), 2011.","title":"REX: explaining relationships between entity pairs","authors":"L Fang, A D Sarma, C Yu, P Bohannon","year":"2011","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gao et al., 2010] J. Gao, X. He, and J.-Y. Nie. Clickthrough-based translation models for web search: from word models to phrase models. In CIKM, 2010.","title":"Clickthrough-based translation models for web search: from word models to phrase models","authors":"J Gao, X He, J.-Y Nie","year":"2010","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Herbrich et al., 1999] R. Herbrich, T. Graepel, and K. Ober- mayer. Large margin rank boundaries for ordinal regres- sion. Advances in Neural Information Processing Systems, 1999.","title":"Large margin rank boundaries for ordinal regression","authors":"R Herbrich, T Graepel, K Obermayer","year":"1999","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2013] P.-S. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. Heck. Learning deep structured semantic models for web search using clickthrough data. In CIKM, 2013.","title":"Learning deep structured semantic models for web search using clickthrough data","authors":"P.-S Huang, X He, J Gao, L Deng, A Acero, L Heck","year":"2013","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2016] J. Huang, S. Zhao, S. Ding, H. Wu, M. Sun, and H. Wang. Generating recommendation evi- dence using translation model. In IJCAI, 2016.","title":"Generating recommendation evidence using translation model","authors":"J Huang, S Zhao, S Ding, H Wu, M Sun, H Wang","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/399"},{"position":16,"raw":"[Jarvelin and Kekalainen, 2002] K. Jarvelin and J. Kekalainen. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4):422-446, 2002.","title":"Cumulated gain-based evaluation of ir techniques","authors":"K Jarvelin, J Kekalainen","year":"2002","venue":"ACM Transactions on Information Systems (TOIS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ji et al., 2016] G. Ji, K. Liu, S. He, and J. Zhao. Knowledge graph completion with adaptive sparse transfer matrix. In AAAI, 2016.","title":"Knowledge graph completion with adaptive sparse transfer matrix","authors":"G Ji, K Liu, S He, J Zhao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Joachims et al., 2005] T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting click- through data as implicit feedback. In SIGIR, 2005.","title":"Accurately interpreting clickthrough data as implicit feedback","authors":"T Joachims, L Granka, B Pan, H Hembrooke, G Gay","year":"2005","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Joachims et al., 2007] T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query refor- mulations in web search. ACM Transactions on Informa- tion Systems (TOIS), 25(2), 2007.","title":"Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search","authors":"T Joachims, L Granka, B Pan, H Hembrooke, F Radlinski, G Gay","year":"2007","venue":"ACM Transactions on Information Systems (TOIS)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Joachims, 2002] T. Joachims. Optimizing search engines using clickthrough data. In SIGKDD, 2002.","title":"Optimizing search engines using clickthrough data","authors":"T Joachims","year":"2002","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kim, 2014] Y. Kim. Convolutional neural networks for sen- tence classification. In EMNLP, 2014.","title":"Convolutional neural networks for sentence classification","authors":"Y Kim","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Landis and Koch, 1977] J R. Landis and G. G Koch. The measurement of observer agreement for categorical data. Biometrics, 33(1), March 1977.","title":"The measurement of observer agreement for categorical data","authors":"J R Landis, G G Koch","year":"1977","venue":"Biometrics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lin et al., 2015] Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu. Learning entity and relation embeddings for knowledge graph completion. In AAAI, 2015.","title":"Learning entity and relation embeddings for knowledge graph completion","authors":"Y Lin, Z Liu, M Sun, Y Liu, X Zhu","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Liu, 2009] T.-Y. Liu. Learning to rank for information re- trieval. Foundations and Trends in Information Retrieval, 3(3):225-331, 2009.","title":"Learning to rank for information retrieval","authors":"T.-Y Liu","year":"2009","venue":"Foundations and Trends in Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mikolov et al., 2013] T. Mikolov, I. Sutskever, K. Chen, G. S Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 2013.","title":"Distributed representations of words and phrases and their compositionality","authors":"T Mikolov, I Sutskever, K Chen, G S Corrado, J Dean","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Santos et al., 2015] C. N. D. Santos, B. Xiang, and B. Zhou. Classifying relations by ranking with convolutional neural networks. In ACL-IJCNLP, 2015.","title":"Classifying relations by ranking with convolutional neural networks","authors":"C N D Santos, B Xiang, B Zhou","year":"2015","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Severyn and Moschitti, 2015] A. Severyn and A. Moschitti. Learning to rank short text pairs with convolutional deep neural networks. In SIGIR, 2015.","title":"Learning to rank short text pairs with convolutional deep neural networks","authors":"A Severyn, A Moschitti","year":"2015","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Singhal, 2012] A. Singhal. Introducing the knowledge graph: things, not strings. Official Blog of Google: https://googleblog.blogspot.com/2012/05/introducing- knowledge-graph-things-not.html, 2012.","title":"Introducing the knowledge graph: things, not strings","authors":"A Singhal","year":null,"venue":null,"doi":null,"url":"https://googleblog.blogspot.com/2012/05/introducing-knowledge-graph-things-not.html,2012","internal_url":null},{"position":29,"raw":"[Socher et al., 2012] R. Socher, B. Huval, C. Manning, and A. Ng. Semantic compositionality through recursive matrix-vector spaces. In EMNLP, 2012.","title":"Semantic compositionality through recursive matrix-vector spaces","authors":"R Socher, B Huval, C Manning, A Ng","year":"2012","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Srivastava et al., 2014] N. Srivastava, G. E Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"N Srivastava, G Hinton, A Krizhevsky, I Sutskever, R Salakhutdinov","year":"2014","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Voskarides et al., 2015] N. Voskarides, E. Meij, M. Tsagkias, M. De Rijke, and W. Weerkamp. Learning to explain entity relationships in knowledge graphs. In ACL-IJCNLP, 2015.","title":"Learning to explain entity relationships in knowledge graphs","authors":"N Voskarides, E Meij, M Tsagkias, M De Rijke, W Weerkamp","year":"2015","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2014] Z. Wang, J. Zhang, J. Feng, and Z. Chen. Knowledge graph embedding by translating on hyper- planes. In AAAI, 2014.","title":"Knowledge graph embedding by translating on hyperplanes","authors":"Z Wang, J Zhang, J Feng, Z Chen","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yu et al., 2014] X. Yu, H. Ma, B. P. Hsu, and J. Han. On building entity recommender systems using user click log and freebase knowledge. In WSDM, 2014.","title":"On building entity recommender systems using user click log and freebase knowledge","authors":"Yu","year":"2014","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zeng et al., 2015] D. Zeng, K. Liu, Y. Chen, and J. Zhao. Distant supervision for relation extraction via piecewise convolutional neural networks. In EMNLP, 2015.","title":"Distant supervision for relation extraction via piecewise convolutional neural networks","authors":"D Zeng, K Liu, Y Chen, J Zhao","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhao et al., 2010] S. Zhao, H. Wang, and T. Liu. Paraphras- ing with search engine query logs. In COLING, 2010.","title":"Paraphrasing with search engine query logs","authors":"S Zhao, H Wang, T Liu","year":"2010","venue":"COLING","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zheng et al., 2007] Z. Zheng, K. Chen, G. Sun, and H. Zha. A regression framework for learning ranking functions us- ing relative relevance judgments. In SIGIR, 2007.","title":"A regression framework for learning ranking functions using relative relevance judgments","authors":"Z Zheng, K Chen, G Sun, H Zha","year":"2007","venue":"SIGIR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}