{"type":"ScholarlyArticle","title":"Patent Citation Dynamics Modeling via Multi-Attention Recurrent Networks","authors":[{"name":"Taoran Ji","slug":"taoran-ji","given":"Taoran","family":"Ji","affiliation":"Discovery Analytics Center , Virginia Tech , Arlington , VA 22203 , USA; Department of Computer Science , Virginia Tech , Arlington , VA 22203 , USA"},{"name":"Zhiqian Chen","slug":"zhiqian-chen","given":"Zhiqian","family":"Chen","affiliation":"Discovery Analytics Center , Virginia Tech , Arlington , VA 22203 , USA; Department of Computer Science , Virginia Tech , Arlington , VA 22203 , USA"},{"name":"Nathan Self","slug":"nathan-self","given":"Nathan","family":"Self","affiliation":"Discovery Analytics Center , Virginia Tech , Arlington , VA 22203 , USA; Department of Computer Science , Virginia Tech , Arlington , VA 22203 , USA"},{"name":"Kaiqun Fu","slug":"kaiqun-fu","given":"Kaiqun","family":"Fu","affiliation":"Discovery Analytics Center , Virginia Tech , Arlington , VA 22203 , USA; Department of Computer Science , Virginia Tech , Arlington , VA 22203 , USA"},{"name":"Chang-Tien Lu","slug":"chang-tien-lu","given":"Chang-Tien","family":"Lu","affiliation":"Discovery Analytics Center , Virginia Tech , Arlington , VA 22203 , USA; Department of Computer Science , Virginia Tech , Arlington , VA 22203 , USA"},{"name":"Naren Ramakrishnan","slug":"naren-ramakrishnan","given":"Naren","family":"Ramakrishnan","affiliation":"Discovery Analytics Center , Virginia Tech , Arlington , VA 22203 , USA; Department of Computer Science , Virginia Tech , Arlington , VA 22203 , USA"}],"abstract":"Modeling and forecasting forward citations to a patent is a central task for the discovery of emerging technologies and for measuring the pulse of inventive progress. Conventional methods for forecasting these forward citations cast the problem as analysis of temporal point processes which rely on the conditional intensity of previously received citations. Recent approaches model the conditional intensity as a chain of recurrent neural networks to capture memory dependency in hopes of reducing the restrictions of the parametric form of the intensity function. For the problem of patent citations, we observe that forecasting a patent's chain of citations benefits from not only the patent's history itself but also from the historical citations of assignees and inventors associated with that patent. In this paper, we propose a sequence-to-sequence model which employs an attention-of-attention mechanism to capture the dependencies of these multiple time sequences. Furthermore, the proposed model is able to forecast both the timestamp and the category of a patent's next citation. Extensive experiments on a large patent citation dataset collected from USPTO demonstrate that the proposed model outperforms state-of-the-art models at forward citation forecasting.","doi":"10.24963/ijcai.2019/364","url":"https://ijcai.org/proceedings/2019/364","pdf":"https://ijcai.org/proceedings/2019/0364.pdf","markdown":"https://ijcai.org/proceedings/2019/364.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/364","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":364,"pages":{"first":2621,"last":2627},"keywords":["Machine Learning: Deep Learning","Machine Learning: Time-series;Data Streams","Machine Learning Applications: Other Applications"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bessen, 2008] James Bessen. The value of us patents by owner and patent characteristics. Research Policy, 37(5):932-945, 2008.","title":"The value of us patents by owner and patent characteristics","authors":"James Bessen","year":"2008","venue":"Research Policy","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Du et al., 2016] Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Em- bedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1555-1564. ACM, 2016.","title":"Recurrent marked temporal point processes: Embedding event history to vector","authors":"Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, Le Song","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Filimonov and Sornette, 2015] Vladimir Filimonov and Di- dier Sornette. Apparent criticality and calibration issues in the hawkes self-excited point process model: application to high-frequency financial data. Quantitative Finance, 15(8):1293-1314, 2015.","title":"Vladimir Filimonov and Didier Sornette. Apparent criticality and calibration issues in the hawkes self-excited point process model: application to high-frequency financial data","authors":"Sornette Filimonov","year":"2015","venue":"Quantitative Finance","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hall et al., 2001] Bronwyn H Hall, Adam B Jaffe, and Manuel Trajtenberg. The nber patent citation data file: Lessons, insights and methodological tools. Technical re- port, National Bureau of Economic Research, 2001.","title":"The nber patent citation data file: Lessons, insights and methodological tools","authors":"Bronwyn H Hall, Adam B Jaffe, Manuel Trajtenberg","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hawkes, 1971] Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.","title":"Spectra of some self-exciting and mutually exciting point processes","authors":"Alan G Hawkes","year":"1971","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Helmstetter and Sornette, 2002] Agnès Helmstetter and Di- dier Sornette. Subcritical and supercritical regimes in epi- demic models of earthquake aftershocks. Journal of Geo- physical Research: Solid Earth, 107(B10):ESE-10, 2002.","title":"Agnès Helmstetter and Didier Sornette. Subcritical and supercritical regimes in epidemic models of earthquake aftershocks","authors":"Sornette Helmstetter","year":"2002","venue":"Journal of Geophysical Research: Solid Earth","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Isham and Westcott, 1979] Valerie Isham and Mark West- cott. A self-correcting point process. Stochastic Processes and Their Applications, 8(3):335-347, 1979.","title":"A self-correcting point process","authors":"Valerie Isham, Mark Westcott","year":"1979","venue":"Stochastic Processes and Their Applications","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jang et al., 2017] Hyun Jin Jang, Han-Gyun Woo, and Changyong Lee. Hawkes process-based technology im- pact analysis. Journal of Informetrics, 11(2):511-529, 2017.","title":"Hawkes process-based technology impact analysis","authors":"Jin Hyun, Han-Gyun Jang, Changyong Woo, Lee","year":"2017","venue":"Journal of Informetrics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee et al., 2018] Changyong Lee, Ohjin Kwon, Myeongjung Kim, and Daeil Kwon. Early identifi- cation of emerging technologies: A machine learning approach using multiple patent indicators. Technological Forecasting and Social Change, 127:291-303, 2018.","title":"Early identification of emerging technologies: A machine learning approach using multiple patent indicators","authors":"Changyong Lee, Ohjin Kwon, Myeongjung Kim, Daeil Kwon","year":"2018","venue":"Technological Forecasting and Social Change","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2013] Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. arXiv preprint arXiv:1312.4400, 2013.","title":"Network in network","authors":"Lin","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2017] Xin Liu, Junchi Yan, Shuai Xiao, Xi- angfeng Wang, Hongyuan Zha, and Stephen M Chu. On predictive patent valuation: Forecasting patent citations and their types. In Thirty-First AAAI Conference on Ar- tificial Intelligence, 2017.","title":"On predictive patent valuation: Forecasting patent citations and their types","authors":"Xin Liu, Junchi Yan, Shuai Xiao, Xiangfeng Wang, Hongyuan Zha, Stephen M Chu","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Luong et al., 2015] Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025, 2015.","title":"Effective approaches to attention-based neural machine translation","authors":"Minh-Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mishra et al., 2016] Swapnil Mishra, Marian-Andrei Ri- zoiu, and Lexing Xie. Feature driven and point process approaches for popularity prediction. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pages 1069-1078. ACM, 2016.","title":"Feature driven and point process approaches for popularity prediction","authors":"Swapnil Mishra, Marian-Andrei Rizoiu, Lexing Xie","year":"2016","venue":"Proceedings of the 25th ACM International on Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112, 2014.","title":"Quoc V Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Von Wartburg et al., 2005] Iwan Von Wartburg, Thorsten Teichert, and Katja Rost. Inventive progress measured by multi-stage patent citation analysis. Research Policy, 34(10):1591-1607, 2005.","title":"Inventive progress measured by multi-stage patent citation analysis","authors":"Von Iwan, Thorsten Wartburg, Katja Teichert, Rost","year":"2005","venue":"Research Policy","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wallinga and Teunis, 2004] Jacco Wallinga and Peter Teu- nis. Different epidemic curves for severe acute respi- ratory syndrome reveal similar impacts of control mea- sures. American Journal of epidemiology, 160(6):509- 516, 2004.","title":"Different epidemic curves for severe acute respiratory syndrome reveal similar impacts of control measures","authors":"Teunis Wallinga, Jacco Wallinga, Peter Teunis","year":"2004","venue":"American Journal of epidemiology","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2017] Yongqing Wang, Huawei Shen, Shenghua Liu, Jinhua Gao, and Xueqi Cheng. Cas- cade dynamics modeling with attention-based recurrent neural network. In Proceedings of the 26th Interna- tional Joint Conference on Artificial Intelligence, pages 2985-2991. AAAI Press, 2017.","title":"Cascade dynamics modeling with attention-based recurrent neural network","authors":"Yongqing Wang, Huawei Shen, Shenghua Liu, Jinhua Gao, Xueqi Cheng","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/416"},{"position":22,"raw":"[Xiao et al., 2016] Shuai Xiao, Junchi Yan, Changsheng Li, Bo Jin, Xiangfeng Wang, Xiaokang Yang, Stephen M Chu, and Hongyuan Zha. On modeling and predicting individ- ual paper citation count over time. In IJCAI, pages 2676- 2682, 2016.","title":"On modeling and predicting individual paper citation count over time","authors":"Shuai Xiao, Junchi Yan, Changsheng Li, Bo Jin, Xiangfeng Wang, Xiaokang Yang, Stephen M Chu, Hongyuan Zha","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiao et al., 2017] Shuai Xiao, Junchi Yan, Xiaokang Yang, Hongyuan Zha, and Stephen M Chu. Modeling the in- tensity function of point process via recurrent neural net- works. In Thirty-First AAAI Conference on Artificial In- telligence, 2017.","title":"Modeling the intensity function of point process via recurrent neural networks","authors":"Shuai Xiao, Junchi Yan, Xiaokang Yang, Hongyuan Zha, Stephen M Chu","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}