{"type":"ScholarlyArticle","title":"Hierarchical Diffusion Attention Network","authors":[{"name":"Zhitao Wang","slug":"zhitao-wang","given":"Zhitao","family":"Wang","affiliation":"Department of Computing , The Hong Kong Polytechnic University , Hong Kong"},{"name":"Wenjie Li","slug":"wenjie-li","given":"Wenjie","family":"Li","affiliation":"Department of Computing , The Hong Kong Polytechnic University , Hong Kong"}],"abstract":"A series of recent studies formulated the diffusion prediction problem as a sequence prediction task and proposed several sequential models based on recurrent neural networks. However, non-sequential properties exist in real diffusion cascades, which do not strictly follow the sequential assumptions of previous work. In this paper, we propose a hierarchical diffusion attention network (HiDAN), which adopts a non-sequential framework and two-level attention mechanisms, for diffusion prediction. At the user level, a dependency attention mechanism is proposed to dynamically capture historical user-to-user dependencies and extract the dependency-aware user information. At the cascade (i.e., sequence) level, a time-aware influence attention is designed to infer possible future user's dependencies on historical users by considering both inherent user importance and time decay effects. Significantly higher effectiveness and efficiency of HiDAN over state-of-the-art sequential models are demonstrated when evaluated on three real diffusion datasets. The further case studies illustrate that HiDAN can accurately capture diffusion dependencies.","doi":"10.24963/ijcai.2019/531","url":"https://ijcai.org/proceedings/2019/531","pdf":"https://ijcai.org/proceedings/2019/0531.pdf","markdown":"https://ijcai.org/proceedings/2019/531.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/531","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":531,"pages":{"first":3828,"last":3834},"keywords":["Machine Learning: Deep Learning","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Bahdanau et al., 2015] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations, 2015.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bourigault et al., 2016] Simon Bourigault, Sylvain Lam- prier, and Patrick Gallinari. Representation learning for information diffusion through social networks: an embed- ded cascade model. In Proceedings of the Ninth ACM In- ternational Conference on Web Search and Data Mining, pages 573-582. ACM, 2016.","title":"Representation learning for information diffusion through social networks: an embedded cascade model","authors":"Simon Bourigault, Sylvain Lamprier, Patrick Gallinari","year":"2016","venue":"Proceedings of the Ninth ACM International Conference on Web Search and Data Mining","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2017] Qi Cao, Huawei Shen, Keting Cen, Wen- tao Ouyang, and Xueqi Cheng. Deephawkes: Bridging the gap between prediction and understanding of information cascades. In Proceedings of the 2017 ACM on Confer- ence on Information and Knowledge Management, pages 1149-1158. ACM, 2017.","title":"Deephawkes: Bridging the gap between prediction and understanding of information cascades","authors":"Qi Cao, Huawei Shen, Keting Cen, Wentao Ouyang, Xueqi Cheng","year":"2017","venue":"Proceedings of the 2017 ACM on Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Clevert et al., 2016] Djork-Arné Clevert, Thomas Un- terthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). International Conference on Learning Representations, 2016.","title":"Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units","authors":"Clevert","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Du et al., 2016] Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Em- bedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1555-1564. ACM, 2016.","title":"Recurrent marked temporal point processes: Embedding event history to vector","authors":"Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, Le Song","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gomez Rodriguez et al., 2011] Manuel Gomez Rodriguez, David Balduzzi, and Bernhard Schölkopf. Uncovering the temporal dynamics of diffusion networks. In Proceedings of the 28th International Conference on Machine Learn- ing, pages 561-568. ACM, 2011.","title":"Uncovering the temporal dynamics of diffusion networks","authors":"Manuel Gomez Rodriguez, David Balduzzi, Bernhard Schölkopf","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kempe et al., 2003] David Kempe, Jon Kleinberg, and Éva Tardos. Maximizing the spread of influence through a so- cial network. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 137-146. ACM, 2003.","title":"Maximizing the spread of influence through a social network","authors":"David Kempe, Jon Kleinberg, Éva Tardos","year":"2003","venue":"Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Leskovec et al., 2009] Jure Leskovec, Lars Backstrom, and Jon Kleinberg. Meme-tracking and the dynamics of the news cycle. In Proceedings of the 15th ACM SIGKDD In- ternational Conference on Knowledge Discovery and Data Mining, pages 497-506. ACM, 2009.","title":"Meme-tracking and the dynamics of the news cycle","authors":"Jure Leskovec, Lars Backstrom, Jon Kleinberg","year":"2009","venue":"Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Leskovec et al., 2010] Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg, Christos Faloutsos, and Zoubin Ghahramani. Kronecker graphs: An approach to modeling networks. Journal of Machine Learning Research, 11(Feb):985-1042, 2010.","title":"Kronecker graphs: An approach to modeling networks","authors":"Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg, Christos Faloutsos, Zoubin Ghahramani","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2017] Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-attentive sentence embedding. International Conference on Learning Repre- sentations, 2017.","title":"A structured self-attentive sentence embedding","authors":"Lin","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Luong et al., 2015] Thang Luong, Hieu Pham, and Christo- pher D Manning. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412-1421, 2015.","title":"Effective approaches to attention-based neural machine translation","authors":"Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Saito et al., 2009] Kazumi Saito, Masahiro Kimura, Kouzou Ohara, and Hiroshi Motoda. Learning continuous- time information diffusion model for social behavioral data analysis. In Asian Conference on Machine Learning, pages 322-337. Springer, 2009.","title":"Learning continuoustime information diffusion model for social behavioral data analysis","authors":"Kazumi Saito, Masahiro Kimura, Kouzou Ohara, Hiroshi Motoda","year":"2009","venue":"Asian Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shen et al., 2018] Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. Disan: Di- rectional self-attention network for rnn/cnn-free language understanding. In AAAI Conference on Artificial Intelli- gence, 2018.","title":"Disan: Directional self-attention network for rnn/cnn-free language understanding","authors":"Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, Chengqi Zhang","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey E Hin- ton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Sys- tems, pages 6000-6010. 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł Ukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2017a] Jia Wang, Vincent W Zheng, Zemin Liu, and Kevin Chen-Chuan Chang. Topological recurrent neural network for diffusion prediction. In Data Mining (ICDM), 2017 IEEE International Conference on, pages 475-484. IEEE, 2017.","title":"Topological recurrent neural network for diffusion prediction","authors":"Wang","year":"2017","venue":"Data Mining (ICDM), 2017 IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2017b] Yongqing Wang, Huawei Shen, Shenghua Liu, Jinhua Gao, and Xueqi Cheng. Cascade dynamics modeling with attention-based recurrent neural network. In Proceedings of the Twenty-Sixth Interna- tional Joint Conference on Artificial Intelligence, pages 2985-2991, 2017.","title":"Cascade dynamics modeling with attention-based recurrent neural network","authors":"Wang","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/416"},{"position":20,"raw":"[Weng et al., 2013] L. Weng, F Menczer, and Y. Y. Ahn. Vi- rality prediction and community structure in social net- works. Scientific Reports, 3(8):618-618, 2013.","title":"Virality prediction and community structure in social networks","authors":"L Weng, Y Y Menczer, Ahn","year":"2013","venue":"Scientific Reports","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2013] Jing Zhang, Biao Liu, Jie Tang, Ting Chen, and Juanzi Li. Social influence locality for mod- eling retweeting behaviors. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pages 2761-2767, 2013.","title":"Social influence locality for modeling retweeting behaviors","authors":"Jing Zhang, Biao Liu, Jie Tang, Ting Chen, Juanzi Li","year":"2013","venue":"Proceedings of the 23rd International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2013/406"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}