{"type":"ScholarlyArticle","title":"Personalized Multimedia Item and Key Frame Recommendation","authors":[{"name":"Le Wu","slug":"le-wu","given":"Le","family":"Wu","affiliation":"Hefei University of Technology"},{"name":"Lei Chen","slug":"lei-chen","given":"Lei","family":"Chen","affiliation":"Hefei University of Technology"},{"name":"Yonghui Yang","slug":"yonghui-yang","given":"Yonghui","family":"Yang","affiliation":"Hefei University of Technology"},{"name":"Richang Hong","slug":"richang-hong","given":"Richang","family":"Hong","affiliation":"Hefei University of Technology"},{"name":"Yong Ge","slug":"yong-ge","given":"Yong","family":"Ge","affiliation":"The University of Arizona"},{"name":"Xing Xie","slug":"xing-xie","given":"Xing","family":"Xie","affiliation":"Microsoft Research"},{"name":"Meng Wang","slug":"meng-wang","given":"Meng","family":"Wang","affiliation":"Hefei University of Technology"}],"abstract":"When recommending or advertising items to users, an emerging trend is to present each multimedia item with  a key frame image (e.g., the poster of a movie). As each multimedia item can be represented as  multiple fine-grained  visual images (e.g., related images of the movie), personalized key frame recommendation is necessary in these applications to attract users' unique visual preferences. However, previous personalized key frame recommendation models relied on users' fine grained image  behavior of  multimedia items (e.g., user-image interaction behavior), which is often not available in real scenarios.  In this paper, we study the general problem of joint multimedia item and key frame recommendation in the absence of the fine-grained user-image behavior. We argue that the key challenge of this problem lies in discovering users' visual profiles for key frame recommendation, as most recommendation models  would fail without any users' fine-grained image behavior. To tackle this challenge, we leverage users' item behavior by projecting users(items) in two latent spaces: a collaborative latent space and a visual latent space. We further design a model to discern both the collaborative and  visual dimensions of users, and model how users make decisive item preferences from these two spaces. As a result, the learned user visual profiles could be directly applied for key frame recommendation. Finally, experimental results on a real-world dataset clearly show the effectiveness of our proposed model on the two recommendation tasks.","doi":"10.24963/ijcai.2019/198","url":"https://ijcai.org/proceedings/2019/198","pdf":"https://ijcai.org/proceedings/2019/0198.pdf","markdown":"https://ijcai.org/proceedings/2019/198.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/198","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Humans and AI","track":"Main track","number":198,"pages":{"first":1431,"last":1437},"keywords":["Humans and AI: Personalization and User Modeling","Multidisciplinary Topics and Applications: Recommender Systems"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: a system for large-scale machine learn- ing. In OSDI, volume 16, pages 265-283, 2016.","title":"Tensorflow: a system for large-scale machine learning","authors":"Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard","year":"2016","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Adomavicius and Tuzhilin, 2005] Gediminas Adomavicius and Alexander Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. TKDE, 17(6):734-749, 2005.","title":"Gediminas Adomavicius and Alexander Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions","authors":"Tuzhilin Adomavicius","year":"2005","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017a] Jingyuan Chen, Hanwang Zhang, Xi- angnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. Attentive collaborative filtering: Multimedia recommen- dation with item-and component-level attention. In SIGIR, pages 335-344, 2017.","title":"Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention","authors":"Chen","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2017b] Xu Chen, Yongfeng Zhang, Qingyao Ai, Hongteng Xu, Junchi Yan, and Zheng Qin. Person- alized key frame recommendation. In SIGIR, pages 315- 324, 2017.","title":"Personalized key frame recommendation","authors":"Chen","year":"2017","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2018] Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. Neural attentional rating regression with review-level explanations. In WWW, pages 1583-1592, 2018.","title":"Neural attentional rating regression with review-level explanations","authors":"Chong Chen, Min Zhang, Yiqun Liu, Shaoping Ma","year":"2018","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2019] Lei Chen, Le Wu, Zhenzhen Hu, and Meng Wang. Quality-aware unpaired image-to-image translation. TMM, 2019.","title":"Quality-aware unpaired image-to-image translation","authors":"Lei Chen, Le Wu, Zhenzhen Hu, Meng Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2018] Zhiyong Cheng, Ying Ding, Xiangnan He, Lei Zhu, Xuemeng Song, and Mohan S Kankanhalli. Aˆ3ncf: An adaptive aspect attention model for rating pre- diction. In IJCAI, pages 3748-3754, 2018.","title":"Aˆ3ncf: An adaptive aspect attention model for rating prediction","authors":"Zhiyong Cheng, Ying Ding, Xiangnan He, Lei Zhu, Xuemeng Song, Mohan S Kankanhalli","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/521"},{"position":8,"raw":"[Covington et al., 2016] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recom- mendations. In Recsys, pages 191-198, 2016.","title":"Deep neural networks for youtube recommendations","authors":"Paul Covington, Jay Adams, Emre Sargin","year":"2016","venue":"Recsys","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He and McAuley, 2016] Ruining He and Julian McAuley. Vbpr: Visual bayesian personalized ranking from implicit feedback. In AAAI, pages 144-150, 2016.","title":"Ruining He and Julian McAuley. Vbpr: Visual bayesian personalized ranking from implicit feedback","authors":"Mcauley He","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2018] Xiangnan He, Zhenkui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. Nais: Neural attentive item similarity model for recommenda- tion. TKDE, 2018.","title":"Nais: Neural attentive item similarity model for recommendation","authors":"He","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2018] Liang Hu, Songlei Jian, Longbing Cao, and Qingkui Chen. Interpretable recommendation via attrac- tion modeling: Learning multilevel attractiveness over multimodal movie contents. In IJCAI, pages 3400-3406, 2018.","title":"Interpretable recommendation via attraction modeling: Learning multilevel attractiveness over multimodal movie contents","authors":"Liang Hu, Songlei Jian, Longbing Cao, Qingkui Chen","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/472"},{"position":12,"raw":"[Koren et al., 2009] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recom- mender systems. Computer, (8):30-37, 2009.","title":"Matrix factorization techniques for recommender systems","authors":"Koren","year":"2009","venue":"Computer","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lee et al., 2011] Yong Jae Lee, Jaechul Kim, and Kristen Grauman. Key-segments for video object segmentation. In CVPR, pages 1995-2002, 2011.","title":"Key-segments for video object segmentation","authors":"Jae Yong, Jaechul Lee, Kristen Kim, Grauman","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lei et al., 2016] Chenyi Lei, Dong Liu, Weiping Li, Zheng- Jun Zha, and Houqiang Li. Comparative deep learning of hybrid representations for image recommendations. In CVPR, pages 2545-2553, 2016.","title":"Comparative deep learning of hybrid representations for image recommendations","authors":"Chenyi Lei, Dong Liu, Weiping Li, Zheng-Jun Zha, Houqiang Li","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Matz et al., 2017] Sandra C Matz, Michal Kosinski, Gideon Nave, and David J Stillwell. Psychological targeting as an effective approach to digital mass persuasion. PNAS, 114(48):12714-12719, 2017.","title":"Psychological targeting as an effective approach to digital mass persuasion","authors":"Michal Sandra C Matz, Gideon Kosinski, David J Nave, Stillwell","year":"2017","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mundur et al., 2006] Padmavathi Mundur, Yong Rao, and Yelena Yesha. Keyframe-based video summarization us- ing delaunay clustering. International Journal on Digital Libraries, 6(2):219-232, 2006.","title":"Keyframe-based video summarization using delaunay clustering","authors":"Padmavathi Mundur, Yong Rao, Yelena Yesha","year":"2006","venue":"International Journal on Digital Libraries","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rendle et al., 2009] Steffen Rendle, Christoph Freuden- thaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In UAI, pages 452-461, 2009.","title":"Bpr: Bayesian personalized ranking from implicit feedback","authors":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme","year":"2009","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi et al., 2018] Shaoyun Shi, Min Zhang, Yiqun Liu, and Shaoping Ma. Attention-based adaptive model to unify warm and cold starts recommendation. In CIKM, pages 127-136, 2018.","title":"Attention-based adaptive model to unify warm and cold starts recommendation","authors":"Shi","year":"2018","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Stelzner, 2018] Michael Stelzner. 2018 social media mar- keting industry report. Social Media Examiner, pages 1- 44, 2018.","title":"social media marketing industry report","authors":"Michael Stelzner","year":"2018","venue":"Social Media Examiner","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sun et al., 2018] Peijie Sun, Le Wu, and Meng Wang. At- tentive recurrent social recommendation. In SIGIR, pages 185-194, 2018.","title":"Attentive recurrent social recommendation","authors":"Sun","year":"2018","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wan et al., 2014] Ji Wan, Dayong Wang, Steven Chu Hong Hoi, Pengcheng Wu, Jianke Zhu, Yongdong Zhang, and Jintao Li. Deep learning for content-based image retrieval: A comprehensive study. In MM, pages 157-166, 2014.","title":"Deep learning for content-based image retrieval: A comprehensive study","authors":"Ji Wan, Dayong Wang, Steven Chu, Hong Hoi, Pengcheng Wu, Jianke Zhu, Yongdong Zhang, Jintao Li","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2017] Suhang Wang, Yilin Wang, Jiliang Tang, Kai Shu, Suhas Ranganath, and Huan Liu. What your images reveal: Exploiting visual contents for point-of- interest recommendation. In WWW, pages 391-400, 2017.","title":"What your images reveal: Exploiting visual contents for point-ofinterest recommendation","authors":"Wang","year":"2017","venue":"WWW","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2017] Le Wu, Yong Ge, Qi Liu, Enhong Chen, Richang Hong, Junping Du, and Meng Wang. Modeling the evolution of users' preferences and social links in so- cial networking services. TKDE, 29(6):1240-1253, 2017.","title":"Modeling the evolution of users' preferences and social links in social networking services","authors":"Le Wu, Yong Ge, Qi Liu, Enhong Chen, Richang Hong, Junping Du, Meng Wang","year":"2017","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2019] Le Wu, Lei Chen, Richang Hong, Yanjie Fu, Xing Xie, and Meng Wang. A hierarchical atten- tion model for social contextual image recommendation. TKDE, 2019.","title":"A hierarchical attention model for social contextual image recommendation","authors":"Le Wu, Lei Chen, Richang Hong, Yanjie Fu, Xing Xie, Meng Wang","year":"2019","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yin et al., 2018] Yu Yin, Zhenya Huang, Enhong Chen, Qi Liu, Fuzheng Zhang, Xing Xie, and Guoping Hu. Transcribing content from structural images with spotlight mechanism. In SIGKDD, pages 2643-2652, 2018.","title":"Transcribing content from structural images with spotlight mechanism","authors":"Yu Yin, Zhenya Huang, Enhong Chen, Qi Liu, Fuzheng Zhang, Xing Xie, Guoping Hu","year":"2018","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2013] Dong Zhang, Omar Javed, and Mubarak Shah. Video object segmentation through spatially accu- rate and temporally dense extraction of primary object re- gions. In CVPR, pages 628-635, 2013.","title":"Video object segmentation through spatially accurate and temporally dense extraction of primary object regions","authors":"Dong Zhang, Omar Javed, Mubarak Shah","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2018] Kun Zhang, Guangyi Lv, Le Wu, En- hong Chen, Qi Liu, Han Wu, and Fangzhao Wu. Image- enhanced multi-level sentence representation net for natu- ral language inference. In ICDM, pages 747-756, 2018.","title":"Imageenhanced multi-level sentence representation net for natural language inference","authors":"Kun Zhang, Guangyi Lv, Le Wu, Enhong Chen, Qi Liu, Han Wu, Fangzhao Wu","year":"2018","venue":"ICDM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}