{"type":"ScholarlyArticle","title":"Binarized Collaborative Filtering with Distilling Graph Convolutional Network","authors":[{"name":"Haoyu Wang","slug":"haoyu-wang","given":"Haoyu","family":"Wang","affiliation":"School of Computer Science and Technology , University of Science and Technology of China; University of Electronic Science and Technology of China"},{"name":"Defu Lian","slug":"defu-lian","given":"Defu","family":"Lian","affiliation":"School of Computer Science and Technology , University of Science and Technology of China"},{"name":"Yong Ge","slug":"yong-ge","given":"Yong","family":"Ge","affiliation":"University of Arizona"}],"abstract":"The efficiency of top-K item recommendation based on implicit feedback are vital to  recommender systems in real world, but it is very challenging due to the lack of negative samples and the large number of candidate items. To address the challenges, we firstly introduce an improved Graph Convolutional Network~(GCN) model with high-order feature interaction considered. Then we distill the ranking information derived from GCN into binarized collaborative filtering, which makes use of binary representation to improve the efficiency of online recommendation. However, binary codes are not only hard to be optimized but also likely to incur the loss of information during the training processing. Therefore, we propose a novel framework to convert the binary constrained optimization problem into an equivalent continuous optimization problem with a stochastic penalty. The binarized collaborative filtering model is then easily optimized by many popular solvers like SGD and Adam. The proposed algorithm is finally evaluated on three real-world datasets and shown the superiority to the competing baselines.","doi":"10.24963/ijcai.2019/667","url":"https://ijcai.org/proceedings/2019/667","pdf":"https://ijcai.org/proceedings/2019/0667.pdf","markdown":"https://ijcai.org/proceedings/2019/667.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/667","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":667,"pages":{"first":4802,"last":4808},"keywords":["Multidisciplinary Topics and Applications: Recommender Systems","Multidisciplinary Topics and Applications: Information Retrieval"],"videos":[],"references":[{"position":1,"raw":"[Berg et al., 2017] Rianne van den Berg, Thomas N Kipf, and Max Welling. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263, 2017.","title":"Graph convolutional matrix completion","authors":"Rianne Van Den, Thomas N Berg, Max Kipf, Welling","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2007] Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. Learning to rank: from pairwise approach to list- wise approach. In Proceedings of ICML'07, pages 129-136. ACM, 2007.","title":"Learning to rank: from pairwise approach to listwise approach","authors":"Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, Hang Li","year":"2007","venue":"Proceedings of ICML'07","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. Darkrank: Accelerating deep metric learning via cross sample similarities transfer. In Proceedings of AAAI'18, 2018.","title":"Darkrank: Accelerating deep metric learning via cross sample similarities transfer","authors":"Yuntao Chen, Naiyan Wang, Zhaoxiang Zhang","year":"2018","venue":"Proceedings of AAAI'18","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Courbariaux et al., 2015] Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural net- works with binary weights during propagations. In Proceedings of NeurIPS'15, pages 3123-3131, 2015.","title":"Binaryconnect: Training deep neural networks with binary weights during propagations","authors":"Matthieu Courbariaux, Yoshua Bengio, Jean-Pierre David","year":"2015","venue":"Proceedings of NeurIPS'15","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dai et al., 2016] Qi Dai, Jianguo Li, Jingdong Wang, and Yu-Gang Jiang. Binary optimized hashing. In Proceedings of MM'16, pages 1247-1256. ACM, 2016.","title":"Binary optimized hashing","authors":"Qi Dai, Jianguo Li, Jingdong Wang, Yu-Gang Jiang","year":"2016","venue":"Proceedings of MM'16","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Das et al., 2007] Abhinandan S Das, Mayur Datar, Ashutosh Garg, and Shyam Rajaram. Google news personalization: scalable on- line collaborative filtering. In Proceedings of WWW'07, pages 271-280. ACM, 2007.","title":"Google news personalization: scalable online collaborative filtering","authors":"Mayur Abhinandan S Das, Ashutosh Datar, Shyam Garg, Rajaram","year":"2007","venue":"Proceedings of WWW'07","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Datar et al., 2004] Mayur Datar, Nicole Immorlica, Piotr Indyk, and Vahab S Mirrokni. Locality-sensitive hashing scheme based on p-stable distributions. In Proceedings of the twentieth annual symposium on Computational geometry, pages 253-262. ACM, 2004.","title":"Locality-sensitive hashing scheme based on p-stable distributions","authors":"Mayur Datar, Nicole Immorlica, Piotr Indyk, Vahab, Mirrokni","year":"2004","venue":"Proceedings of the twentieth annual symposium on Computational geometry","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2018] Hongyang Gao, Zhengyang Wang, and Shui- wang Ji. Large-scale learnable graph convolutional networks. In Proceedings of KDD'18, pages 1416-1424. ACM, 2018.","title":"Large-scale learnable graph convolutional networks","authors":"Gao","year":"2018","venue":"Proceedings of KDD'18","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Giannessi and Tardella, 1998] Franco Giannessi and Fabio Tardella. Connections between nonlinear programming and dis- crete optimization. In Handbook of combinatorial optimization, pages 149-188. Springer, 1998.","title":"Franco Giannessi and Fabio Tardella. Connections between nonlinear programming and discrete optimization","authors":"Tardella Giannessi","year":"1998","venue":"Handbook of combinatorial optimization","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Håstad, 2001] Johan Håstad. Some optimal inapproximability re- sults. Journal of the ACM (JACM), 48(4):798-859, 2001.","title":"Some optimal inapproximability results","authors":"Johan Håstad, Håstad","year":"2001","venue":"Journal of the ACM (JACM)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Henaff et al., 2015] Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163, 2015.","title":"Deep convolutional networks on graph-structured data","authors":"Mikael Henaff, Joan Bruna, Yann Lecun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by re- ducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.","title":"Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Karatzoglou et al., 2010] Alexandros Karatzoglou, Alex Smola, and Markus Weimer. Collaborative filtering on a budget. In Pro- ceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 389-396, 2010.","title":"Collaborative filtering on a budget","authors":"Alexandros Karatzoglou, Alex Smola, Markus Weimer","year":"2010","venue":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2018a] Zhi Li, Hongke Zhao, Qi Liu, Zhenya Huang, Tao Mei, and Enhong Chen. Learning from history and present: Next- item recommendation via discriminatively exploiting user behav- iors. In Proceedings of KDD'18, pages 1734-1743. ACM, 2018.","title":"Learning from history and present: Nextitem recommendation via discriminatively exploiting user behaviors","authors":"Li","year":"2018","venue":"Proceedings of KDD'18","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2018b] Zhuohan Li, Di He, Fei Tian, Wei Chen, Tao Qin, Liwei Wang, and Tie-Yan Liu. Towards binary-valued gates for robust lstm training. arXiv preprint arXiv:1806.02988, 2018.","title":"Towards binary-valued gates for robust lstm training","authors":"Li","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lian et al., 2016] Defu Lian, Yuyang Ye, Wenya Zhu, Qi Liu, Xing Xie, and Hui Xiong. Mutual reinforcement of academic perfor- mance prediction and library book recommendation. In Proceed- ings of ICDM'16, pages 1023-1028. IEEE, 2016.","title":"Mutual reinforcement of academic performance prediction and library book recommendation","authors":"Defu Lian, Yuyang Ye, Wenya Zhu, Qi Liu, Xing Xie, Hui Xiong","year":"2016","venue":"Proceedings of ICDM'16","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2011] Qi Liu, Yong Ge, Zhongmou Li, Enhong Chen, and Hui Xiong. Personalized travel package recommendation. In Proceedings of ICDM'11, pages 407-416. IEEE, 2011.","title":"Personalized travel package recommendation","authors":"Qi Liu, Yong Ge, Zhongmou Li, Enhong Chen, Hui Xiong","year":"2011","venue":"Proceedings of ICDM'11","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2014] Xianglong Liu, Junfeng He, Cheng Deng, and Bo Lang. Collaborative hashing. In Proceedings of CVPR'14, pages 2139-2146, 2014.","title":"Collaborative hashing","authors":"Xianglong Liu, Junfeng He, Cheng Deng, Bo Lang","year":"2014","venue":"Proceedings of CVPR'14","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lucidi and Rinaldi, 2010] Stefano Lucidi and Francesco Rinaldi. Exact penalty functions for nonlinear integer programming problems. Journal of optimization theory and applications, 145(3):479-488, 2010.","title":"Exact penalty functions for nonlinear integer programming problems","authors":"Stefano Lucidi, Francesco Rinaldi","year":"2010","venue":"Journal of optimization theory and applications","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Monti et al., 2017] Federico Monti, Michael Bronstein, and Xavier Bresson. Geometric matrix completion with recurrent multi-graph neural networks. In Advances in Neural Information Processing Systems, pages 3697-3707, 2017.","title":"Geometric matrix completion with recurrent multi-graph neural networks","authors":"Federico Monti, Michael Bronstein, Xavier Bresson","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Muja and Lowe, 2009] Marius Muja and David G Lowe. Fast ap- proximate nearest neighbors with automatic algorithm configura- tion. VISAPP (1), 2(331-340):2, 2009.","title":"Fast approximate nearest neighbors with automatic algorithm configuration","authors":"Lowe Muja, Marius Muja, David G Lowe","year":"2009","venue":"VISAPP","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pan et al., 2008] Rong Pan, Yunhong Zhou, Bin Cao, Nathan N Liu, Rajan Lukose, Martin Scholz, and Qiang Yang. One-class collaborative filtering. In Proceedings of ICDM'08, pages 502- 511. IEEE, 2008.","title":"One-class collaborative filtering","authors":"Rong Pan, Yunhong Zhou, Bin Cao, Nathan N Liu, Rajan Lukose, Martin Scholz, Qiang Yang","year":"2008","venue":"Proceedings of ICDM'08","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rendle et al., 2009] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian person- alized ranking from implicit feedback. In Proceedings of UAI'09, pages 452-461. AUAI Press, 2009.","title":"Bpr: Bayesian personalized ranking from implicit feedback","authors":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme","year":"2009","venue":"Proceedings of UAI'09","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tang and Wang, 2018] Jiaxi Tang and Ke Wang. Ranking distilla- tion: Learning compact ranking models with high performance for recommender system. In Proceedings of KDD'18, pages 2289-2298. ACM, 2018.","title":"Ranking distillation: Learning compact ranking models with high performance for recommender system","authors":"Wang ; ] Jiaxi Tang, Ke Tang, Wang","year":"2018","venue":"Proceedings of KDD'18","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2012] Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. Semi-supervised hashing for large-scale search. IEEE Transac- tions on Pattern Analysis & Machine Intelligence, (12):2393- 2406, 2012.","title":"Semi-supervised hashing for large-scale search","authors":"Jun Wang, Sanjiv Kumar, Shih-Fu Chang","year":"2012","venue":"IEEE Transactions on Pattern Analysis & Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2017] Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. In Pro- ceedings of the ADKDD'17, page 12. ACM, 2017.","title":"Deep & cross network for ad click predictions","authors":"Wang","year":"2017","venue":"Proceedings of the ADKDD'17","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Ying et al., 2018] Rex Ying, Ruining He, Kaifeng Chen, Pong Ek- sombatchai, William L Hamilton, and Jure Leskovec. Graph con- volutional neural networks for web-scale recommender systems. In Proceedings of KDD'18, pages 974-983. ACM, 2018.","title":"Graph convolutional neural networks for web-scale recommender systems","authors":"Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, Jure Leskovec","year":"2018","venue":"Proceedings of KDD'18","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2014] Zhiwei Zhang, Qifan Wang, Lingyun Ruan, and Luo Si. Preference preserving hashing for efficient recom- mendation. In Proceedings of SIGIR'14, pages 183-192. ACM, 2014.","title":"Preference preserving hashing for efficient recommendation","authors":"Zhiwei Zhang, Qifan Wang, Lingyun Ruan, Luo Si","year":"2014","venue":"Proceedings of SIGIR'14","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2016] Hanwang Zhang, Fumin Shen, Wei Liu, Xi- angnan He, Huanbo Luan, and Tat-Seng Chua. Discrete collab- orative filtering. In Proceedings of SIGIR'16, pages 325-334. ACM, 2016.","title":"Discrete collaborative filtering","authors":"Hanwang Zhang, Fumin Shen, Wei Liu, Xiangnan He, Huanbo Luan, Tat-Seng Chua","year":"2016","venue":"Proceedings of SIGIR'16","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zheng et al., 2018] Lei Zheng, Chun-Ta Lu, Fei Jiang, Jiawei Zhang, and Philip S Yu. Spectral collaborative filtering. In Pro- ceedings of RecSys'18, pages 311-319. ACM, 2018.","title":"Spectral collaborative filtering","authors":"Lei Zheng, Chun-Ta Lu, Fei Jiang, Jiawei Zhang, Philip S Yu","year":"2018","venue":"Proceedings of RecSys'18","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhou and Zha, 2012] Ke Zhou and Hongyuan Zha. Learning bi- nary codes for collaborative filtering. In Proceedings of KDD'18, pages 498-506. ACM, 2012.","title":"Learning binary codes for collaborative filtering","authors":"Zha ; ] Ke Zhou, Hongyuan Zhou, Zha","year":"2012","venue":"Proceedings of KDD'18","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}