{"type":"ScholarlyArticle","title":"Low-Bit Quantization for Attributed Network Representation Learning","authors":[{"name":"Hong Yang","slug":"hong-yang","given":"Hong","family":"Yang","affiliation":"Centre for Artificial Intelligence , University of Technology Sydney , Australia"},{"name":"Shirui Pan","slug":"shirui-pan","given":"Shirui","family":"Pan","affiliation":"Faculty of Information Technology , Monash University , Australia"},{"name":"Ling Chen","slug":"ling-chen","given":"Ling","family":"Chen","affiliation":"Centre for Artificial Intelligence , University of Technology Sydney , Australia"},{"name":"Chuan Zhou","slug":"chuan-zhou","given":"Chuan","family":"Zhou","affiliation":"Institute of Information Engineering , Chinese Academy of Sciences , China"},{"name":"Peng Zhang","slug":"peng-zhang","given":"Peng","family":"Zhang","affiliation":"Ant Financial Services Group , Hangzhou , China"}],"abstract":"Attributed network embedding plays an important role in transferring network data into compact vectors for effective network analysis. Existing attributed network embedding models are designed either in continuous Euclidean spaces which introduce data redundancy or in binary coding spaces which incur significant loss of representation accuracy. To this end, we present a new Low-Bit Quantization for Attributed Network Representation Learning model (LQANR for short) that can learn compact node representations with low bitwidth values while preserving high representation accuracy. Specifically, we formulate a new representation learning function based on matrix factorization that can jointly learn the low-bit node representations and the layer aggregation weights under the low-bit quantization constraint. Because the new learning function falls into the category of mixed integer optimization, we propose an efficient mixed-integer based alternating direction method of multipliers (ADMM) algorithm as the solution. Experiments on real-world node classification and link prediction tasks validate the promising results of the proposed LQANR model.","doi":"10.24963/ijcai.2019/562","url":"https://ijcai.org/proceedings/2019/562","pdf":"https://ijcai.org/proceedings/2019/0562.pdf","markdown":"https://ijcai.org/proceedings/2019/562.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/562","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":562,"pages":{"first":4047,"last":4053},"keywords":["Machine Learning: Data Mining","Machine Learning Applications: Networks"],"videos":[],"references":[{"position":1,"raw":"[Chang et al., 2015] Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C Aggarwal, and Thomas S Huang. Heterogeneous network embedding via deep architectures. In KDD, pages 119-128. ACM, 2015.","title":"Heterogeneous network embedding via deep architectures","authors":"Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C Aggarwal, Thomas S Huang","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Courbariaux et al., 2016] Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Ben- gio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830, 2016.","title":"Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1","authors":"Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, Yoshua Bengio","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Denil et al., 2013] Misha Denil, Babak Shakibi, Laurent Dinh, Nando De Freitas, et al. Predicting parameters in deep learning. In Advances in neural information process- ing systems, pages 2148-2156, 2013.","title":"Predicting parameters in deep learning","authors":"Misha Denil, Babak Shakibi, Laurent Dinh, Nando De Freitas","year":"2013","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fan et al., 2008] Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. JMLR, 9(Aug):1871- 1874, 2008.","title":"Liblinear: A library for large linear classification","authors":"Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, Chih-Jen Lin","year":"2008","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Grover and Leskovec, 2016] Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In KDD, pages 855-864. ACM, 2016.","title":"Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks","authors":"Leskovec Grover","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hanley and McNeil, 1982] James A Hanley and Barbara J McNeil. The meaning and use of the area under a re- ceiver operating characteristic (roc) curve. Radiology, 143(1):29-36, 1982.","title":"The meaning and use of the area under a receiver operating characteristic (roc) curve","authors":"A James, Barbara J Hanley, Mcneil","year":"1982","venue":"Radiology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2017a] Xiao Huang, Jundong Li, and Xia Hu. Accelerated attributed network embedding. In SDM, pages 633-641. SIAM, 2017.","title":"Accelerated attributed network embedding","authors":"Huang","year":"2017","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2017b] Xiao Huang, Jundong Li, and Xia Hu. Label informed attributed network embedding. In WSDM, pages 731-739. ACM, 2017.","title":"Label informed attributed network embedding","authors":"Huang","year":"2017","venue":"WSDM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kong and Li, 2012] Weihao Kong and Wu Jun Li. Double- bit quantization for hashing. In AAAI, pages 634-640, 2012.","title":"Doublebit quantization for hashing","authors":"Li ; Kong, Weihao Kong, Wu Jun, Li","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kong et al., 2012] Weihao Kong, Wu-Jun Li, and Minyi Guo. Manhattan hashing for large-scale image retrieval. In SIGIR, pages 45-54. ACM, 2012.","title":"Manhattan hashing for large-scale image retrieval","authors":"Kong","year":"2012","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Le and Lauw, 2014] Tuan MV Le and Hady W Lauw. Prob- abilistic latent document network embedding. In ICDM, pages 270-279. IEEE, 2014.","title":"Probabilistic latent document network embedding","authors":"M V Tuan, Hady W Le, Lauw","year":"2014","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Leng et al., 2017] Cong Leng, Hao Li, Shenghuo Zhu, and Rong Jin. Extremely low bit neural network: Squeeze the last bit out with admm. In AAAI, 2017.","title":"Extremely low bit neural network: Squeeze the last bit out with admm","authors":"Cong Leng, Hao Li, Shenghuo Zhu, Rong Jin","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2017] Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, and Huan Liu. Attributed network em- bedding for learning in a dynamic environment. In CIKM, pages 387-396. ACM, 2017.","title":"Attributed network embedding for learning in a dynamic environment","authors":"Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, Huan Liu","year":"2017","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2014] Wei Liu, Cun Mu, Sanjiv Kumar, and Shih- Fu Chang. Discrete graph hashing. In NIPS, pages 3419- 3427, 2014.","title":"Discrete graph hashing","authors":"Wei Liu, Cun Mu, Sanjiv Kumar, Shih-Fu Chang","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Moran et al., 2013] Sean Moran, Victor Lavrenko, and Miles Osborne. Variable bit quantisation for lsh. In ACL, pages 753-758, 2013.","title":"Variable bit quantisation for lsh","authors":"Sean Moran, Victor Lavrenko, Miles Osborne","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ortega et al., 2018] Antonio Ortega, Pascal Frossard, Jelena Kovacevic, Jos M. F. Moura, and Pierre Vandergheynst. Graph signal processing: Overview, challenges, and appli- cations. Proceedings of the IEEE, 106(5):808-828, 2018.","title":"Graph signal processing: Overview, challenges, and applications","authors":"Ortega","year":"2018","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Perozzi et al., 2014] Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social rep- resentations. In KDD, pages 701-710. ACM, 2014.","title":"Deepwalk: Online learning of social representations","authors":"Bryan Perozzi, Rami Al-Rfou, Steven Skiena","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sandryhaila and Moura, 2013] Aliaksei Sandryhaila and Jos M. F. Moura. Discrete signal processing on graphs. TSP, 61(7):1644-1656, 2013.","title":"Discrete signal processing on graphs","authors":"Aliaksei Sandryhaila, M F Jos, Moura","year":"2013","venue":"TSP","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, pages 37-45, 2015.","title":"Supervised discrete hashing","authors":"Fumin Shen, Chunhua Shen, Wei Liu, Heng Tao Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shen et al., 2018a] Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, and Heng Tao Shen. Unsupervised deep hashing with similarity-adaptive and discrete optimization. TPAMI, 2018.","title":"Unsupervised deep hashing with similarity-adaptive and discrete optimization","authors":"Shen","year":"2018","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen et al., 2018b] Xiaobo Shen, Shirui Pan, Weiwei Liu, Yew-Soon Ong, and Quan-Sen Sun. Discrete network em- bedding. In IJCAI, pages 3549-3555, 2018.","title":"Discrete network embedding","authors":"Shen","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/493"},{"position":22,"raw":"[Wang et al., 2017] Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen, et al. A survey on learning to hash. TPAMI, 2017.","title":"A survey on learning to hash","authors":"Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2018] Wei Wu, Bin Li, Ling Chen, and Chengqi Zhang. Efficient Attributed Network Embedding via Re- cursive Randomized Hashing. In IJCAI-18, pages 2861- 2867, 2018.","title":"Efficient Attributed Network Embedding via Recursive Randomized Hashing","authors":"Wei Wu, Bin Li, Ling Chen, Chengqi Zhang","year":"2018","venue":"IJCAI-18","doi":null,"url":null,"internal_url":"/proceedings/2018/397"},{"position":24,"raw":"[Xia et al., 2010] Tian Xia, Dacheng Tao, Tao Mei, and Yongdong Zhang. Multiview spectral embedding. TSMC- B, 40(6):1438-1446, 2010.","title":"Multiview spectral embedding","authors":"Tian Xia, Dacheng Tao, Tao Mei, Yongdong Zhang","year":"2010","venue":"TSMC-B","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yan et al., 2007] Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, and Stephen Lin. Graph embedding and extensions: A general framework for di- mensionality reduction. TPAMI, 29(1):40-51, 2007.","title":"Graph embedding and extensions: A general framework for dimensionality reduction","authors":"Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, Stephen Lin","year":"2007","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2015] Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. Network represen- tation learning with rich text information. In IJCAI, pages 2111-2117, 2015.","title":"Network representation learning with rich text information","authors":"Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, Edward Y Chang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/299"},{"position":27,"raw":"[Yang et al., 2018] Hong Yang, Shirui Pan, Peng Zhang, Ling Chen, Defu Lian, and Chengqi Zhang. Binarized at- tributed network embedding. In 2018 IEEE International Conference on Data Mining (ICDM), pages 1476-1481. IEEE, 2018.","title":"Binarized attributed network embedding","authors":"Yang","year":"2018","venue":"2018 IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2016] Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. Homophily, structure, and content augmented network representation learning. In ICDM, pages 609-618. IEEE, 2016.","title":"Homophily, structure, and content augmented network representation learning","authors":"Zhang","year":"2016","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhu et al., 2016] Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally. Trained ternary quantization. arXiv preprint arXiv:1612.01064, 2016.","title":"Trained ternary quantization","authors":"Zhu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}