{"type":"ScholarlyArticle","title":"Efficient Tuning and Inference for Large Language Models on Textual Graphs","authors":[{"name":"Yun Zhu","slug":"yun-zhu","given":"Yun","family":"Zhu","affiliation":"Zhejiang University"},{"name":"Yaoke Wang","slug":"yaoke-wang","given":"Yaoke","family":"Wang","affiliation":"Zhejiang University"},{"name":"Haizhou Shi","slug":"haizhou-shi","given":"Haizhou","family":"Shi","affiliation":"Rutgers University"},{"name":"Siliang Tang","slug":"siliang-tang","given":"Siliang","family":"Tang","affiliation":"Zhejiang University"}],"abstract":"Rich textual and topological information of textual graphs need to be modeled in real-world applications such as webpages, e-commerce, and academic articles. Practitioners have been long following the path of adopting a shallow text encoder and a subsequent graph neural network (GNN) to solve this problem. In light of recent advancements in large language models (LLMs), it is apparent that integrating LLMs for enhanced textual encoding can substantially improve the performance of textual graphs. Nevertheless, the efficiency of these methods poses a significant challenge. In this paper, we propose ENGINE, a parameter- and memory-efficient fine-tuning method for textual graphs with an LLM encoder. The key insight is to combine the LLMs and GNNs through a tunable side structure, which significantly reduces the training complexity without impairing the joint model's capacity. Extensive experiments on textual graphs demonstrate our method's effectiveness by achieving the best model performance, meanwhile having the lowest training cost compared to previous methods. Moreover, we introduce two variants with caching and dynamic early exit to further enhance training and inference speed. Specifically, caching accelerates ENGINE's training by 12x, and dynamic early exit achieves up to 5x faster inference with a negligible performance drop (at maximum 1.17% relevant drop across 7 datasets). Our codes are available at: https://github.com/ZhuYun97/ENGINE.","doi":"10.24963/ijcai.2024/634","url":"https://ijcai.org/proceedings/2024/634","pdf":"https://ijcai.org/proceedings/2024/0634.pdf","markdown":"https://ijcai.org/proceedings/2024/634.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/634","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":634,"pages":{"first":5734,"last":5742},"keywords":["Machine Learning: ML: Supervised Learning","Machine Learning: ML: Sequence and graph learning","Natural Language Processing: NLP: Language models","Data Mining: DM: Mining graphs"],"videos":[],"references":[{"position":1,"raw":"[Almazrouei et al., 2023] Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, et al. Falcon-40b: an open large language model with state- of-the-art performance. Findings of the Association for Computational Linguistics: ACL, 2023.","title":"Falcon-40b: an open large language model with stateof-the-art performance","authors":"Almazrouei","year":"2023","venue":"Findings of the Association for Computational Linguistics: ACL","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2023] Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. Label-free node classification on graphs with large language models (llms). arXiv preprint arXiv:2310.04668, 2023.","title":"Label-free node classification on graphs with large language models","authors":"Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, Jiliang Tang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chiang et al., 2019] Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proc. of KDD, 2019.","title":"Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks","authors":"Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, Cho-Jui Hsieh","year":"2019","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chien et al., 2021] Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjit S Dhillon. Node feature extraction by self- supervised multi-scale neighborhood prediction. In Proc. of ICLR, 2021.","title":"Node feature extraction by selfsupervised multi-scale neighborhood prediction","authors":"Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, Inderjit S Dhillon","year":"2021","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Duan et al., 2023] Keyu Duan, Qian Liu, Tat-Seng Chua, Shuicheng Yan, Wei Tsang Ooi, Qizhe Xie, and Junxian He. Simteg: A frustratingly simple approach improves textual graph learning. arXiv preprint arXiv:2308.02565, 2023.","title":"Simteg: A frustratingly simple approach improves textual graph learning","authors":"Qian Keyu Duan, Tat-Seng Liu, Shuicheng Chua, Wei Tsang Yan, Qizhe Ooi, Junxian Xie, He","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Giles et al., 1998] C Lee Giles, Kurt D Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing sys- tem. In Proceedings of the third ACM conference on Dig- ital libraries, 1998.","title":"Citeseer: An automatic citation indexing system","authors":"Lee Giles, Kurt D Bollacker, Steve Lawrence","year":"1998","venue":"Proceedings of the third ACM conference on Digital libraries","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gordon et al., 2020] Mitchell A. Gordon, Kevin Duh, and Nicholas Andrews. Compressing BERT: studying the ef- fects of weight pruning on transfer learning. CoRR, 2020.","title":"Compressing BERT: studying the effects of weight pruning on transfer learning","authors":"Mitchell A Gordon, Kevin Duh, Nicholas Andrews","year":"2020","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Proc. of NeurIPS, 2017. [He et al., 2020]","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. In Proc. of ICLR, 2020.","title":"Deberta: Decoding-enhanced bert with disentangled attention","authors":"Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2023a] Haoyu He, Jianfei Cai, Jing Zhang, Dacheng Tao, and Bohan Zhuang. Sensitivity-aware visual parameter-efficient fine-tuning. In Proc. of ICCV, 2023.","title":"Sensitivity-aware visual parameter-efficient fine-tuning","authors":"He","year":"2023","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2023b] Xiaoxin He, Xavier Bresson, Thomas Laurent, and Bryan Hooi. Explanations as features: Llm- based features for text-attributed graphs. arXiv preprint arXiv:2305.19523, 2023.","title":"Explanations as features: Llmbased features for text-attributed graphs","authors":"He","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Houlsby et al., 2019] Neil Houlsby, Andrei Giurgiu, Stanis- law Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In Proc. of ICML, 2019.","title":"Parameter-efficient transfer learning for nlp","authors":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, Sylvain Gelly","year":"2019","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hu et al., 2020] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Proc. of NeurIPS, 2020.","title":"Open graph benchmark: Datasets for machine learning on graphs","authors":"Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, Jure Leskovec","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hu et al., 2021] Edward J Hu, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In Proc. of ICLR, 2021.","title":"Lora: Low-rank adaptation of large language models","authors":"J Edward, Phillip Hu, Zeyuan Wallis, Yuanzhi Allen-Zhu, Shean Li, Lu Wang, Weizhu Wang, Chen","year":"2021","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2023] Boren Hu, Yun Zhu, Jiacheng Li, and Sil- iang Tang. Smartbert: A promotion of dynamic early exit- ing mechanism for accelerating bert inference. In Proc. of IJCAI, 2023.","title":"Smartbert: A promotion of dynamic early exiting mechanism for accelerating bert inference","authors":"Boren Hu, Yun Zhu, Jiacheng Li, Siliang Tang","year":"2023","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2023/563"},{"position":16,"raw":"[Jia et al., 2022] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In Proc. of ECCV, 2022.","title":"Visual prompt tuning","authors":"Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, Ser-Nam Lim","year":"2022","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kenton and Toutanova, 2019] Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proc. of AACL, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton, Lee Kristina, Toutanova","year":"2019","venue":"Proc. of AACL","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In Proc. of ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proc. of EMNLP, 2021.","title":"The power of scale for parameter-efficient prompt tuning","authors":"Brian Lester, Rami Al-Rfou, Noah Constant","year":"2021","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix- tuning: Optimizing continuous prompts for generation. In Proc. of ACL, 2021.","title":"Prefixtuning: Optimizing continuous prompts for generation","authors":"Lisa Xiang, Percy Li, Liang","year":"2021","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2022] Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Proc. of NeurIPS, 2022.","title":"Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning","authors":"Liu","year":"2022","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Mernyei and Cangea, 2020] Péter Mernyei and Cȃtȃlina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks. arXiv preprint arXiv:2007.02901, 2020.","title":"Péter Mernyei and Cȃtȃlina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks","authors":"Cangea Mernyei","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mesquita et al., 2020] Diego Mesquita, Amauri Souza, and Samuel Kaski. Rethinking pooling in graph neural net- works. Proc. of NeurIPS, 2020.","title":"Rethinking pooling in graph neural networks","authors":"Mesquita","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed rep- resentations of words and phrases and their composition- ality. Proc. of NeurIPS, 2013. [OpenAI, 2023] OpenAI. Gpt-4 technical report, 2023.","title":"Distributed representations of words and phrases and their compositionality","authors":"Mikolov","year":"2013","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Pan et al., 2023] Kaihang Pan, Juncheng Li, Hongye Song, Jun Lin, Xiaozhong Liu, and Siliang Tang. Self- supervised meta-prompt learning with meta-gradient reg- ularization for few-shot generalization. arXiv preprint arXiv:2303.12314, 2023.","title":"Selfsupervised meta-prompt learning with meta-gradient regularization for few-shot generalization","authors":"Pan","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Pan et al., 2024a] Kaihang Pan, Juncheng Li, Wenjie Wang, Hao Fei, Hongye Song, Wei Ji, Jun Lin, Xiaozhong Liu, Tat-Seng Chua, and Siliang Tang. I3: Intent-introspective retrieval conditioned on instructions, 2024.","title":"I3: Intent-introspective retrieval conditioned on instructions","authors":"Pan","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Pan et al., 2024b] Shirui Pan, Yizhen Zheng, and Yixin Liu. Integrating graphs with large language models: Methods and prospects. IEEE Intelligent Systems, 39(1):64-68, 2024.","title":"Integrating graphs with large language models: Methods and prospects","authors":"Pan","year":"2024","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Qiu et al., 2020] Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. Gcc: Graph contrastive coding for graph neural network pre-training. In Proc. of KDD, 2020.","title":"Gcc: Graph contrastive coding for graph neural network pre-training","authors":"Qiu","year":"2020","venue":"Proc. of KDD","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Reimers and Gurevych, 2019] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proc. of EMNLP, 2019.","title":"Sentence-bert: Sentence embeddings using siamese bert-networks","authors":"Gurevych Reimers, Nils Reimers, Iryna Gurevych","year":"2019","venue":"Proc. of EMNLP","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sanh et al., 2019] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled ver- sion of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.","title":"Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter","authors":"Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sen et al., 2008] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 2008.","title":"Collective classification in network data","authors":"Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, Tina Eliassi-Rad","year":"2008","venue":"AI magazine","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sung et al., 2022] Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. Lst: Ladder side-tuning for parameter and mem- ory efficient transfer learning. Proc. of NeurIPS, 2022.","title":"Lst: Ladder side-tuning for parameter and memory efficient transfer learning","authors":"Yi-Lin Sung, Jaemin Cho, Mohit Bansal","year":"2022","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Touvron et al., 2023] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.","title":"Llama 2: Open foundation and fine-tuned chat models","authors":"Touvron","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Veličković et al., 2018] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In Proc. of ICLR, 2018.","title":"Graph attention networks","authors":"Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio","year":"2018","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2020] Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Microsoft academic graph: When experts are not enough. Quantitative Science Studies, 2020.","title":"Microsoft academic graph: When experts are not enough","authors":"Wang","year":"2020","venue":"Quantitative Science Studies","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wang et al., 2022] Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Ran- gan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022.","title":"Text embeddings by weakly-supervised contrastive pre-training","authors":"Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wu et al., 2022] Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. Nodeformer: A scalable graph structure learning transformer for node classifica- tion. Proc. of NeurIPS, 2022.","title":"Nodeformer: A scalable graph structure learning transformer for node classification","authors":"Wu","year":"2022","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Xie et al., 2021] Keli Xie, Siyuan Lu, Meiqi Wang, and Zhongfeng Wang. Elbert: Fast albert with confidence- window based early exit. In Proc. of ICASSP, 2021.","title":"Elbert: Fast albert with confidencewindow based early exit","authors":"Keli Xie, Siyuan Lu, Meiqi Wang, Zhongfeng Wang","year":"2021","venue":"Proc. of ICASSP","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Xin et al., 2020] Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, and Jimmy Lin. DeeBERT: Dynamic early exiting for accelerating BERT inference. In Proc. of ACL, 2020.","title":"DeeBERT: Dynamic early exiting for accelerating BERT inference","authors":"Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, Jimmy Lin","year":"2020","venue":"Proc. of ACL","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Xue et al., 2023] Rui Xue, Xipeng Shen, Ruozhou Yu, and Xiaorui Liu. Efficient large language models fine-tuning on graphs. arXiv preprint arXiv:2312.04737, 2023.","title":"Efficient large language models fine-tuning on graphs","authors":"Rui Xue, Xipeng Shen, Ruozhou Yu, Xiaorui Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yan et al., 2023] Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. A comprehensive study on text-attributed graphs: Benchmarking and re- thinking. In Proc. of NeurIPS, 2023.","title":"A comprehensive study on text-attributed graphs: Benchmarking and rethinking","authors":"Chaozhuo Hao Yan, Ruosong Li, Chao Long, Jianan Yan, Wenwen Zhao, Jun Zhuang, Peiyan Yin, Weihao Zhang, Hao Han, Sun","year":"2023","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yang et al., 2021] Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, and Xing Xie. Graphformers: Gnn- nested transformers for representation learning on textual graph. Proc. of NeurIPS, 2021.","title":"Graphformers: Gnnnested transformers for representation learning on textual graph","authors":"Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, Xing Xie","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang et al., 2010] Yin Zhang, Rong Jin, and Zhi-Hua Zhou. Understanding bag-of-words model: a statistical framework. International journal of machine learning and cybernetics, 2010.","title":"Understanding bag-of-words model: a statistical framework","authors":"Yin Zhang, Rong Jin, Zhi-Hua Zhou","year":"2010","venue":"International journal of machine learning and cybernetics","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhao et al., 2022] Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Learning on large-scale text-attributed graphs via varia- tional inference. In Proc. of ICLR, 2022.","title":"Learning on large-scale text-attributed graphs via variational inference","authors":"Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, Jian Tang","year":"2022","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhou et al., 2020] Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei. Bert loses patience: Fast and robust inference with early exit. Proc. of NeurIPS, 2020.","title":"Bert loses patience: Fast and robust inference with early exit","authors":"Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian Mcauley, Ke Xu, Furu Wei","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Zhu et al., 2022] Yun Zhu, Jianhao Guo, Fei Wu, and Sil- iang Tang. Rosa: A robust self-aligned framework for node-node graph contrastive learning. In Proc. of IJCAI, 2022.","title":"Rosa: A robust self-aligned framework for node-node graph contrastive learning","authors":"Yun Zhu, Jianhao Guo, Fei Wu, Siliang Tang","year":"2022","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/527"},{"position":47,"raw":"[Zhu et al., 2023a] Yun Zhu, Jianhao Guo, and Siliang Tang. Sgl-pt: A strong graph learner with graph prompt tuning. arXiv preprint arXiv:2302.12449, 2023.","title":"Sgl-pt: A strong graph learner with graph prompt tuning","authors":"Zhu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zhu et al., 2023b] Yun Zhu, Haizhou Shi, Zhenshuo Zhang, and Siliang Tang. Mario: Model agnostic recipe for im- proving ood generalization of graph contrastive learning. arXiv preprint arXiv:2307.13055, 2023.","title":"Mario: Model agnostic recipe for improving ood generalization of graph contrastive learning","authors":"Zhu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhu et al., 2023c] Yun Zhu, Yaoke Wang, Haizhou Shi, Zhenshuo Zhang, and Siliang Tang. Graphcontrol: Adding conditional control to universal graph pre-trained models for graph domain transfer learning, 2023.","title":"Graphcontrol: Adding conditional control to universal graph pre-trained models for graph domain transfer learning","authors":"Zhu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}