{"type":"ScholarlyArticle","title":"SVD-AE: Simple Autoencoders for Collaborative Filtering","authors":[{"name":"Seoyoung Hong","slug":"seoyoung-hong","given":"Seoyoung","family":"Hong","affiliation":"Boeing Korea Engineering and Technology Center (BKETC)"},{"name":"Jeongwhan Choi","slug":"jeongwhan-choi","given":"Jeongwhan","family":"Choi","affiliation":"Yonsei University"},{"name":"Yeon-Chang Lee","slug":"yeon-chang-lee","given":"Yeon-Chang","family":"Lee","affiliation":"Ulsan National Institute of Science and Technology (UNIST)"},{"name":"Srijan Kumar","slug":"srijan-kumar","given":"Srijan","family":"Kumar","affiliation":"Georgia Institute of Technology"},{"name":"Noseong Park","slug":"noseong-park","given":"Noseong","family":"Park","affiliation":"Korea Advanced Institute of Science and Technology (KAIST)"}],"abstract":"Collaborative filtering (CF) methods for recommendation systems have been extensively researched, ranging from matrix factorization and autoencoder-based to graph filtering-based methods. Recently, lightweight methods that require almost no training have been recently proposed to reduce overall computation. However, existing methods still have room to improve the trade-offs among accuracy, efficiency, and robustness. In particular, there are no well-designed closed-form studies for balanced CF in terms of the aforementioned trade-offs. In this paper, we design SVD-AE, a simple yet effective singular vector decomposition (SVD)-based linear autoencoder, whose closed-form solution can be defined based on SVD for CF. SVD-AE does not require iterative training processes as its closed-form solution can be calculated at once. Furthermore, given the noisy nature of the rating matrix, we explore the robustness against such noisy interactions of existing CF methods and our SVD-AE. As a result, we demonstrate that our simple design choice based on truncated SVD can be used to strengthen the noise robustness of the recommendation while improving efficiency. Code is available at https://github.com/seoyoungh/svd-ae.","doi":"10.24963/ijcai.2024/227","url":"https://ijcai.org/proceedings/2024/227","pdf":"https://ijcai.org/proceedings/2024/0227.pdf","markdown":"https://ijcai.org/proceedings/2024/227.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/227","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":227,"pages":{"first":2054,"last":2062},"keywords":["Multidisciplinary Topics and Applications: MTA: Web and social networks","Data Mining: DM: Information retrieval","Data Mining: DM: Collaborative filtering"],"videos":[],"references":[{"position":1,"raw":"[Amatriain et al., 2009a] Xavier Amatriain, Josep M Pujol, and Nuria Oliver. I like it... i like it not: Evaluating user ratings noise in recommender systems. In International Conference on User Modeling, Adaptation, and Personal- ization, 2009.","title":"i like it not: Evaluating user ratings noise in recommender systems","authors":"Amatriain","year":"2009","venue":"International Conference on User Modeling, Adaptation, and Personalization","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amatriain et al., 2009b] Xavier Amatriain, Josep M Pujol, Nava Tintarev, and Nuria Oliver. Rate it again: increasing recommendation accuracy by user re-rating. In RecSys, 2009.","title":"Rate it again: increasing recommendation accuracy by user re-rating","authors":"Amatriain","year":"2009","venue":"RecSys","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cho et al., 2011] Eunjoon Cho, Seth A Myers, and Jure Leskovec. Friendship and mobility: user movement in location-based social networks. In KDD, 2011.","title":"Friendship and mobility: user movement in location-based social networks","authors":"Cho","year":"2011","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Choi and Ryu, 2023] Jeongwhan Choi and Duksan Ryu. Qos-aware graph contrastive learning for web service rec- ommendation. In 2023 30th Asia-Pacific Software En- gineering Conference (APSEC), pages 171-180. IEEE, 2023.","title":"Qos-aware graph contrastive learning for web service recommendation","authors":"Ryu Choi, Jeongwhan Choi, Duksan Ryu","year":"2023","venue":"2023 30th Asia-Pacific Software Engineering Conference (APSEC)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Choi et al., 2021] Jeongwhan Choi, Jinsung Jeon, and Noseong Park. LT-OCF: Learnable-time ode-based col- laborative filtering. In CIKM, 2021.","title":"LT-OCF: Learnable-time ode-based collaborative filtering","authors":"Choi","year":"2021","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi et al., 2023a] Jeongwhan Choi, Seoyoung Hong, Noseong Park, and Sung-Bae Cho. Blurring-sharpening process models for collaborative filtering. In SIGIR, 2023.","title":"Blurring-sharpening process models for collaborative filtering","authors":"Choi","year":"2023","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Choi et al., 2023b] Jeongwhan Choi, Hyowon Wi, Chae- jeong Lee, Sung-Bae Cho, Dongha Lee, and Noseong Park. Rdgcl: Reaction-diffusion graph contrastive learning for recommendation. arXiv preprint arXiv:2312.16563, 2023.","title":"Rdgcl: Reaction-diffusion graph contrastive learning for recommendation","authors":"Choi","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Feng et al., 2018] Xu Feng, Wenjian Yu, and Yaohang Li. Faster matrix completion using randomized svd. In 2018 IEEE 30th International conference on tools with artificial intelligence (ICTAI), pages 608-615. IEEE, 2018.","title":"Faster matrix completion using randomized svd","authors":"Wenjian Xu Feng, Yaohang Yu, Li","year":"2018","venue":"IEEE 30th International conference on tools with artificial intelligence (ICTAI)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Halko et al., 2011] Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate ma- trix decompositions. SIAM review, 53(2):217-288, 2011.","title":"Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions","authors":"Halko","year":"2011","venue":"SIAM review","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Harper and Konstan, 2015] F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm Transactions on Interactive Intelligent Systems, 2015.","title":"The movielens datasets: History and context","authors":"Maxwell Harper, Joseph A Konstan","year":"2015","venue":"Acm Transactions on Interactive Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[He et al., 2017] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collabo- rative filtering. In TheWebConf, 2017. [He et al., 2020] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. LightGCN: Simplifying and powering graph convolution network for recommendation. In SIGIR, 2020.","title":"LightGCN: Simplifying and powering graph convolution network for recommendation","authors":"He","year":"2017","venue":"TheWebConf","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hong et al., 2022] Seoyoung Hong, Minju Jo, Seungji Kook, Jaeeun Jung, Hyowon Wi, Noseong Park, and Sung- Bae Cho. Timekit: A time-series forecasting-based up- grade kit for collaborative filtering. In IEEE International Conference on Big Data (Big Data), 2022.","title":"Timekit: A time-series forecasting-based upgrade kit for collaborative filtering","authors":"Hong","year":"2022","venue":"IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hong et al., 2024] Seoyoung Hong, Jeongwhan Choi, Yeon-Chang Lee, Srijan Kumar, and Noseong Park. SVD-AE: Simple autoencoders for collaborative filtering. arXiv preprint arXiv:2405.04746, 2024.","title":"SVD-AE: Simple autoencoders for collaborative filtering","authors":"Hong","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jain et al., 2016] Himanshu Jain, Yashoteja Prabhu, and Manik Varma. Extreme multi-label loss functions for rec- ommendation, tagging, ranking & other missing label ap- plications. In KDD, 2016.","title":"Extreme multi-label loss functions for recommendation, tagging, ranking & other missing label applications","authors":"Himanshu Jain, Yashoteja Prabhu, Manik Varma","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kim et al., 2023] Min-Jeong Kim, Yeon-Chang Lee, and Sang-Wook Kim. Trustsgcn: Learning trustworthiness on edge signs for effective signed graph convolutional net- works. In SIGIR, 2023.","title":"Trustsgcn: Learning trustworthiness on edge signs for effective signed graph convolutional networks","authors":"Kim","year":"2023","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kipf and Welling, 2017] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"Thomas N Kipf, Max Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kong et al., 2022] Taeyong Kong, Taeri Kim, Jinsung Jeon, Jeongwhan Choi, Yeon-Chang Lee, Noseong Park, and Sang-Wook Kim. Linear, or non-linear, that is the ques- tion! In WSDM, pages 517-525, 2022.","title":"Linear, or non-linear, that is the question! In WSDM","authors":"Kong","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lee et al., 2021] Dongha Lee, SeongKu Kang, Hyunjun Ju, Chanyoung Park, and Hwanjo Yu. Bootstrapping user and item representations for one-class collaborative filtering. In SIGIR, 2021.","title":"Bootstrapping user and item representations for one-class collaborative filtering","authors":"Lee","year":"2021","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lee et al., 2024] Chaejeong Lee, Jeongwhan Choi, Hyowon Wi, Sung-Bae Cho, and Noseong Park. Stochastic sam- pling for contrastive views and hard negative samples in graph-based collaborative filtering. arXiv preprint arXiv:2405.00287, 2024.","title":"Stochastic sampling for contrastive views and hard negative samples in graph-based collaborative filtering","authors":"Lee","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liang et al., 2018] Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. Variational autoen- coders for collaborative filtering. In TheWebConf, 2018.","title":"Variational autoencoders for collaborative filtering","authors":"Dawen Liang, G Rahul, Matthew D Krishnan, Tony Hoffman, Jebara","year":"2018","venue":"TheWebConf","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mao et al., 2021] Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, and Xiuqiang He. UltraGCN: ultra simplification of graph convolutional networks for recom- mendation. In CIKM, 2021.","title":"UltraGCN: ultra simplification of graph convolutional networks for recommendation","authors":"Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, Xiuqiang He","year":"2021","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Moon et al., 2023] Jaewan Moon, Hye-young Kim, and Jongwuk Lee. It's enough: Relaxing diagonal constraints in linear autoencoders for recommendation. In SIGIR, 2023.","title":"Jongwuk Lee. It's enough: Relaxing diagonal constraints in linear autoencoders for recommendation","authors":"Moon","year":"2023","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Murakami and Aibara, 1987] Kenji Murakami and Tsune- hiro Aibara. An improvement on the moore-penrose gen- eralized inverse associative memory. IEEE transactions on systems, man, and cybernetics, 17(4):699-707, 1987.","title":"An improvement on the moore-penrose generalized inverse associative memory","authors":"Kenji Murakami, Tsunehiro Aibara","year":"1987","venue":"IEEE transactions on systems, man, and cybernetics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ortega et al., 2018] Antonio Ortega, Pascal Frossard, Jelena Kovačević, José MF Moura, and Pierre Vandergheynst. Graph signal processing: Overview, challenges, and appli- cations. Proceedings of the IEEE, 106(5):808-828, 2018.","title":"Graph signal processing: Overview, challenges, and applications","authors":"Antonio Ortega, Pascal Frossard, Jelena Kovačević, M F José, Pierre Moura, Vandergheynst","year":"2018","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Park et al., 2024] Jin-Duk Park, Yong-Min Shin, and Won- Yong Shin. Turbo-cf: Matrix decomposition-free graph filtering for fast recommendation. arXiv preprint arXiv:2404.14243, 2024.","title":"Turbo-cf: Matrix decomposition-free graph filtering for fast recommendation","authors":"Jin-Duk Park, Yong-Min Shin, Won-Yong Shin","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Peng et al., 2022] Shaowen Peng, Kazunari Sugiyama, and Tsunenori Mine. Svd-gcn: A simplified graph convolution paradigm for recommendation. In CIKM, 2022.","title":"Svd-gcn: A simplified graph convolution paradigm for recommendation","authors":"Shaowen Peng, Kazunari Sugiyama, Tsunenori Mine","year":"2022","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rendle et al., 2012] Steffen Rendle, Christoph Freuden- thaler, Zeno Gantner, and Lars Schmidt-Thieme. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618, 2012.","title":"BPR: Bayesian personalized ranking from implicit feedback","authors":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sachdeva et al., 2022] Noveen Sachdeva, Mehak Preet Dhaliwal, Carole-Jean Wu, and Julian McAuley. Infinite recommendation networks: A data-centric approach. In NeurIPS, 2022.","title":"Infinite recommendation networks: A data-centric approach","authors":"Sachdeva","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Said et al., 2012] Alan Said, Brijnesh J Jain, Sascha Narr, and Till Plumbaum. Users and noise: The magic barrier of recommender systems. In International Conference on User Modeling, Adaptation, and Personalization, 2012.","title":"Users and noise: The magic barrier of recommender systems","authors":"Alan Said, J Brijnesh, Sascha Jain, Till Narr, Plumbaum","year":"2012","venue":"International Conference on User Modeling, Adaptation, and Personalization","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sar Shalom et al., 2015] Oren Sar Shalom, Shlomo Berkovsky, Royi Ronen, Elad Ziklik, and Amir Amihood. Data quality matters in recommender systems. In RecSys, 2015.","title":"Data quality matters in recommender systems","authors":"Oren Sar, Shalom, Shlomo Berkovsky, Royi Ronen, Elad Ziklik, Amir Amihood","year":"2015","venue":"RecSys","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sharma et al., 2022] Kartik Sharma, Yeon-Chang Lee, Sivagami Nambi, Aditya Salian, Shlok Shah, Sang-Wook Kim, and Srijan Kumar. A survey of graph neural net- works for social recommender systems. arXiv preprint arXiv:2212.04481, 2022.","title":"A survey of graph neural networks for social recommender systems","authors":"Kartik Sharma, Yeon-Chang Lee, Sivagami Nambi, Aditya Salian, Shlok Shah, Sang-Wook Kim, Srijan Kumar","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Shen et al., 2021] Yifei Shen, Yongji Wu, Yao Zhang, Cai- hua Shan, Jun Zhang, B Khaled Letaief, and Dongsheng Li. How powerful is graph convolution for recommenda- tion? In CIKM, 2021.","title":"How powerful is graph convolution for recommendation?","authors":"Yifei Shen, Yongji Wu, Yao Zhang, Caihua Shan, Jun Zhang, B Khaled Letaief, Dongsheng Li","year":"2021","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Shin et al., 2024] Yehjin Shin, Jeongwhan Choi, Hyowon Wi, and Noseong Park. An attentive inductive bias for sequential recommendation beyond the self-attention. In AAAI, 2024.","title":"An attentive inductive bias for sequential recommendation beyond the self-attention","authors":"Shin","year":"2024","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Shishkin et al., 2019] Serge L Shishkin, Arkadi Shalaginov, and Shaunak D Bopardikar. Fast approximate truncated svd. Numerical Linear Algebra with Applications, 2019.","title":"Fast approximate truncated svd","authors":"Arkadi Serge L Shishkin, Shaunak D Shalaginov, Bopardikar","year":"2019","venue":"Numerical Linear Algebra with Applications","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Steck, 2019] Harald Steck. Embarrassingly shallow autoen- coders for sparse data. In TheWebConf, 2019.","title":"Embarrassingly shallow autoencoders for sparse data","authors":"Harald Steck, Steck","year":"2019","venue":"TheWebConf","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Steck, 2020] Harald Steck. Autoencoders that don't overfit towards the identity. In NeurIPS, 2020.","title":"Autoencoders that don't overfit towards the identity","authors":"Harald Steck, Steck","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Stiles and Denq, 1985] Gardiner S Stiles and Dong-Lih Denq. On the effect of noise on the moore-penrose gener- alized inverse associative memory. IEEE transactions on pattern analysis and machine intelligence, 1985.","title":"On the effect of noise on the moore-penrose generalized inverse associative memory","authors":"S Gardiner, Dong-Lih Stiles, Denq","year":"1985","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2019] Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph col- laborative filtering. In SIGIR, 2019.","title":"Neural graph collaborative filtering","authors":"Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, Tat-Seng Chua","year":"2019","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Ying et al., 2018] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web- scale recommender systems. In KDD, 2018.","title":"Graph convolutional neural networks for webscale recommender systems","authors":"Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, Jure Leskovec","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}