{"type":"ScholarlyArticle","title":"Fast Second-Order Online Kernel Learning Through Incremental Matrix Sketching and Decomposition","authors":[{"name":"Dongxie Wen","slug":"dongxie-wen","given":"Dongxie","family":"Wen","affiliation":""},{"name":"Xiao Zhang","slug":"xiao-zhang","given":"Xiao","family":"Zhang","affiliation":""},{"name":"Zhewei Wei","slug":"zhewei-wei","given":"Zhewei","family":"Wei","affiliation":""},{"name":"Chenping Hou","slug":"chenping-hou","given":"Chenping","family":"Hou","affiliation":""},{"name":"Shuai Li","slug":"shuai-li","given":"Shuai","family":"Li","affiliation":""},{"name":"Weinan Zhang","slug":"weinan-zhang","given":"Weinan","family":"Zhang","affiliation":""}],"abstract":"Second-order Online Kernel Learning (OKL) has attracted considerable research interest due to its promising predictive performance in streaming environments. However, existing second-order OKL approaches suffer from at least quadratic time complexity with respect to the pre-set budget, rendering them unsuitable for large-scale datasets. Moreover, the singular value decomposition required to obtain explicit feature mapping is computationally expensive due to the complete decomposition process. To address these issues, we propose FORKS, a fast incremental matrix sketching and decomposition approach tailored for second-order OKL. FORKS constructs an incremental maintenance paradigm for second-order kernelized gradient descent, which includes incremental matrix sketching for kernel approximation and incremental matrix decomposition for explicit feature mapping construction. Theoretical analysis demonstrates that FORKS achieves a logarithmic regret guarantee on par with other second-order approaches while maintaining a linear time complexity w.r.t. the budget, significantly enhancing efficiency over existing methods. We validate the performance of our method through extensive experiments conducted on real-world datasets, demonstrating its superior scalability and robustness against adversarial attacks.","doi":"10.24963/ijcai.2025/729","url":"https://ijcai.org/proceedings/2025/729","pdf":"https://ijcai.org/proceedings/2025/0729.pdf","markdown":"https://ijcai.org/proceedings/2025/729.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/729","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":729,"pages":{"first":6552,"last":6560},"keywords":["Machine Learning: ML: Online learning","Machine Learning: ML: Incremental learning","Machine Learning: ML: Kernel methods"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}