{"type":"ScholarlyArticle","title":"Bandit Online Learning on Graphs via Adaptive Optimization","authors":[{"name":"Peng Yang","slug":"peng-yang","given":"Peng","family":"Yang","affiliation":"King Abdullah University of Science and Technology , Saudi Arabia"},{"name":"Peilin Zhao","slug":"peilin-zhao","given":"Peilin","family":"Zhao","affiliation":"South China University of Technology , China; Tencent AI Lab , China"},{"name":"Xin Gao","slug":"xin-gao","given":"Xin","family":"Gao","affiliation":"King Abdullah University of Science and Technology , Saudi Arabia"}],"abstract":"Traditional online learning on graphs adapts graph Laplacian into ridge regression, which may not guarantee reasonable accuracy when the data are adversarially generated. To solve this issue, we exploit an adaptive optimization framework for online classification on graphs. The derived model can achieve a min-max regret under an adversarial mechanism of data generation. To take advantage of the informative labels, we propose an adaptive large-margin update rule, which enjoys a lower regret than the algorithms using error-driven update rules. However, this algorithm assumes that the full information label is provided for each node, which is violated in many practical applications where labeling is expensive and the oracle may only tell whether the prediction is correct or not. To address this issue, we propose a bandit online algorithm on graphs. It derives per-instance confidence region of the prediction, from which the model can be learned adaptively to minimize the online regret. Experiments on benchmark graph datasets show that the proposed bandit algorithm outperforms state-of-the-art competitors, even sometimes beats the algorithms using full information label feedback.","doi":"10.24963/ijcai.2018/415","url":"https://ijcai.org/proceedings/2018/415","pdf":"https://ijcai.org/proceedings/2018/0415.pdf","markdown":"https://ijcai.org/proceedings/2018/415.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/415","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":415,"pages":{"first":2991,"last":2997},"keywords":["Uncertainty in AI: Uncertainty in AI","Machine Learning Applications: Networks","Machine Learning: Online Learning"],"videos":[],"references":[{"position":1,"raw":"[Abernethy et al., 2009] Jacob Abernethy, Alekh Agarwal, and Peter L Bartlett. A stochastic view of optimal regret through minimax duality. In COLT, 2009.","title":"A stochastic view of optimal regret through minimax duality","authors":"Jacob Abernethy, Alekh Agarwal, Peter L Bartlett","year":"2009","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Alon et al., 2015] Noga Alon, Nicolo Cesa-Bianchi, Ofer Dekel, and Tomer Koren. Online learning with feedback graphs: Beyond bandits. In COLT, pages 23-35, 2015.","title":"Online learning with feedback graphs: Beyond bandits","authors":"Nicolo Noga Alon, Ofer Cesa-Bianchi, Tomer Dekel, Koren","year":"2015","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Auer, 2002] Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. JMLR, 3(Nov):397- 422, 2002.","title":"Using confidence bounds for exploitation-exploration trade-offs","authors":"Peter Auer","year":"2002","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Azoury and Warmuth, 2001] Katy S Azoury and Manfred K Warmuth. Relative loss bounds for on-line density estima- tion with the exponential family of distributions. Machine Learning, 43(3):211-246, 2001.","title":"Relative loss bounds for on-line density estimation with the exponential family of distributions","authors":"S Katy, Manfred K Azoury, Warmuth","year":"2001","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled exam- ples. JMLR, 7:2399-2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"Mikhail Belkin, Partha Niyogi, Vikas Sindhwani","year":"2006","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Beygelzimer et al., 2017] Alina Beygelzimer, Francesco Orabona, and Chicheng Zhang. Efficient online bandit multiclass learning with Õ( √ T ) regret. In ICML, pages 488-497, 2017.","title":"Efficient online bandit multiclass learning with Õ( √ T ) regret","authors":"Alina Beygelzimer, Francesco Orabona, Chicheng Zhang","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Carpentier and Valko, 2016] Alexandra Carpentier and Michal Valko. Revealing graph bandits for maximizing local influence. In AISTATS, pages 10-18, 2016.","title":"Revealing graph bandits for maximizing local influence","authors":"Valko Carpentier, Alexandra Carpentier, Michal Valko","year":"2016","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Crammer and Gentile, 2013] Koby Crammer and Claudio Gentile. Multiclass classification with bandit feedback us- ing adaptive regularization. Machine learning, 90(3):347- 383, 2013.","title":"Multiclass classification with bandit feedback using adaptive regularization","authors":"Koby Crammer, Claudio Gentile","year":"2013","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dekel et al., 2010] Ofer Dekel, Claudio Gentile, and Karthik Sridharan. Robust selective sampling from single and multiple teachers. In COLT, pages 346-358, 2010.","title":"Robust selective sampling from single and multiple teachers","authors":"Ofer Dekel, Claudio Gentile, Karthik Sridharan","year":"2010","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Eckart and Young, 1936] Carl Eckart and Gale Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211-218, 1936.","title":"The approximation of one matrix by another of lower rank","authors":"Young ; Eckart, Carl Eckart, Gale Young","year":"1936","venue":"Psychometrika","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Forster, 1999] Jürgen Forster. On relative loss bounds in generalized linear regression. In Fundamentals of Com- putation Theory, pages 269-280, 1999.","title":"On relative loss bounds in generalized linear regression","authors":"Jürgen Forster","year":"1999","venue":"Fundamentals of Computation Theory","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gu and Han, 2014] Quanquan Gu and Jiawei Han. Online spectral learning on a graph with bandit feedback. In ICDM-14, pages 833-838. IEEE, 2014.","title":"Online spectral learning on a graph with bandit feedback","authors":"Han ; Gu, Quanquan Gu, Jiawei Han","year":"2014","venue":"ICDM-14","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gu et al., 2013] Quanquan Gu, Charu Aggarwal, Jialu Liu, and Jiawei Han. Selective sampling on graphs for classi- fication. In Proceedings of the 19th ACM SIGKDD, pages 131-139, 2013.","title":"Selective sampling on graphs for classification","authors":"Gu","year":"2013","venue":"Proceedings of the 19th ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Herbster and Pontil, 2006] Mark Herbster and Massimil- iano Pontil. Prediction on a graph with a perceptron. In NIPS-06, pages 577-584, 2006.","title":"Mark Herbster and Massimiliano Pontil. Prediction on a graph with a perceptron","authors":"Pontil Herbster","year":"2006","venue":"NIPS-06","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hoi et al., 2018] Steven C. H. Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. Online learning: A comprehensive sur- vey. CoRR, abs/1802.02871, 2018.","title":"Online learning: A comprehensive survey","authors":"C H Steven, Doyen Hoi, Jing Sahoo, Peilin Lu, Zhao","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kakade et al., 2008] Sham M Kakade, Shai Shalev- Shwartz, and Ambuj Tewari. Efficient bandit algorithms for online multiclass prediction. In ICML, pages 440-447. ACM, 2008.","title":"Efficient bandit algorithms for online multiclass prediction","authors":"Shai Sham M Kakade, Ambuj Shalev-Shwartz, Tewari","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kuznetsov and Mohri, 2016] Vitaly Kuznetsov and Mehryar Mohri. Time series prediction and online learning. In COLT, pages 1190-1213, 2016.","title":"Time series prediction and online learning","authors":"Vitaly Kuznetsov, Mehryar Mohri","year":"2016","venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Moroshko and Crammer, 2014] Edward Moroshko and Koby Crammer. Weighted last-step min-max algorithm with improved sub-logarithmic regret. Theoretical Computer Science, 558:107-124, 2014.","title":"Edward Moroshko and Koby Crammer. Weighted last-step min-max algorithm with improved sub-logarithmic regret","authors":"Crammer Moroshko","year":"2014","venue":"Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Moroshko et al., 2015] Edward Moroshko, Nina Vaits, and Koby Crammer. Second-order non-stationary online learn- ing for regression. JMLR, 16:1481-1517, 2015.","title":"Second-order non-stationary online learning for regression","authors":"Edward Moroshko, Nina Vaits, Koby Crammer","year":"2015","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Smola and Kondor, 2003] Alexander J Smola and Risi Kon- dor. Kernels and regularization on graphs. In Learning the- ory and kernel machines, pages 144-158. Springer, 2003.","title":"Kernels and regularization on graphs","authors":"J Alexander, Risi Smola, Kondor","year":"2003","venue":"Learning theory and kernel machines","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vovk, 1998] Volodya Vovk. Competitive on-line linear re- gression. NIPS-98, pages 364-370, 1998.","title":"Volodya Vovk. Competitive on-line linear regression","authors":"Vovk","year":"1998","venue":"NIPS-98","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2012] Jialei Wang, Peilin Zhao, and Steven C Hoi. Exact soft confidence-weighted learning. In ICML, pages 121-128, 2012.","title":"Exact soft confidence-weighted learning","authors":"Wang","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yang and Zhao, 2015] Peng Yang and Peilin Zhao. A min- max optimization framework for online graph classifica- tion. In CIKM, pages 643-652. ACM, 2015.","title":"A minmax optimization framework for online graph classification","authors":"Peng Yang, Peilin Zhao","year":"2015","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2015] Peng Yang, Peilin Zhao, Vincent W Zheng, and Xiao-Li Li. An aggressive graph-based selec- tive sampling algorithm for classification. In ICDM, pages 509-518. IEEE, 2015.","title":"An aggressive graph-based selective sampling algorithm for classification","authors":"Peng Yang, Peilin Zhao, Vincent W Zheng, Xiao-Li Li","year":"2015","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2016] Peng Yang, Peilin Zhao, Zhen Hai, Wei Liu, Steven CH Hoi, and Xiao-Li Li. Efficient multi-class selective sampling on graphs. In UAI, page 805. AUAI Press, 2016.","title":"Efficient multi-class selective sampling on graphs","authors":"Peng Yang, Peilin Zhao, Zhen Hai, Wei Liu, Steven Ch Hoi, Xiao-Li Li","year":"2016","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2017] Peng Yang, Peilin Zhao, and Xin Gao. Robust online multi-task learning with correlative and per- sonalized structures. IEEE Trans. Knowl. Data Eng., 29(11):2510-2521, 2017.","title":"Robust online multi-task learning with correlative and personalized structures","authors":"Peng Yang, Peilin Zhao, Xin Gao","year":"2017","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2016] Xiaoxuan Zhang, Tianbao Yang, and Padmini Srinivasan. Online asymmetric active learning with imbalanced data. In ACM SIGKDD, pages 2055- 2064. ACM, 2016.","title":"Online asymmetric active learning with imbalanced data","authors":"Zhang","year":"2016","venue":"ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhao et al., 2011] Peilin Zhao, Steven C. H. Hoi, Rong Jin, and Tianbao Yang. Online AUC maximization. In Pro- ceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 -July 2, 2011, pages 233-240, 2011.","title":"Online AUC maximization","authors":"Peilin Zhao, C H Steven, Hoi, Jin Rong, Tianbao Yang","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning, ICML 2011","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}