{"type":"ScholarlyArticle","title":"Minimizing Adaptive Regret with One Gradient per Iteration","authors":[{"name":"Guanghui Wang","slug":"guanghui-wang","given":"Guanghui","family":"Wang","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"},{"name":"Dakuan Zhao","slug":"dakuan-zhao","given":"Dakuan","family":"Zhao","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"},{"name":"Lijun Zhang","slug":"lijun-zhang","given":"Lijun","family":"Zhang","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , Nanjing 210023 , China"}],"abstract":"To cope with non-stationary environments, recent advances in online optimization have introduced the notion of adaptive regret, which measures the performance of an online learner against different comparators within different time intervals. Previous studies have proposed various algorithms to yield low adaptive regret under different scenarios. However, all of existing algorithms need to query the gradient of the loss function at least O(log t)  times in every iteration t, which hinders their applications to broad domains, especially when the evaluation of gradients is expensive. To address this limitation, we propose a series of computationally efficient algorithms for minimizing the adaptive regret of general convex, strongly convex and  exponentially concave functions respectively. The key idea is to replace each loss function with a carefully designed surrogate loss, which bounds the original loss function from below. We show that the proposed algorithms only query the gradient once per iteration, and attain the same theoretical guarantees as previous optimal algorithms. Empirical results demonstrate the efficiency and effectiveness of our methods.","doi":"10.24963/ijcai.2018/383","url":"https://ijcai.org/proceedings/2018/383","pdf":"https://ijcai.org/proceedings/2018/0383.pdf","markdown":"https://ijcai.org/proceedings/2018/383.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/383","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":383,"pages":{"first":2762,"last":2768},"keywords":["Machine Learning: Online Learning","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Abernethy et al., 2009] Jacob Abernethy, Alekh Agarwal, and Peter L Bartlett. A stochastic view of optimal regret through minimax duality. In Proceedings of the 22nd An- nual Conference on Learning Theory, 2009.","title":"A stochastic view of optimal regret through minimax duality","authors":"Jacob Abernethy, Alekh Agarwal, Peter L Bartlett","year":"2009","venue":"Proceedings of the 22nd Annual Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bach, 2008] Francis R Bach. Consistency of trace norm minimization. Journal of Machine Learning Research, 9:1019-1048, 2008.","title":"Consistency of trace norm minimization","authors":"Francis R Bach","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Blum, 1997] Avrim Blum. Empirical support for winnow and weighted-majority algorithms: Results on a calendar scheduling domain. Machine Learning, 26:5-23, 1997.","title":"Empirical support for winnow and weighted-majority algorithms: Results on a calendar scheduling domain","authors":"Avrim Blum","year":"1997","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boyd and Vandenberghe, 2004] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge univer- sity press, 2004.","title":"Convex optimization","authors":"Vandenberghe Boyd","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cesa-Bianchi and Lugosi, 2006] Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cam- bridge university press, 2006.","title":"Nicolo Cesa-Bianchi and Gábor Lugosi","authors":"Cesa-Bianchi, Lugosi","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chang and Lin, 2011] Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:1-27, 2011.","title":"LIBSVM: A library for support vector machines","authors":"Chang, Lin ; Chih-Chung Chang, Chih-Jen Lin","year":"2011","venue":"ACM Transactions on Intelligent Systems and Technology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Daniely et al., 2015] Amit Daniely, Alon Gonen, and Shai Shalev-Shwartz. Strongly adaptive online learning. In Pro- ceedings of the 32nd International Conference on Machine Learning, pages 1405-1411, 2015.","title":"Strongly adaptive online learning","authors":"Amit Daniely, Alon Gonen, Shai Shalev-Shwartz","year":"2015","venue":"Proceedings of the 32nd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hazan and Seshadhri, 2007] Elad Hazan and C Seshadhri. Adaptive algorithms for online decision problems. In Elec- tronic Colloquium on Computational Complexity, 2007. [Hazan and Seshadhri, 2009] Elad Hazan and Comandur Se- shadhri. Efficient learning algorithms for changing envi- ronments. In Proceedings of the 26th International Con- ference on Machine Learning, pages 393-400, 2009.","title":"Elad Hazan and Comandur Seshadhri. Efficient learning algorithms for changing environments","authors":"Elad Hazan, C Seshadhri","year":"2007","venue":"Proceedings of the 26th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hazan et al., 2007] Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex op- timization. Machine Learning, 69:169-192, 2007.","title":"Logarithmic regret algorithms for online convex optimization","authors":"Elad Hazan, Amit Agarwal, Satyen Kale","year":"2007","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hazan, 2016] Elad Hazan. Introduction to online convex optimization. Foundations and Trends R in Optimization, 2:157-325, 2016.","title":"Introduction to online convex optimization","authors":"Elad Hazan","year":"2016","venue":"Foundations and Trends R in Optimization","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ji and Ye, 2009] Shuiwang Ji and Jieping Ye. An acceler- ated gradient method for trace norm minimization. In Pro- ceedings of the 26th International Conference on Machine Learning, pages 457-464, 2009.","title":"Shuiwang Ji and Jieping Ye. An accelerated gradient method for trace norm minimization","authors":"Ji, Ye","year":"2009","venue":"Proceedings of the 26th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jun et al., 2017] Kwang-Sung Jun, Francesco Orabona, Stephen Wright, and Rebecca Willett. Improved strongly adaptive online learning using coin betting. In Proceed- ings of the 20th International Conference on Artificial In- telligence and Statistics, pages 943-951, 2017.","title":"Improved strongly adaptive online learning using coin betting","authors":"Kwang-Sung Jun, Francesco Orabona, Stephen Wright, Rebecca Willett","year":"2017","venue":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2014] Mu Li, Tong Zhang, Yuqiang Chen, and Alexander J Smola. Efficient mini-batch training for stochastic optimization. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining, pages 661-670, 2014.","title":"Efficient mini-batch training for stochastic optimization","authors":"Mu Li, Tong Zhang, Yuqiang Chen, Alexander J Smola","year":"2014","venue":"Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Orabona and Pál, 2016] Francesco Orabona and Dávid Pál. Coin betting and parameter-free online learning. In Ad- vances in Neural Information Processing Systems 29, pages 577-585, 2016.","title":"Coin betting and parameter-free online learning","authors":"Francesco Orabona, Dávid Pál","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Prokhorov, 2001] Danil Prokhorov. Ijcnn 2001 neural net- work competition. Slide Presentation in IJCNN, 1:97, 2001.","title":"Ijcnn 2001 neural network competition","authors":"Danil Prokhorov","year":"2001","venue":"Slide Presentation in IJCNN","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shalev-Shwartz and Zhang, 2013] Shai Shalev-Shwartz and Tong Zhang. Accelerated mini-batch stochastic dual coor- dinate ascent. In Advances in Neural Information Process- ing Systems 26, pages 378-385, 2013.","title":"Accelerated mini-batch stochastic dual coordinate ascent","authors":"Shalev-Shwartz Zhang, Shai Shalev-Shwartz, Tong Zhang","year":"2013","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shalev-Shwartz, 2012] Shai Shalev-Shwartz. Online learn- ing and online convex optimization. Foundations and Trends R in Machine Learning, 4:107-194, 2012.","title":"Online learning and online convex optimization","authors":"Shai Shalev-Shwartz","year":"2012","venue":"Foundations and Trends R in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[van Erven and Koolen, 2016] Tim van Erven and Wouter M Koolen. Metagrad: Multiple learning rates in online learn- ing. In Advances in Neural Information Processing Sys- tems 29, pages 3666-3674, 2016.","title":"Metagrad: Multiple learning rates in online learning","authors":"Van Erven, ; ] Tim Koolen, Van Erven, Wouter M Koolen","year":"2016","venue":"Advances in Neural Information Processing Systems 29","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Watson, 1992] G Alistair Watson. Characterization of the subdifferential of some matrix norms. Linear Algebra and Its Applications, 170:33-45, 1992.","title":"Characterization of the subdifferential of some matrix norms","authors":"Watson Alistair","year":"1992","venue":"Linear Algebra and Its Applications","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zhang et al., 2017] Lijun Zhang, Tianbao Yang, Rong Jin, and Zhi-Hua Zhou. Dynamic regret of strongly adaptive methods. arXiv preprint arXiv:1701.07570, 2017.","title":"Dynamic regret of strongly adaptive methods","authors":"Lijun Zhang, Tianbao Yang, Rong Jin, Zhi-Hua Zhou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zinkevich, 2003] Martin Zinkevich. Online convex pro- gramming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Ma- chine Learning, pages 928-936, 2003.","title":"Online convex programming and generalized infinitesimal gradient ascent","authors":"Martin Zinkevich","year":"2003","venue":"Proceedings of the 20th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}