{"type":"ScholarlyArticle","title":"Solving Separable Nonsmooth Problems Using Frank-Wolfe with Uniform Affine Approximations","authors":[{"name":"Edward Cheung","slug":"edward-cheung","given":"Edward","family":"Cheung","affiliation":"School of Computer Science , University of Waterloo , Waterloo , Canada"},{"name":"Yuying Li","slug":"yuying-li","given":"Yuying","family":"Li","affiliation":"School of Computer Science , University of Waterloo , Waterloo , Canada"}],"abstract":"Frank-Wolfe methods (FW) have gained significant interest in the machine learning community due to their ability to efficiently solve large problems that admit a sparse structure (e.g. sparse vectors and low-rank matrices). However the performance of the existing FW method hinges on the quality of the linear approximation. This typically restricts FW to smooth functions for which the approximation quality,  indicated by a  global curvature measure, is reasonably good. \r\nIn this paper, we propose a modified FW algorithm amenable to nonsmooth functions, subject to a separability assumption, by optimizing for approximation quality over all affine functions, given a neighborhood of interest. We analyze theoretical properties of the proposed algorithm and demonstrate that it overcomes many issues associated with existing methods in the context of nonsmooth low-rank matrix estimation.","doi":"10.24963/ijcai.2018/281","url":"https://ijcai.org/proceedings/2018/281","pdf":"https://ijcai.org/proceedings/2018/0281.pdf","markdown":"https://ijcai.org/proceedings/2018/281.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/281","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":281,"pages":{"first":2035,"last":2041},"keywords":["Machine Learning: Feature Selection ; Learning Sparse Models","Machine Learning: Recommender Systems","Machine Learning: Structured Prediction","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Argyriou et al., 2014] Andreas Argyriou, Marco Sig- noretto, and Johan AK Suykens. Hybrid conditional gradient-smoothing algorithms with applications to sparse and low rank regularization. In Regularization, Optimiza- tion, Kernels, and Support Vector Machines, pages 53-82.","title":"Hybrid conditional gradient-smoothing algorithms with applications to sparse and low rank regularization","authors":"Andreas Argyriou, Marco Signoretto, Johan Ak Suykens","year":null,"venue":"Regularization, Optimization, Kernels, and Support Vector Machines","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Chapman and Hall/CRC, 2014.","title":null,"authors":"Hall / Chapman, Crc","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cambier and Absil, 2016] Léopold Cambier and P-A Ab- sil. Robust low-rank matrix completion by riemannian optimization. SIAM Journal on Scientific Computing, 38(5):S440-S460, 2016.","title":"Robust low-rank matrix completion by riemannian optimization","authors":"Absil Cambier","year":"2016","venue":"SIAM Journal on Scientific Computing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheung and Li, 2017] Edward Cheung and Yuying Li. Pro- jection free rank-drop steps. In Proceedings of the Twenty- Sixth International Joint Conference on Artificial Intelli- gence, IJCAI-17, pages 1539-1545, 2017.","title":"Projection free rank-drop steps","authors":"Li ; Cheung, Edward Cheung, Yuying Li","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17","doi":null,"url":null,"internal_url":"/proceedings/2017/213"},{"position":5,"raw":"[Clarkson, 2010] Kenneth L Clarkson. Coresets, sparse greedy approximation, and the frank-wolfe algorithm. ACM Transactions on Algorithms (TALG), 6(4):63, 2010.","title":"Coresets, sparse greedy approximation, and the frank-wolfe algorithm","authors":"Kenneth L Clarkson","year":"2010","venue":"ACM Transactions on Algorithms (TALG)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Davis, 1975] Philip J Davis. Interpolation and approxima- tion. Courier Corporation, 1975.","title":"Interpolation and approximation","authors":"Davis Philip","year":"1975","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Frank and Wolfe, 1956] Marguerite Frank and Philip Wolfe. An algorithm for quadratic programming. Naval Research Logistics (NRL), 3(1-2):95-110, 1956.","title":"An algorithm for quadratic programming","authors":"Wolfe Frank, Marguerite Frank, Philip Wolfe","year":"1956","venue":"Naval Research Logistics (NRL)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hsieh and Olsen, 2014] Cho-Jui Hsieh and Peder Olsen. Nuclear norm minimization via active subspace selection. In International Conference on Machine Learning, pages 575-583, 2014.","title":"Nuclear norm minimization via active subspace selection","authors":"Olsen ; ] Cho-Jui Hsieh, Peder Hsieh, Olsen","year":"2014","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jaggi, 2011] Martin Jaggi. Sparse Convex Optimization Methods for Machine Learning. PhD thesis, ETH Zurich, October 2011.","title":"Sparse Convex Optimization Methods for Machine Learning","authors":"Martin Jaggi","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Leskovec and Krevl, 2014] Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data, June 2014.","title":"SNAP Datasets: Stanford large network dataset collection","authors":"Jure Leskovec, Andrej Krevl","year":"2014","venue":null,"doi":null,"url":"http://snap.stanford.edu/data","internal_url":null},{"position":11,"raw":"[Mayans, 2006] Robert Mayans. The chebyshev equioscil- lation theorem. Journal of Online Mathematics and Its Applications, 6, 2006.","title":"The chebyshev equioscillation theorem","authors":"Robert Mayans","year":"2006","venue":"Journal of Online Mathematics and Its Applications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Odor et al., 2016] Gergely Odor, Yen-Huan Li, Alp Yurt- sever, Ya-Ping Hsieh, Quoc Tran-Dinh, Marwa El Ha- labi, and Volkan Cevher. Frank-wolfe works for non- lipschitz continuous gradient objectives: Scalable poisson phase retrieval. In Acoustics, Speech and Signal Process- ing (ICASSP), 2016 IEEE International Conference on, pages 6230-6234. Ieee, 2016.","title":"Frank-wolfe works for nonlipschitz continuous gradient objectives: Scalable poisson phase retrieval","authors":"Gergely Odor, Yen-Huan Li, Alp Yurtsever, Ya-Ping Hsieh, Quoc Tran-Dinh, Marwa El Halabi, Volkan Cevher","year":"2016","venue":"Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Parikh et al., 2014] Neal Parikh, Stephen Boyd, et al. Prox- imal algorithms. Foundations and Trends R in Optimiza- tion, 1(3):127-239, 2014.","title":"Proximal algorithms","authors":"Neal Parikh, Stephen Boyd","year":"2014","venue":"Foundations and Trends R in Optimization","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Pierucci et al., 2014] Federico Pierucci, Zaid Harchaoui, and Jérôme Malick. A smoothing approach for compos- ite conditional gradient with nonsmooth loss. 2014.","title":"A smoothing approach for composite conditional gradient with nonsmooth loss","authors":"Federico Pierucci, Zaid Harchaoui, Jérôme Malick","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ravi et al., 2017] Sathya N Ravi, Maxwell D Collins, and Vikas Singh. A deterministic nonsmooth frank wolfe algorithm with coreset guarantees. arXiv preprint arXiv:1708.06714, 2017.","title":"A deterministic nonsmooth frank wolfe algorithm with coreset guarantees","authors":"N Sathya, Maxwell D Ravi, Vikas Collins, Singh","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Richard et al., 2012] Emile Richard, Pierre-André Savalle, and Nicolas Vayatis. Estimation of simultaneously sparse and low rank matrices. In Proceedings of the 29th Interna- tional Coference on International Conference on Machine Learning, ICML'12, pages 51-58, USA, 2012. Omnipress.","title":"Estimation of simultaneously sparse and low rank matrices","authors":"Richard","year":"2012","venue":"Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML'12","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[White, 1993] DJ White. Extension of the frank-wolfe algorithm to concave nondifferentiable objective func- tions. Journal of optimization theory and applications, 78(2):283-301, 1993.","title":"Extension of the frank-wolfe algorithm to concave nondifferentiable objective functions","authors":"White","year":"1993","venue":"Journal of optimization theory and applications","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Yao and Kwok, 2016] Quanming Yao and James T Kwok. Greedy learning of generalized low-rank models. In IJ- CAI, pages 2294-2300, 2016.","title":"Greedy learning of generalized low-rank models","authors":"Kwok Yao, Quanming Yao, James T Kwok","year":"2016","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2016/327"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}