{"type":"ScholarlyArticle","title":"Distributing Frank-Wolfe via Map-Reduce","authors":[{"name":"Armin Moharrer","slug":"armin-moharrer","given":"Armin","family":"Moharrer","affiliation":"Northeastern University"},{"name":"Stratis Ioannidis","slug":"stratis-ioannidis","given":"Stratis","family":"Ioannidis","affiliation":"Northeastern University"}],"abstract":"We identify structural properties under which a\r\nconvex optimization over the simplex can be massively\r\nparallelized via map-reduce operations using\r\nthe Frank-Wolfe (FW) algorithm. A broad class\r\nof problems, e.g., Convex Approximation, Experimental\r\nDesigns, and Adaboost, can be tackled this\r\nway. We implement FW over Apache Spark, and\r\nsolve problems with 20 million variables using 350\r\ncores in 79 minutes; the same operation takes 165\r\nhours when executed serially.","doi":"10.24963/ijcai.2018/748","url":"https://ijcai.org/proceedings/2018/748","pdf":"https://ijcai.org/proceedings/2018/0748.pdf","markdown":"https://ijcai.org/proceedings/2018/748.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/748","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Sister Conferences Best Papers","track":"Best Sister Conferences","number":748,"pages":{"first":5334,"last":5338},"keywords":["Machine Learning: Machine Learning","Computer Vision: Big Data and Large Scale Methods","Machine Learning Applications: Big data ; Scalability","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Bellet et al., 2015] Aurélien Bellet, Yingyu Liang, Alireza Bagheri Garakani, Maria-Florina Balcan, and Fei Sha. A Distributed Frank-Wolfe Algorithm for Communication-Efficient Sparse Learning. In SDM, 2015.","title":"A Distributed Frank-Wolfe Algorithm for Communication-Efficient Sparse Learning","authors":"Aurélien Bellet, Yingyu Liang, Alireza Bagheri Garakani, Maria-Florina Balcan, Fei Sha","year":"2015","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bialecki et al., 2005] Andrzej Bialecki, Michael Cafarella, Doug Cutting, and Owen O?malley. Hadoop: a framework for running applications on large clusters built of commod- ity hardware, 2005.","title":"Hadoop: a framework for running applications on large clusters built of commodity hardware","authors":"Andrzej Bialecki, Michael Cafarella, Doug Cutting, Owen O? Malley","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bian et al., 2017] Yatao Bian, Baharan Mirzasoleiman, Joachim M. Buhmann, and Andreas Krause. Guaranteed non-convex optimization: Submodular maximization over continuous domains. In AISTATS, 2017.","title":"Guaranteed non-convex optimization: Submodular maximization over continuous domains","authors":"Bian","year":"2017","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boyd and Vandenberghe, 2004] Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge Univer- sity Press, New York, NY, USA, 2004.","title":"Convex Optimization","authors":"Vandenberghe Boyd","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Calinescu et al., 2011] Gruia Calinescu, Chandra Chekuri, Martin Pál, and Jan Vondrák. Maximizing a monotone submodular function subject to a matroid constraint. SIAM Journal on Computing, 40(6):1740-1766, 2011.","title":"Maximizing a monotone submodular function subject to a matroid constraint","authors":"Gruia Calinescu, Chandra Chekuri, Martin Pál, Jan Vondrák","year":"2011","venue":"SIAM Journal on Computing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Clarkson, 2010] Kenneth L. Clarkson. Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm. ACM Trans. Algorithms, 6(4):63:1-63:30, September 2010.","title":"Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm","authors":"L Kenneth, Clarkson","year":"2010","venue":"ACM Trans. Algorithms","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dean and Ghemawat, 2008] Jeffrey Dean and Sanjay Ghe- mawat. Mapreduce: simplified data processing on large clusters. Communications of the ACM, 51(1):107-113, 2008.","title":"Mapreduce: simplified data processing on large clusters","authors":"Ghemawat Dean","year":"2008","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dudik et al., 2012] Miro Dudik, Zaid Harchaoui, and Jérôme Malick. Lifted coordinate descent for learning with trace-norm regularization. In AISTATS, 2012.","title":"Lifted coordinate descent for learning with trace-norm regularization","authors":"Dudik","year":"2012","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Frank and Wolfe, 1956] Marguerite Frank and Philip Wolfe. An Algorithm for Quadratic Programming. Naval Re- search Logistics Quarterly, 3(1-2):95-110, 1956.","title":"An Algorithm for Quadratic Programming","authors":"Wolfe Frank, Marguerite Frank, Philip Wolfe","year":"1956","venue":"Naval Research Logistics Quarterly","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hazan and Kale, 2012] Elad Hazan and Satyen Kale. Projection-free online learning. In ICML, 2012.","title":"Projection-free online learning","authors":"Kale ; ] Elad Hazan, Satyen Hazan, Kale","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jaggi, 2013] Martin Jaggi. Revisiting Frank-Wolfe: Projection-free sparse convex optimization. In ICML, 2013.","title":"Revisiting Frank-Wolfe: Projection-free sparse convex optimization","authors":"Martin Jaggi","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Joulin et al., 2014] Armand Joulin, Kevin Tang, and Li Fei- Fei. Efficient image and video co-localization with Frank- Wolfe algorithm. In ECVV, 2014.","title":"Efficient image and video co-localization with Frank-Wolfe algorithm","authors":"Armand Joulin, Kevin Tang, Li Fei-Fei","year":"2014","venue":"ECVV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Leighton, 2014] F Thomson Leighton. Introduction to par- allel algorithms and architectures: Trees Hypercubes. El- sevier, 2014.","title":"Introduction to parallel algorithms and architectures: Trees Hypercubes","authors":"Leighton Thomson","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Moharrer and Ioannidis, 2017] Armin Moharrer and Stratis Ioannidis. Distributing frank-wolfe via map-reduce. In ICDM, 2017.","title":"Distributing frank-wolfe via map-reduce","authors":"Ioannidis Moharrer, Armin Moharrer, Stratis Ioannidis","year":"2017","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Reddi et al., 2016] Sashank J Reddi, Suvrit Sra, Barnabás Póczós, and Alex Smola. Stochastic Frank-Wolfe methods for non-convex optimization. In Allerton, 2016.","title":"Stochastic Frank-Wolfe methods for non-convex optimization","authors":"J Sashank, Suvrit Reddi, Barnabás Sra, Alex Póczós, Smola","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Tran et al., 2015] N. L. Tran, T. Peel, and S. Skhiri. Dis- tributed frank-wolfe under pipelined stale synchronous parallelism. In 2015 IEEE International Conference on Big Data (Big Data), 2015.","title":"Distributed frank-wolfe under pipelined stale synchronous parallelism","authors":"N L Tran, T Peel, S Skhiri","year":"2015","venue":"2015 IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ying and Li, 2012] Yiming Ying and Peng Li. Distance metric learning with eigenvalue optimization. Journal of Machine Learning Research, 13(Jan):1-26, 2012.","title":"Distance metric learning with eigenvalue optimization","authors":"Yiming Ying, Peng Li","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Zaharia et al., 2010] Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion Stoica. Spark: cluster computing with working sets. In HotCloud, 2010.","title":"Spark: cluster computing with working sets","authors":"Matei Zaharia, Mosharaf Chowdhury, J Michael, Scott Franklin, Ion Shenker, Stoica","year":"2010","venue":"HotCloud","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}