{"type":"ScholarlyArticle","title":"Projection Free Rank-Drop Steps","authors":[{"name":"Edward Cheung","slug":"edward-cheung","given":"Edward","family":"Cheung","affiliation":"School of Computer Science , University of Waterloo , Waterloo , Canada"},{"name":"Yuying Li","slug":"yuying-li","given":"Yuying","family":"Li","affiliation":"School of Computer Science , University of Waterloo , Waterloo , Canada"}],"abstract":"The Frank-Wolfe (FW) algorithm has been widely used in solving nuclear norm constrained problems, since it does not require projections. However, FW often yields high rank intermediate iterates, which can be very expensive in time and space costs for large problems. To address this issue, we propose a rank-drop method for nuclear norm constrained problems. The goal is to generate descent steps that lead to rank decreases, maintaining low-rank solutions throughout the algorithm. Moreover, the optimization problems are constrained to ensure that the rank-drop step is also feasible and can be readily incorporated into a projection-free minimization method, e.g., Frank-Wolfe. We demonstrate that by incorporating rank-drop steps into the Frank-Wolfe algorithm, the rank of the solution is greatly reduced compared to the original Frank-Wolfe or its common variants.","doi":"10.24963/ijcai.2017/213","url":"https://ijcai.org/proceedings/2017/213","pdf":"https://ijcai.org/proceedings/2017/0213.pdf","markdown":"https://ijcai.org/proceedings/2017/213.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/213","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":213,"pages":{"first":1539,"last":1545},"keywords":["Machine Learning: Semi-Supervised Learning","Machine Learning: Structured Learning"],"videos":[],"references":[{"position":1,"raw":"[Brand, 2006] Matthew Brand. Fast low-rank modifications of the thin singular value decomposition. Linear algebra and its applications, 415(1):20-30, 2006.","title":"Fast low-rank modifications of the thin singular value decomposition","authors":"Matthew Brand","year":"2006","venue":"Linear algebra and its applications","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Egerváry, 1960] Eugen Egerváry. On rank-diminishing op- erations and their applications to the solution of linear equations. Zeitschrift für angewandte Mathematik und Physik ZAMP, 11(5):376-386, 1960.","title":"On rank-diminishing operations and their applications to the solution of linear equations","authors":"Eugen Egerváry","year":"1960","venue":"Zeitschrift für angewandte Mathematik und Physik ZAMP","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Fazel et al., 2001] Maryam Fazel, Haitham Hindi, and Stephen P Boyd. A rank minimization heuristic with appli- cation to minimum order system approximation. In Amer- ican Control Conference, 2001. Proceedings of the 2001, volume 6, pages 4734-4739. IEEE, 2001.","title":"A rank minimization heuristic with application to minimum order system approximation","authors":"Maryam Fazel, Haitham Hindi, Stephen P Boyd","year":"2001","venue":"American Control Conference","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Frank and Wolfe, 1956] Marguerite Frank and Philip Wolfe. An algorithm for quadratic programming. Naval research logistics quarterly, 3(1-2):95-110, 1956.","title":"An algorithm for quadratic programming","authors":"Wolfe Frank, Marguerite Frank, Philip Wolfe","year":"1956","venue":"Naval research logistics quarterly","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Freund et al., 2015] Robert M Freund, Paul Grigas, and Rahul Mazumder. An extended frank-wolfe method with\" in-face\" directions, and its application to low-rank matrix completion. arXiv preprint arXiv:1511.02204, 2015.","title":"An extended frank-wolfe method with\" in-face\" directions, and its application to low-rank matrix completion","authors":"Paul Robert M Freund, Rahul Grigas, Mazumder","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guélat and Marcotte, 1986] Jacques Guélat and Patrice Marcotte. Some comments on wolfe's away step. Mathe- matical Programming, 35(1):110-119, 1986.","title":"Some comments on wolfe's away step","authors":"Marcotte Guélat, Jacques Guélat, Patrice Marcotte","year":"1986","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hsieh and Olsen, 2014] Cho-Jui Hsieh and Peder A Olsen. Nuclear norm minimization via active subspace selection. In ICML, pages 575-583, 2014.","title":"Nuclear norm minimization via active subspace selection","authors":"Cho-Jui Hsieh, Peder A Olsen","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jaggi, 2013] Martin Jaggi. Revisiting frank-wolfe: Projection-free sparse convex optimization. In ICML (1), pages 427-435, 2013.","title":"Revisiting frank-wolfe: Projection-free sparse convex optimization","authors":"Martin Jaggi, Jaggi","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lacoste-Julien and Jaggi, 2015] Simon Lacoste-Julien and Martin Jaggi. On the global linear convergence of frank- wolfe optimization variants. In Advances in Neural Infor- mation Processing Systems, pages 496-504, 2015.","title":"On the global linear convergence of frankwolfe optimization variants","authors":"Jaggi Lacoste-Julien, Simon Lacoste-Julien, Martin Jaggi","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Wolfe, 1970] Philip Wolfe. Convergence theory in nonlin- ear programming. Integer and nonlinear programming, pages 1-36, 1970.","title":"Convergence theory in nonlinear programming","authors":"Philip Wolfe","year":"1970","venue":"Integer and nonlinear programming","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Yao et al., 2016] Quanming Yao, James Kwok, et al. Ef- ficient learning with a family of nonconvex regular- izers by redistributing nonconvexity. arXiv preprint arXiv:1606.03841, 2016.","title":"Efficient learning with a family of nonconvex regularizers by redistributing nonconvexity","authors":"Yao","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}