{"type":"ScholarlyArticle","title":"Accelerated Difference of Convex functions Algorithm and its Application to Sparse Binary Logistic Regression","authors":[{"name":"Duy Nhat Phan","slug":"duy-nhat-phan","given":"Duy Nhat","family":"Phan","affiliation":"Department Computer Science and Application , LGIPM , University of Lorraine"},{"name":"Hoai Minh Le","slug":"hoai-minh-le","given":"Hoai Minh","family":"Le","affiliation":"Department Computer Science and Application , LGIPM , University of Lorraine"},{"name":"Hoai An Le Thi","slug":"hoai-an-le-thi","given":"Hoai An","family":"Le Thi","affiliation":"Department Computer Science and Application , LGIPM , University of Lorraine"}],"abstract":"In this work, we present a variant of DCA (Difference of Convex function Algorithm) with the aim to improve its convergence speed. The proposed algorithm, named Accelerated DCA (ADCA), consists in incorporating the Nesterov's acceleration technique into DCA. We first investigate ADCA for solving the standard DC program and rigorously study its convergence properties and the convergence rate. Secondly, we develop ADCA for a special case of the standard DC program whose the objective function is the sum of a differentiable with L-Lipschitz gradient function (possibly nonconvex) and a nonsmooth DC function. We exploit the special structure of the problem to propose an efficient DC decomposition for which the corresponding ADCA scheme is inexpensive. As an application, we consider the sparse binary logistic regression problem. Numerical experiments on several benchmark datasets illustrate the efficiency of our algorithm and its superiority over well-known methods.","doi":"10.24963/ijcai.2018/190","url":"https://ijcai.org/proceedings/2018/190","pdf":"https://ijcai.org/proceedings/2018/0190.pdf","markdown":"https://ijcai.org/proceedings/2018/190.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/190","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Constraints and SAT","track":"Main track","number":190,"pages":{"first":1369,"last":1375},"keywords":["Constraints and SAT: Constraint Optimisation","Machine Learning: Classification","Machine Learning: Feature Selection ; Learning Sparse Models"],"videos":[],"references":[{"position":1,"raw":"[Attouch and Bolte, 2009] Hedy Attouch and Jerome Bolte. On the convergence of the proximal algorithm for nons- mooth functions involving analytic features. Mathematical Programming, 116(1):5-16, 2009.","title":"Hedy Attouch and Jerome Bolte. On the convergence of the proximal algorithm for nonsmooth functions involving analytic features","authors":"Bolte Attouch","year":"2009","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Beck and Teboulle, 2009a] Amir Beck and Marc Teboulle. Fast gradient-based algorithms for constrained total vari- ation image denoising and deblurring problems. IEEE Transactions on Image Processing, 18(11):2419-2434, 2009.","title":"Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems","authors":"Teboulle ; Beck, Amir Beck, Marc Teboulle","year":"2009","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beck and Teboulle, 2009b] Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for lin- ear inverse problems. SIAM Journal on Imaging Sciences, 2:183-202, 2009.","title":"A fast iterative shrinkage-thresholding algorithm for linear inverse problems","authors":"Teboulle ; Beck, Amir Beck, Marc Teboulle","year":"2009","venue":"SIAM Journal on Imaging Sciences","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bradley and Mangasarian, 1998] Paul Bradley and Olvi Leon Mangasarian. Feature selection via concave minimization and support vector machines. In Machine Learning Proceedings of the Fifteenth International Conference (ICML '98), pages 82-90, 1998.","title":"Feature selection via concave minimization and support vector machines","authors":"Mangasarian Bradley, Paul Bradley, Olvi Leon Mangasarian","year":"1998","venue":"Machine Learning Proceedings of the Fifteenth International Conference (ICML '98)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chambolle et al., 1998] Antonin Chambolle, Ronald De- Vore, Nam-Yong Lee, and Bradley Lucier. Nonlinear wavelet image processing: variational problems, compres- sion, and noise removal through wavelet shrinkage. IEEE Transactions on Image Processing, 7(3):319-335, 1998.","title":"Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage","authors":"Antonin Chambolle, Ronald De-Vore, Nam-Yong Lee, Bradley Lucier","year":"1998","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Combettes and Wajs, 2005] Patrick Combettes and Valerie Wajs. Signal recovery by proximal forward-backward splitting. Multiscale Modeling & Simulation, 4(4):1168- 1200, 2005.","title":"Signal recovery by proximal forward-backward splitting","authors":"Wajs Combettes, Patrick Combettes, Valerie Wajs","year":"2005","venue":"Multiscale Modeling & Simulation","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Daubechies et al., 2004] Ingrid Daubechies, Michel De- frise, and Christine De Mol. An iterative thresholding al- gorithm for linear inverse problems with a sparsity con- straint. Communications on Pure and Applied Mathemat- ics, 57(11):1413-1457, 2004.","title":"An iterative thresholding algorithm for linear inverse problems with a sparsity constraint","authors":"Ingrid Daubechies, Michel Defrise, Christine De Mol","year":"2004","venue":"Communications on Pure and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dempster et al., 1977] Arthur Dempster, Natalie Laird, and Donald Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical So- ciety, Serie B, 39(1):1-38, 1977.","title":"Maximum likelihood from incomplete data via the em algorithm","authors":"Dempster","year":"1977","venue":"Journal of the Royal Statistical Society, Serie B","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Frankel et al., 2015] Pierre Frankel, Guillaume Garrigos, and Peypouquet Peypouquet. Splitting methods with vari- able metric for kurdyka-łojasiewicz functions and general convergence rates. Journal of Optimization Theory and Applications, 165(3):874-900, 2015.","title":"Splitting methods with variable metric for kurdyka-łojasiewicz functions and general convergence rates","authors":"Frankel","year":"2015","venue":"Journal of Optimization Theory and Applications","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grippo and Sciandrone, 2002] Luigi Grippo and Marco Sciandrone. Nonmonotone globalization techniques for the barzilai-borwein gradient method. Computational Op- timization and Applications, 23(2):143-169, 2002.","title":"Nonmonotone globalization techniques for the barzilai-borwein gradient method","authors":"Sciandrone Grippo, Luigi Grippo, Marco Sciandrone","year":"2002","venue":"Computational Optimization and Applications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gu et al., 2018] Bin Gu, Zhouyuan Huo, and Heng Huang. Inexact proximal gradient methods for non-convexand non-smooth optimization. In 32th AAAI Conference on Artificial Intelligence AAAI-18, 2018.","title":"Inexact proximal gradient methods for non-convexand non-smooth optimization","authors":"Gu","year":"2018","venue":"32th AAAI Conference on Artificial Intelligence AAAI-18","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hale et al., 2008] Elaine Hale, Wotao Yin, and Yin Zhang. Fixed-point continuation for ℓ 1 -minimization: Methodol- ogy and convergence. SIAM Journal on Optimization, 19(3):1107-1130, 2008.","title":"Fixed-point continuation for ℓ 1 -minimization: Methodology and convergence","authors":"Elaine Hale, Wotao Yin, Yin Zhang","year":"2008","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Le Thi and Pham Dinh, 2005] Hoai An Le Thi and Tao Pham Dinh. The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Annals of Oper- ations Research, 133:23-46, 2005.","title":"Hoai An Le Thi and Tao Pham Dinh. The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems","authors":"Le Thi, Pham Dinh","year":"2005","venue":"Annals of Operations Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Le Thi and Pham Dinh, 2018] Hoai An Le Thi and Tao Pham Dinh. DC programming and DCA: thirty years of developments. Mathematical Programming, Special Is- sue: DC Programming -Theory, Algorithms and Appli- cations, 169(1):5-64, 2018.","title":"Hoai An Le Thi and Tao Pham Dinh. DC programming and DCA: thirty years of developments","authors":"Le Thi, Pham Dinh","year":"2018","venue":"Mathematical Programming, Special Issue: DC Programming -Theory, Algorithms and Applications","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Le Thi et al., 2015] Hoai An Le Thi, Tao Pham Dinh, Hoai Minh Le, and Xuan Thanh Vo. DC approximation approaches for sparse optimization. European Journal of Operational Research, 244(1):26-46, 2015.","title":"DC approximation approaches for sparse optimization","authors":"Le Hoai An, Tao Thi, Hoai Pham Dinh, Minh Le, Xuan Thanh Vo","year":"2015","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li and Lin, 2015] Huan Li and Zhouchen Lin. Accelerated proximal gradient methods for nonconvex programming. In Advances in Neural Information Processing Systems, pages 377-387, 2015.","title":"Accelerated proximal gradient methods for nonconvex programming","authors":"Lin ; ] Huan Li, Zhouchen Li, Lin","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nesterov, 1983] Yurii Nesterov. A method of solving a convex programming problem with convergence rate O(1/k 2 ). Soviet Mathematics Doklady, 27:372-376, 1983.","title":"A method of solving a convex programming problem with convergence rate O(1/k 2 )","authors":"Yurii Nesterov","year":"1983","venue":"Soviet Mathematics Doklady","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pham Dinh and Le Thi, 1997] Tao Pham Dinh and Hoai An Le Thi. Convex analysis approach to D.C. programming: Theory, algorithms and applications. Acta Mathematica Vietnamica, 22(1):289-355, 1997.","title":"Tao Pham Dinh and Hoai An Le Thi. Convex analysis approach to D.C. programming: Theory, algorithms and applications","authors":"Pham Dinh, Le Thi","year":"1997","venue":"Acta Mathematica Vietnamica","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pham Dinh and Le Thi, 2014] T. Pham Dinh and H. A. Le Thi. Recent advances in dc programming and dca. In Transactions on Computational Intelligence XIII, pages 1- 37, 2014.","title":"Recent advances in dc programming and dca","authors":"Pham Dinh, Le Thi, ; T Pham Dinh, H A Le, Thi","year":"2014","venue":"Transactions on Computational Intelligence XIII","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wright et al., 2009] Stephen Wright, Robert Nowak, and Mario Figueiredo. Sparse reconstruction by separable ap- proximation. IEEE Transactions on Signal Processing, 57(7):2479-2493, 2009.","title":"Sparse reconstruction by separable approximation","authors":"Stephen Wright, Robert Nowak, Mario Figueiredo","year":"2009","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yao et al., 2017] Quanming Yao, James. Kwok, Fei Gao, Wei Chen, and Tie-Yan Liu. Efficient inexact proximal gradient algorithm for nonconvex problems. In Proceed- ings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pages 3308-3314, 2017.","title":"Efficient inexact proximal gradient algorithm for nonconvex problems","authors":"Quanming Yao, James Kwok, Fei Gao, Wei Chen, Tie-Yan Liu","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/462"},{"position":22,"raw":"[Yuille and Rangarajan, 2003] A. L. Yuille and A. Rangara- jan. The concave-convex procedure. Neural Computation, 15(4):915-936, 2003.","title":"The concave-convex procedure","authors":"A L Yuille, A Rangarajan","year":"2003","venue":"Neural Computation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}