{"type":"ScholarlyArticle","title":"Optimization Learning: Perspective, Method, and Applications","authors":[{"name":"Risheng Liu","slug":"risheng-liu","given":"Risheng","family":"Liu","affiliation":"International School of Information Science & Engineering Dalian University of Technology"}],"abstract":"Numerous tasks at the core of statistics, learning, and vision areas are specific cases of ill-posed inverse problems. Recently, learning-based (e.g., deep) iterative methods have been empirically shown to be useful for these problems. Nevertheless, integrating learnable structures into iterations is still a laborious process, which can only be guided by intuitions or empirical insights. Moreover, there is a lack of rigorous analysis of the convergence behaviors of these reimplemented iterations, and thus the significance of such methods is a little bit vague. We move beyond these limits and propose a theoretically guaranteed optimization learning paradigm, a generic and provable paradigm for nonconvex inverse problems, and develop a series of convergent deep models. Our theoretical analysis reveals that the proposed optimization learning paradigm allows us to generate globally convergent trajectories for learning-based iterative methods. Thanks to the superiority of our framework, we achieve state-of-the-art performance on different real applications.","doi":"10.24963/ijcai.2020/728","url":"https://ijcai.org/proceedings/2020/728","pdf":"https://ijcai.org/proceedings/2020/0728.pdf","markdown":"https://ijcai.org/proceedings/2020/728.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/728","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Early Career","track":"Early Career","number":728,"pages":{"first":5164,"last":5168},"keywords":["Computer Vision: Biomedical Image Understanding","Machine Learning: Deep Learning","Computer Vision: Structural and Model-Based Approaches, Knowledge Representation and Reasoning","Constraints and SAT: Constraint Optimization"],"videos":[],"references":[{"position":1,"raw":"[Chen et al., 2017] Yutian Chen, Matthew W. Hoffman, Ser- gio Gomez Colmenarejo, Misha Denil, Timothy P. Lilli- crap, Matthew Botvinick, and Nando de Freitas. Learning to learn without gradient descent by gradient descent. In ICML, pages 748-756, 2017.","title":"Learning to learn without gradient descent by gradient descent","authors":"Yutian Chen, Matthew W Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Timothy P Lillicrap, Matthew Botvinick, Nando De Freitas","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Geof- frey E. Hinton. Deep learning. Nature, 521(7553):436-444, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey E Hinton","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Liu et al., 2018a] Risheng Liu, Shichao Cheng, Yi He, Xin Fan, and Zhongxuan Luo. Toward designing convergent deep operator splitting methods for task-specific nonconvex optimization. In IJCAI, pages 2468-2474, 2018.","title":"Toward designing convergent deep operator splitting methods for task-specific nonconvex optimization","authors":"Liu","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/342"},{"position":5,"raw":"[Liu et al., 2018b] Risheng Liu, Shichao Cheng, Xiaokun Li- u, Long Ma, Xin Fan, and Zhongxuan Luo. A bridging framework for model optimization and deep propagation. In NeurIPS, pages 4323-4332, 2018.","title":"A bridging framework for model optimization and deep propagation","authors":"Liu","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Liu et al., 2018c] Risheng Liu, Xin Fan, Shichao Cheng, X- iangyu Wang, and Zhongxuan Luo. Proximal alternating direction network: A globally converged deep unrolling framework. In AAAI, pages 1371-1378, 2018.","title":"Proximal alternating direction network: A globally converged deep unrolling framework","authors":"Liu","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Liu et al., 2018d] Risheng Liu, Yi He, Shichao Cheng, Xin Fan, and Zhongxuan Luo. Learning collaborative gener- ation correction modules for blind image deblurring and beyond. In ACM Multimedia, pages 1921-1929, 2018.","title":"Learning collaborative generation correction modules for blind image deblurring and beyond","authors":"Liu","year":"2018","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2019a] Risheng Liu, Shichao Cheng, Yi He, Xin Fan, Zhouchen Lin, and Zhongxuan Luo. On the conver- gence of learning-based iterative methods for nonconvex inverse problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.","title":"On the convergence of learning-based iterative methods for nonconvex inverse problems","authors":"Liu","year":"2019","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2019b] Risheng Liu, Shichao Cheng, Long Ma, Xin Fan, and Zhongxuan Luo. Deep proximal unrolling: Algorithmic framework, convergence analysis and ap- plications. IEEE Transactions on Image Processing, 28(10):5013-5026, 2019.","title":"Deep proximal unrolling: Algorithmic framework, convergence analysis and applications","authors":"Liu","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2019c] Risheng Liu, Long Ma, Yiyang Wang, and Lei Zhang. Learning converged propagations with deep prior ensemble for image enhancement. IEEE Transactions on Image Processing, 28(3):1528-1543, 2019.","title":"Learning converged propagations with deep prior ensemble for image enhancement","authors":"Liu","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2019d] Risheng Liu, Yuxi Zhang, Shichao Cheng, Xin Fan, and Zhongxuan Luo. A theoretically guaranteed deep optimization framework for robust compressive sens- ing MRI. In AAAI, pages 4368-4375, 2019.","title":"A theoretically guaranteed deep optimization framework for robust compressive sensing MRI","authors":"Liu","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}