{"type":"ScholarlyArticle","title":"NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm (Extended Abstract)","authors":[{"name":"Zhichao Lu","slug":"zhichao-lu","given":"Zhichao","family":"Lu","affiliation":"Michigan State University"},{"name":"Ian Whalen","slug":"ian-whalen","given":"Ian","family":"Whalen","affiliation":"Michigan State University"},{"name":"Yashesh Dhebar","slug":"yashesh-dhebar","given":"Yashesh","family":"Dhebar","affiliation":"Michigan State University"},{"name":"Kalyanmoy Deb","slug":"kalyanmoy-deb","given":"Kalyanmoy","family":"Deb","affiliation":"Michigan State University"},{"name":"Erik Goodman","slug":"erik-goodman","given":"Erik","family":"Goodman","affiliation":"Michigan State University"},{"name":"Wolfgang Banzhaf","slug":"wolfgang-banzhaf","given":"Wolfgang","family":"Banzhaf","affiliation":"Michigan State University"},{"name":"Vishnu Naresh Boddeti","slug":"vishnu-naresh-boddeti","given":"Vishnu Naresh","family":"Boddeti","affiliation":"Michigan State University"}],"abstract":"Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. This paper introduces NSGA-Net -- an evolutionary search algorithm that explores a space of potential neural network architectures in three steps, namely, a population initialization step that is based on prior-knowledge from hand-crafted architectures, an exploration step comprising crossover and mutation of architectures, and finally an exploitation step that utilizes the hidden useful knowledge stored in the entire history of evaluated neural architectures in the form of a Bayesian Network. The integration of these components allows an efficient design of architectures that are competitive and in many cases outperform both manually and automatically designed architectures on CIFAR-10 classification task. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.","doi":"10.24963/ijcai.2020/659","url":"https://ijcai.org/proceedings/2020/659","pdf":"https://ijcai.org/proceedings/2020/0659.pdf","markdown":"https://ijcai.org/proceedings/2020/659.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/659","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Sister Conferences Best Papers","track":"Sister Conferences Best Papers","number":659,"pages":{"first":4750,"last":4754},"keywords":["Machine Learning: Deep Learning: Convolutional networks","Machine Learning: Classification"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25520/file_25520.m3u8","length_sec":246,"provider":"hls"}],"references":[{"position":1,"raw":"[Deb et al., 2002] Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. A fast and elitist mul- tiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation, 6(2):182-197, 2002.","title":"A fast and elitist multiobjective genetic algorithm: Nsga-ii","authors":"Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, Meyarivan","year":"2002","venue":"IEEE Transactions on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Holland, 1975] J. H. Holland. Adaptation in Natural and Artificial Systems. Ann Arbor, MI: MIT Press, 1975.","title":"Adaptation in Natural and Artificial Systems","authors":"J H Holland","year":"1975","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Huang et al., 2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.","title":"Densely connected convolutional networks","authors":"Gao Huang, Zhuang Liu, Laurens Van Der Maaten, Kilian Q Weinberger","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Sergey Ioffe, Christian Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Juefei-Xu et al., 2017] Felix Juefei-Xu, Vishnu Naresh Bod- deti, and Marios Savvides. Local binary convolutional neural networks. In CVPR, 2017.","title":"Felix Juefei-Xu, Vishnu Naresh Bod","authors":"Juefei-Xu","year":"2017","venue":"Local binary convolutional neural networks","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liu et al., 2019] Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. In ICLR, 2019.","title":"DARTS: Differentiable architecture search","authors":"Liu","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lu et al., 2019] Zhichao Lu, Ian Whalen, Vishnu Boddeti, Yashesh Dhebar, Kalyanmoy Deb, Erik Goodman, and Wolfgang Banzhaf. Nsga-net: Neural architecture search using multi-objective genetic algorithm. In GECCO, 2019.","title":"Nsga-net: Neural architecture search using multi-objective genetic algorithm","authors":"Zhichao Lu, Ian Whalen, Vishnu Boddeti, Yashesh Dhebar, Kalyanmoy Deb, Erik Goodman, Wolfgang Banzhaf","year":"2019","venue":"GECCO","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Pedersen and Yang, 2006] Gerulf KM Pedersen and Zhenyu Yang. Multi-objective pid-controller tuning for a magnetic levitation system using nsga-ii. In GECCO, pages 1737- 1744. ACM, 2006.","title":"Multi-objective pid-controller tuning for a magnetic levitation system using nsga-ii","authors":"Yang ; Pedersen, K M Gerulf, Zhenyu Pedersen, Yang","year":"2006","venue":"GECCO","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Pelikan et al., 1999] Martin Pelikan, David E Goldberg, and Erick Cantú-Paz. Boa: The bayesian optimization algo- rithm. In GECCO, 1999.","title":"Boa: The bayesian optimization algorithm","authors":"Martin Pelikan, David E Goldberg, Erick Cantú-Paz","year":"1999","venue":"GECCO","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Pham et al., 2018] Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In ICML, 2018.","title":"Efficient neural architecture search via parameters sharing","authors":"Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, Jeff Dean","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Real et al., 2017] Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc V. Le, and Alexey Kurakin. Large-scale evolution of image classifiers. In ICML, 2017.","title":"Large-scale evolution of image classifiers","authors":"Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc V Le, Alexey Kurakin","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Real et al., 2019] Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. Regularized evolution for image classifier architecture search. In AAAI, 2019.","title":"Regularized evolution for image classifier architecture search","authors":"Esteban Real, Alok Aggarwal, Yanping Huang, Quoc V Le","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Sandler et al., 2018] Mark Sandler, Andrew Howard, Men- glong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR, 2018.","title":"Mobilenetv2: Inverted residuals and linear bottlenecks","authors":"Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, Liang-Chieh Chen","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Simonyan and Zisserman, 2015] Karen Simonyan and An- drew Zisserman. Very Deep Convolutional Networks for Large-scale Image Recognition. In ICLR, 2015.","title":"Very Deep Convolutional Networks for Large-scale Image Recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR, 2015.","title":"Going deeper with convolutions","authors":"Szegedy","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tapia and Coello, 2007] Ma Guadalupe Castillo Tapia and Carlos A Coello Coello. Applications of multi-objective evolutionary algorithms in economics and finance: A sur- vey. In CEC, pages 532-539. IEEE, 2007.","title":"Applications of multi-objective evolutionary algorithms in economics and finance: A survey","authors":"Ma Guadalupe, Castillo Tapia, Carlos A Coello Coello","year":"2007","venue":"CEC","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xie and Yuille, 2017] L. Xie and A. Yuille. Genetic CNN. In ICCV, 2017.","title":"Genetic CNN","authors":"L Xie, A Yuille","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zagoruyko and Komodakis, 2016] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.","title":"Wide residual networks","authors":"Sergey Zagoruyko, Nikos Komodakis","year":"2016","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2018] Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient con- volutional neural network for mobile devices. In CVPR, 2018.","title":"Shufflenet: An extremely efficient convolutional neural network for mobile devices","authors":"Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, Jian Sun","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zoph and Le, 2016] B. Zoph and Q. V. Le. Neural Architec- ture Search with Reinforcement Learning. arXiv preprint arXiv:1611.01578, 2016.","title":"Neural Architecture Search with Reinforcement Learning","authors":"B Zoph, Q V Le","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zoph et al., 2018] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In CVPR, 2018.","title":"Learning transferable architectures for scalable image recognition","authors":"Barret Zoph, Vijay Vasudevan, Jonathon Shlens, Quoc V Le","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}