{"type":"ScholarlyArticle","title":"Closed-Loop Memory GAN for Continual Learning","authors":[{"name":"Amanda Rios","slug":"amanda-rios","given":"Amanda","family":"Rios","affiliation":"University of Southern California , Los Angeles , USA"},{"name":"Laurent Itti","slug":"laurent-itti","given":"Laurent","family":"Itti","affiliation":"University of Southern California , Los Angeles , USA"}],"abstract":"Sequential learning of tasks using gradient descent leads to an unremitting decline in the accuracy of tasks for which training data is no longer available, termed catastrophic forgetting. Generative models have been explored as a means to approximate the distribution of old tasks and bypass storage of real data. Here we propose a cumulative closed-loop memory replay GAN (CloGAN) provided with external regularization by a small memory unit selected for maximum sample diversity. We evaluate incremental class learning using a notoriously hard paradigm, single-headed learning, in which each task is a disjoint subset of classes in the overall dataset, and performance is evaluated on all previous classes. First, we show that when constructing a dynamic memory unit to preserve sample heterogeneity, model performance asymptotically approaches training on the full dataset. We then show that using a stochastic generator to continuously output fresh new images during training increases performance significantly further meanwhile generating quality images. We compare our approach to several baselines including fine-tuning by gradient descent (FGD), Elastic Weight Consolidation (EWC), Deep Generative Replay (DGR) and Memory Replay GAN (MeRGAN). Our method has very low long-term memory cost, the memory unit, as well as negligible intermediate memory storage.","doi":"10.24963/ijcai.2019/462","url":"https://ijcai.org/proceedings/2019/462","pdf":"https://ijcai.org/proceedings/2019/0462.pdf","markdown":"https://ijcai.org/proceedings/2019/462.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/462","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":462,"pages":{"first":3332,"last":3338},"keywords":["Machine Learning: Classification","Humans and AI: Cognitive Modeling","Machine Learning: Learning Generative Models","Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Deep Learning","Computer Vision: Statistical Methods and Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Azadi et al., 2018] Samaneh Azadi, Catherine Olsson, Trevor Darrell, Ian Goodfellow, and Augustus Odena. Dis- criminator rejection sampling. 2018.","title":"Discriminator rejection sampling","authors":"Azadi","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cohen et al., 2017] Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. Emnist: an exten- sion of mnist to handwritten letters. arXiv preprint arXiv:1702.05373, 2017.","title":"Emnist: an extension of mnist to handwritten letters","authors":"Gregory Cohen, Saeed Afshar, Jonathan Tapson, André Van Schaik","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Farquhar and Gal, 2018] Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09733v1, 2018.","title":"Towards robust evaluations of continual learning","authors":"Gal Farquhar, Sebastian Farquhar, Yarin Gal","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fernando et al., 2017] Chrisantha Fernando, Dylan Ba- narse, Charles Blundell, Yori Zwols, David Ha, A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet : Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734, 2017.","title":"Chrisantha Fernando, Dylan Banarse","authors":"Fernando","year":"2017","venue":"Pathnet : Evolution channels gradient descent in super neural networks","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Flesch et al., 2018] Timo Flesch, Jan Balaguer, Ronald Dekker, Hamed Nili, and Christopher Summerfield. Comparing continual task learning in minds and ma- chines. Proceedings of the National Academy of Sciences, 115(44):E10313-E10322, 2018.","title":"Comparing continual task learning in minds and machines","authors":"Timo Flesch, Jan Balaguer, Ronald Dekker, Hamed Nili, Christopher Summerfield","year":"2018","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[French, 1999] Robert M French. Catastrophic forgetting in connectionist networks. 6613(April):128-135, 1999.","title":"Catastrophic forgetting in connectionist networks","authors":"M Robert, French","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goldberg and Deb, 1991] David E Goldberg and Kalyan- moy Deb. A comparative analysis of selection schemes used in genetic algorithms. In Foundations of genetic al- gorithms, volume 1, pages 69-93. Elsevier, 1991.","title":"A comparative analysis of selection schemes used in genetic algorithms","authors":"E David, Kalyanmoy Goldberg, Deb","year":"1991","venue":"Foundations of genetic algorithms","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kemker and Kanan, 2018] Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. 2018.","title":"Fearnet: Brain-inspired model for incremental learning","authors":"Kanan Kemker, Ronald Kemker, Christopher Kanan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Kingma et al., ] Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi- supervised learning with deep generative models. In Advances in neural information processing systems.","title":"Semisupervised learning with deep generative models","authors":"Kingma","year":null,"venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kirkpatrick et al., 2017] James Kirkpatrick, Razvan Pas- canu, Neil Rabinowitz, Joel Veness, Guillaume Des- jardins, Andrei A Rusu, Kieran Milan, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 2017.","title":"Overcoming catastrophic forgetting in neural networks","authors":"James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan","year":"2017","venue":"Proceedings of the national academy of sciences","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"LeCun et al., ] Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11).","title":"Gradient-based learning applied to document recognition","authors":"Lecun","year":null,"venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Hoiem, 2017] Zhizhong Li and Derek Hoiem. Learning without Forgetting. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2017.","title":"Learning without Forgetting","authors":"Zhizhong Li, Derek Hoiem","year":"2017","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lopez-paz and Ranzato, 2017] David Lopez-paz and Marc Aurelio Ranzato. Gradient episodic memory for continual learning. (Advances in Neural Information Processing Systems), 2017.","title":"Gradient episodic memory for continual learning","authors":"David Lopez-Paz, Marc Aurelio, Ranzato","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[MacKay, 2003] David JC MacKay. Information theory, in- ference and learning algorithms. Cambridge university press, 2003.","title":"Information theory, inference and learning algorithms","authors":"J C David, Mackay","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[McCloskey and Cohen, 1989] Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.","title":"Catastrophic interference in connectionist networks: The sequential learning problem","authors":"Cohen Mccloskey, Michael Mccloskey, Neal J Cohen","year":"1989","venue":"Psychology of learning and motivation","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learn- ing. 2011.","title":"Reading digits in natural images with unsupervised feature learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y Ng","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nguyen et al., 2018] Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, and Richard E. Turner. Variational con- tinual learning. In International Conference on Learning Representations, 2018.","title":"Variational continual learning","authors":"V Cuong, Yingzhen Nguyen, Thang D Li, Richard E Bui, Turner","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Odena et al., 2017] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxil- iary classifier gans. Proceedings of the 34th International Conference on Machine Learning, Sydney, 2017.","title":"Conditional image synthesis with auxiliary classifier gans","authors":"Odena","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Parisi et al., 2019] German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.","title":"Continual lifelong learning with neural networks: A review","authors":"Parisi","year":"2019","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rebuffi et al., 2017] Sylvestre-alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl : Incremental classifier and representation learning. Confer- ence on Computer Vision and Pattern Recognition, 2017.","title":"icarl : Incremental classifier and representation learning","authors":"Alexander Sylvestre-Alvise Rebuffi, Georg Kolesnikov, Christoph H Sperl, Lampert","year":"2017","venue":"Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shin et al., 2017] Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep gener- ative replay. Advances in Neural Information Processing Systems, 2017.","title":"Continual learning with deep generative replay","authors":"Hanul Shin, Jung Kwon Lee, Jaehong Kim, Jiwon Kim","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wu et al., 2018] Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu. Memory replay gans: learning to generate images from new categories without forgetting. Advances in Neural In- formation Processing Systems, 2018.","title":"Memory replay gans: learning to generate images from new categories without forgetting","authors":"Wu","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zenke et al., 2017] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual Learning Through Synaptic In- telligence. Proceedings of the 34th International Confer- ence on Machine Learning, Sydney, Australia, PMLR 70, 2017, 2017.","title":"Continual Learning Through Synaptic Intelligence","authors":"Friedemann Zenke, Ben Poole, Surya Ganguli","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}