{"type":"ScholarlyArticle","title":"CAGAN: Consistent Adversarial Training Enhanced GANs","authors":[{"name":"Yao Ni","slug":"yao-ni","given":"Yao","family":"Ni","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"},{"name":"Dandan Song","slug":"dandan-song","given":"Dandan","family":"Song","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"},{"name":"Xi Zhang","slug":"xi-zhang","given":"Xi","family":"Zhang","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"},{"name":"Hao Wu","slug":"hao-wu","given":"Hao","family":"Wu","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"},{"name":"Lejian Liao","slug":"lejian-liao","given":"Lejian","family":"Liao","affiliation":"Lab of High Volume language Information Processing & Cloud Computing Beijing Lab of Intelligent Information Technology School of Computer Science & Technology , Beijing Institute of Technology"}],"abstract":"Generative adversarial networks (GANs) have shown impressive results, however, the generator and the discriminator are optimized in finite parameter space which means their performance still need to be improved. In this paper, we propose a novel approach of adversarial training between one generator and an exponential number of critics which are sampled from the original discriminative neural network via dropout. As discrepancy between outputs of different sub-networks of a same sample can measure the consistency of these critics, we encourage the critics to be consistent to real samples and inconsistent to generated samples during training, while the generator is trained to generate consistent samples for different critics. Experimental results demonstrate that our method can obtain state-of-the-art Inception scores of 9.17 and 10.02 on supervised CIFAR-10 and unsupervised STL-10 image generation tasks, respectively, as well as achieve competitive semi-supervised classification results on several benchmarks. Importantly, we demonstrate that our method can maintain stability in training and alleviate mode collapse.","doi":"10.24963/ijcai.2018/359","url":"https://ijcai.org/proceedings/2018/359","pdf":"https://ijcai.org/proceedings/2018/0359.pdf","markdown":"https://ijcai.org/proceedings/2018/359.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/359","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":359,"pages":{"first":2588,"last":2594},"keywords":["Machine Learning: Neural Networks","Machine Learning: Learning Generative Models","Machine Learning: Semi-Supervised Learning","Machine Learning: Deep Learning","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In ICML, pages 214-223, 2017.","title":"Wasserstein generative adversarial networks","authors":"Arjovsky","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arora et al., 2017] Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. Generalization and equilibrium in generative adversarial nets (gans). In ICML, pages 224- 232, 2017.","title":"Generalization and equilibrium in generative adversarial nets (gans)","authors":"Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, Yi Zhang","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berthelot et al., 2017] David Berthelot, Tom Schumm, and Luke Metz. BEGAN: boundary equilibrium generative ad- versarial networks. CoRR, abs/1703.10717, 2017.","title":"BEGAN: boundary equilibrium generative adversarial networks","authors":"David Berthelot, Tom Schumm, Luke Metz","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Coates et al., 2011] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In AISTATS, 2011.","title":"An analysis of single-layer networks in unsupervised feature learning","authors":"Adam Coates, Andrew Ng, Honglak Lee","year":"2011","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dai et al., 2017] Zihang Dai, Zhilin Yang, Fan Yang, William W Cohen, and Ruslan R Salakhutdinov. Good semi-supervised learning that requires a bad gan. In NIPS, pages 6513-6523. 2017.","title":"Good semi-supervised learning that requires a bad gan","authors":"Dai","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dumoulin et al., 2016] Vincent Dumoulin, Ishmael Belg- hazi, Ben Poole, Alex Lamb, Martín Arjovsky, Olivier Mastropietro, and Aaron C. Courville. Adversarially learned inference. CoRR, abs/1606.00704, 2016.","title":"Adversarially learned inference","authors":"Ishmael Vincent Dumoulin, Ben Belghazi, Alex Poole, Martín Lamb, Olivier Arjovsky, Aaron C Mastropietro, Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Durugkar et al., 2016] Ishan P. Durugkar, Ian Gemp, and Sridhar Mahadevan. Generative multi-adversarial net- works. CoRR, abs/1611.01673, 2016.","title":"Generative multi-adversarial networks","authors":"P Ishan, Ian Durugkar, Sridhar Gemp, Mahadevan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NIPS, pages 2672-2680. 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow, 2017] Ian Goodfellow. NIPS 2016 tutorial: Generative adversarial networks. CoRR, abs/1701.00160, 2017.","title":"NIPS 2016 tutorial: Generative adversarial networks","authors":"Ian Goodfellow","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grinblat et al., 2017] Guillermo L. Grinblat, Lucas C. Uzal, and Pablo M. Granitto. Class-Splitting Generative Adver- sarial Networks. ArXiv e-prints, September 2017.","title":"Class-Splitting Generative Adversarial Networks","authors":"Guillermo L Grinblat, Lucas C Uzal, Pablo M Granitto","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gulrajani et al., 2017] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In NIPS, pages 5769-5779. 2017.","title":"Improved training of wasserstein gans","authors":"Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Aaron C Vincent Dumoulin, Courville","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hoang et al., 2018] Quan Hoang, Tu Dinh Nguyen, Trung Le, and Dinh Phung. MGAN: Training generative adver- sarial nets with multiple generators. In ICLR, 2018.","title":"MGAN: Training generative adversarial nets with multiple generators","authors":"Quan Hoang, Tu Dinh Nguyen, Trung Le, Dinh Phung","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Technical report, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[LI et al., 2017a] Chongxuan LI, Taufik Xu, Jun Zhu, and Bo Zhang. Triple generative adversarial nets. In NIPS, pages 4091-4101. 2017.","title":"Triple generative adversarial nets","authors":"Li","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2017b] Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabas Poczos. MMD GAN: Towards deeper understanding of moment matching network. In NIPS, pages 2203-2213. 2017.","title":"MMD GAN: Towards deeper understanding of moment matching network","authors":"Li","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu and Tuzel, 2016] Ming-Yu Liu and Oncel Tuzel. Cou- pled generative adversarial networks. In NIPS, pages 469- 477. 2016.","title":"Coupled generative adversarial networks","authors":"Ming-Yu Liu, Oncel Tuzel","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lucic et al., 2017] Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly, and Olivier Bousquet. Are GANs Created Equal? A Large-Scale Study. ArXiv e- prints, November 2017.","title":"Are GANs Created Equal? A Large-Scale Study","authors":"Lucic","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Mao et al., 2016] Xudong Mao, Qing Li, Haoran Xie, Ray- mond Y. K. Lau, and Zhen Wang. Multi-class generative adversarial networks with the L2 loss function. CoRR, abs/1611.04076, 2016.","title":"Multi-class generative adversarial networks with the L2 loss function","authors":"Xudong Mao, Qing Li, Haoran Xie, Y K Raymond, Zhen Lau, Wang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nguyen et al., 2017] Tu Nguyen, Trung Le, Hung Vu, and Dinh Phung. Dual discriminator generative adversarial nets. In NIPS, pages 2667-2677. 2017.","title":"Dual discriminator generative adversarial nets","authors":"Tu Nguyen, Trung Le, Hung Vu, Dinh Phung","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Nowozin et al., 2016] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In NIPS, pages 271-279. 2016.","title":"f-gan: Training generative neural samplers using variational divergence minimization","authors":"Sebastian Nowozin, Botond Cseke, Ryota Tomioka","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pathak et al., 2016] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A. Efros. Con- text encoders: Feature learning by inpainting. In CVPR, June 2016.","title":"Context encoders: Feature learning by inpainting","authors":"Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, Alexei A Efros","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, abs/1511.06434, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Saatchi and Wilson, 2017] Yunus Saatchi and Andrew G Wilson. Bayesian GAN. In NIPS, pages 3625-3634. 2017.","title":"Bayesian GAN","authors":"Wilson ; Saatchi, Yunus Saatchi, Andrew G Wilson","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Salimans et al., 2016] Tim Salimans, Ian Goodfellow, Wo- jciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. In NIPS, pages 2234-2242. 2016.","title":"Improved techniques for training gans","authors":"Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, Xi Chen","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Springenberg, 2015] Jost Tobias Springenberg. Unsuper- vised and Semi-supervised Learning with Categorical Generative Adversarial Networks. ArXiv e-prints, Novem- ber 2015.","title":"Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks","authors":"Jost Tobias, Springenberg","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 15:1929-1958, 2014.","title":"Dropout: A simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Takeru Miyato, 2018] Masanori Koyama Yuichi Yoshida Takeru Miyato, Toshiki Kataoka. Spectral normalization for generative adversarial networks. ICLR, 2018.","title":"Spectral normalization for generative adversarial networks","authors":"Masanori Koyama, Yuichi Yoshida, Takeru Miyato, Toshiki Kataoka","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tolstikhin et al., 2017] Ilya O Tolstikhin, Sylvain Gelly, Olivier Bousquet, Carl-Johann SIMON-GABRIEL, and Bernhard Schölkopf. Adagan: Boosting generative mod- els. In NIPS, pages 5430-5439. 2017.","title":"Adagan: Boosting generative models","authors":"Sylvain Ilya O Tolstikhin, Olivier Gelly, Bousquet, Simon-Gabriel Carl-Johann, Bernhard Schölkopf","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Warde-Farley and Bengio, 2017] David Warde-Farley and Yoshua Bengio. Improving generative adversarial net- works with denoising feature matching. 2017.","title":"Improving generative adversarial networks with denoising feature matching","authors":"Warde-Farley, Bengio ; David Warde-Farley, Yoshua Bengio","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wei et al., 2018] Xiang Wei, Zixia Liu, Liqiang Wang, and Boqing Gong. Improving the improved training of wasser- stein gans. ICLR, 2018.","title":"Improving the improved training of wasserstein gans","authors":"Xiang Wei, Zixia Liu, Liqiang Wang, Boqing Gong","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhou et al., 2018] Zhiming Zhou, Han Cai, Shu Rong, Yux- uan Song, Kan Ren, Weinan Zhang, Jun Wang, and Yong Yu. Activation maximization generative adversarial nets. ICLR, 2018.","title":"Activation maximization generative adversarial nets","authors":"Zhiming Zhou, Han Cai, Shu Rong, Yuxuan Song, Weinan Kan Ren, Jun Zhang, Yong Wang, Yu","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}