{"type":"ScholarlyArticle","title":"Cost Ensemble with Gradient Selecting for GANs","authors":[{"name":"Minghui Liu","slug":"minghui-liu","given":"Minghui","family":"Liu","affiliation":"University of Electronic Science and Technology of China"},{"name":"Jiali Deng","slug":"jiali-deng","given":"Jiali","family":"Deng","affiliation":"University of Electronic Science and Technology of China"},{"name":"Meiyi Yang","slug":"meiyi-yang","given":"Meiyi","family":"Yang","affiliation":"University of Electronic Science and Technology of China"},{"name":"Xuan Cheng","slug":"xuan-cheng","given":"Xuan","family":"Cheng","affiliation":"University of Electronic Science and Technology of China"},{"name":"Nianbo Liu","slug":"nianbo-liu","given":"Nianbo","family":"Liu","affiliation":"University of Electronic Science and Technology of China"},{"name":"Ming Liu","slug":"ming-liu","given":"Ming","family":"Liu","affiliation":"University of Electronic Science and Technology of China"},{"name":"Xiaomin Wang","slug":"xiaomin-wang","given":"Xiaomin","family":"Wang","affiliation":"University of Electronic Science and Technology of China"}],"abstract":"Generative Adversarial Networks(GANs) are powerful generative models on numerous tasks and datasets but are also known for their training instability and mode collapse. The latter is because the optimal transportation map is discontinuous, but DNNs can only approximate continuous ones. One way to solve the problem is to introduce multiple discriminators or generators. However, their impacts are limited because the cost function of each component is the same. That is, they are homogeneous. In contrast, multiple discriminators with different cost functions can yield various gradients for the generator, which indicates we can use them to search for more transportation maps in the latent space. Inspired by this, we have proposed a framework to combat the mode collapse problem, containing multiple discriminators with different cost functions, named CES-GAN. Unfortunately, it may also lead to the generator being hard to train because the performance between discriminators is unbalanced, according to the Cannikin Law. Thus, a gradient selecting mechanism is also proposed to pick up proper gradients. We provide mathematical statements to prove our assumptions and conduct extensive experiments to verify the performance. The results show that CES-GAN is lightweight and more effective for fighting against the mode collapse problem than similar works.","doi":"10.24963/ijcai.2022/167","url":"https://ijcai.org/proceedings/2022/167","pdf":"https://ijcai.org/proceedings/2022/0167.pdf","markdown":"https://ijcai.org/proceedings/2022/167.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/167","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":167,"pages":{"first":1194,"last":1200},"keywords":["Computer Vision: Adversarial learning, adversarial attack and defense methods","Computer Vision: Neural generative models, auto encoders, GANs"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986659","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984802","length_sec":344,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abhishek et al., 2017] Abhishek, Prasanna, and Fletcher. Improved semi-supervised learning with gans using mani- fold invariances. CoRR, abs/1705.08850, 2017.","title":"Improved semi-supervised learning with gans using manifold invariances","authors":"Prasanna Abhishek, Fletcher","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[An et al., 2020] An, Guo, Lei, Luo, Yau, and Gu. Ae-ot: A new generative model based on extended semi-discrete optimal transport. In ICLR, 2020.","title":"Ae-ot: A new generative model based on extended semi-discrete optimal transport","authors":"An","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Arjovsky and Bottou, 2017] Arjovsky and Bottou. Towards principled methods for training generative adversarial net- works. In ICLR, 2017.","title":"Arjovsky and Bottou. Towards principled methods for training generative adversarial networks","authors":"Bottou Arjovsky","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Arjovsky et al., 2017] Arjovsky, Chintala, and Bottou. Wasserstein generative adversarial networks. In ICML, pages 214-223, 2017.","title":"Arjovsky, Chintala, and Bottou. Wasserstein generative adversarial networks","authors":"Arjovsky","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Berthelot et al., 2017] Berthelot, Thomas, and Luke. Be- gan: Boundary equilibrium generative adversarial net- works. CoRR, abs/1703.10717, 2017.","title":"Began: Boundary equilibrium generative adversarial networks","authors":"Thomas Berthelot, Luke","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dowson and Landau, 1982] Dowson and Landau. The frechet distance between multvariate normal distributions. Journal of Multivariate Analysis, 12(3):450-455, 1982.","title":"Dowson and Landau. The frechet distance between multvariate normal distributions","authors":"Landau Dowson","year":"1982","venue":"Journal of Multivariate Analysis","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Durugkar et al., 2017] Durugkar, Gemp, and Mahadevan. Generative multi-adversarial networks. In ICLR, 2017.","title":"Durugkar, Gemp, and Mahadevan. Generative multi-adversarial networks","authors":"Durugkar","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ghosh et al., 2018] Ghosh, Kulharia, Namboodiri, Torr, and Dokania. Multi-agent diverse generative adversarial net- works. In CVPR, pages 8513-8521, 2018.","title":"Multi-agent diverse generative adversarial networks","authors":"Kulharia Ghosh, Torr Namboodiri, Dokania","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Goodfellow, Pouget, Mirza, Xu, Warde, Ozair, Courville, and Bengio. Generative adver- sarial nets. In NIPS, 2014.","title":"Generative adversarial nets","authors":"Pouget Goodfellow, Mirza, Warde Xu, Ozair, Courville, Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2016] Goodfellow, Bengio, and Courville. Deep Learning. MIT Press, 2016.","title":"Deep Learning","authors":"Bengio Goodfellow, Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gulrajani et al., 2017] Gulrajani, Ahmed, Arjovsky, Du- moulin, and Courville. Improved training of wasserstein gans. In NIPS, 2017.","title":"Dumoulin, and Courville. Improved training of wasserstein gans","authors":"Gulrajani","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2016] He, Zhang, Ren, and Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"He","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Heusel et al., 2017] Heusel, Ramsauer, Unterthiner, Nessler, and Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NIPS, 2017.","title":"Gans trained by a two time-scale update rule converge to a local nash equilibrium","authors":"Heusel, Ramsauer, Unterthiner, Nessler, Hochreiter","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hoang et al., 2018] Hoang, Nguyen, Le, and Phung. Multi- generator generative adversarial nets. In ICLR, 2018.","title":"Multigenerator generative adversarial nets","authors":"Nguyen Hoang, Le, Phung","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Karras et al., 2018] Karras, Aila, Laine, and Lehtinen. Pro- gressive growing of gans for improved quality, stability, and variation. In ICLR, 2018.","title":"Progressive growing of gans for improved quality, stability, and variation","authors":"Aila Karras, Laine Lehtinen","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma et al., 2014] Kingma, Diederik, and Welling. Auto- encoding variational bayes. In ICLR, 2014.","title":"Autoencoding variational bayes","authors":"Diederik Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kodali et al., 2017] Kodali, Abernethy, Hays, and Kira. How to train your dragan. CoRR, abs/1705.07215, 2017.","title":"How to train your dragan","authors":"Abernethy Kodali, Kira Hays","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Krizhevsky and Hinton, 2009] Krizhevsky and Hinton. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4), 2009.","title":"Krizhevsky and Hinton. Learning multiple layers of features from tiny images","authors":"Hinton Krizhevsky","year":"2009","venue":"Handbook of Systemic Autoimmune Diseases","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Laine and Aila, 2016] Laine and Aila. Temporal en- sembling for semi-supervised learning. CoRR, abs/1610.02242, 2016.","title":"Temporal ensembling for semi-supervised learning","authors":"Laine, Aila ; Laine, Aila","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[LeCun et al., 1998] LeCun, Bottou, Bengio, and Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Bottou Lecun, Bengio, Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lei et al., 2019] Lei, Guo, An, Qi, Luo, Yau, and Gu. Mode collapse and regularity of optimal transportation maps. CoRR, abs/1902.02934, 2019.","title":"Mode collapse and regularity of optimal transportation maps","authors":"Guo Lei, An, Qi, Luo, Yau, Gu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lin et al., 2018] Lin, Khetan, Fanti, and Sewoong. Pacgan: The power of two samples in generative adversarial net- works. In NIPS, 2018.","title":"Pacgan: The power of two samples in generative adversarial networks","authors":"Lin","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2015] Liu, Luo, Wang, and Tang. Deep learning face attributes in the wild. In ICCV, pages 3730-3738, 2015.","title":"Deep learning face attributes in the wild","authors":"Luo Liu, Wang Tang","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Miyato et al., 2017] Miyato, Maeda, Koyama, and Ishii. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE trans- actions on pattern analysis and machine intelligence, 41(8):1979-1993, 2017.","title":"Virtual adversarial training: A regularization method for supervised and semi-supervised learning","authors":"Miyato","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Miyato et al., 2018] Miyato, Kataoka, Koyama, and Yoshida. Spectral normalization for generative adversarial networks. In ICLR, 2018.","title":"Spectral normalization for generative adversarial networks","authors":"Miyato","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Mordido et al., 2020] Mordido, Yang, and Meinel. micro- batchgan: Stimulating diversity with multi-adversarial dis- crimination. In WACV, pages 3061-3070, 2020.","title":"microbatchgan: Stimulating diversity with multi-adversarial discrimination","authors":"Mordido","year":"2020","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Nguyen et al., 2017] Nguyen, Le, Vu, and Phung. Dual dis- criminator generative adversarial nets. In NIPS, 2017.","title":"Dual discriminator generative adversarial nets","authors":"Le Nguyen, Phung Vu","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Paszke et al., 2019] Paszke, Gross, Massa, Lerer, Bradbury, Chanan, Killeen, Lin, Gimelshein, and Antiga. Pytorch: An imperative style, high-performance deep learning li- brary. In NIPS, 2019.","title":"Pytorch: An imperative style, high-performance deep learning library","authors":"Paszke","year":"2019","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rasmus et al., 2015] Rasmus, Berglund, Honkala, Valpola, and Raiko. Semi-supervised learning with ladder net- works. In NIPS, 2015.","title":"Semi-supervised learning with ladder networks","authors":"Berglund Rasmus, Honkala, Raiko Valpola","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Salimans et al., 2016] Salimans, Goodfellow, Zaremba, Cheung, Radford, and Chen. Improved techniques for training gans. In NIPS, 2016.","title":"Improved techniques for training gans","authors":"Salimans, Goodfellow, Zaremba, Cheung, Chen Radford","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Salimans et al., 2018] Salimans, Zhang, Radford, and Metaxas. Improving gans using optimal transport. CoRR, abs/1805.05573, 2018.","title":"Improving gans using optimal transport","authors":"Zhang Salimans, Metaxas Radford","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tarvainen and Valpola, 2017] Tarvainen and Valpola. Mean teachers are better role models: Weight-averaged consis- tency targets improve semi-supervised deep learning re- sults. In NIPS, 2017.","title":"Tarvainen and Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results","authors":"Valpola Tarvainen","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Tolstikhin et al., 2017] Tolstikhin, Gelly, Bousquet, Simon, and Schölkopf. Adagan: Boosting generative models. In NIPS, 2017.","title":"Adagan: Boosting generative models","authors":"Tolstikhin","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Tolstikhin et al., 2018] Tolstikhin, Bousquet, Gelly, and Schoelkopf. Wasserstein auto-encoders. In ICLR, 2018.","title":"Wasserstein auto-encoders","authors":"Bousquet Tolstikhin, Gelly Schoelkopf","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wei et al., 2018] Wei, Gong, Liu, Lu, and Wang. Improving the improved training of wasserstein gans: A consistency term and its dual effect. In ICLR, 2018.","title":"Improving the improved training of wasserstein gans: A consistency term and its dual effect","authors":"Gong Wei, Lu Liu, Wang","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2018] Zhang, Goodfellow, Metaxas, and Odena. Self-attention generative adversarial networks. In ICML, pages 7354-7363, 2018.","title":"Self-attention generative adversarial networks","authors":"Goodfellow Zhang, Metaxas Odena","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhaoyu and Jun, 2019] Zhaoyu and Jun. Stdgan: Resblock based generative adversarial nets using spectral normaliza- tion and two different discriminators. In ACM MM, pages 674-682, 2019.","title":"Zhaoyu and Jun. Stdgan: Resblock based generative adversarial nets using spectral normalization and two different discriminators","authors":"Jun Zhaoyu","year":"2019","venue":"ACM MM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}