MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation

MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation

David Keetae Park, Seungjoo Yoo, Hyojin Bahng, Jaegul Choo, Noseong Park

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 878-884. https://doi.org/10.24963/ijcai.2018/122

Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.
Keywords:
Machine Learning: Deep Learning
Machine Learning: Learning Generative Models
Computer Vision: Computer Vision