{"type":"ScholarlyArticle","title":"MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation","authors":[{"name":"David Keetae Park","slug":"david-keetae-park","given":"David Keetae","family":"Park","affiliation":"† University of North Carolina at Charlotte {heykeetae ,"},{"name":"Seungjoo Yoo","slug":"seungjoo-yoo","given":"Seungjoo","family":"Yoo","affiliation":"† University of North Carolina at Charlotte {heykeetae ,"},{"name":"Hyojin Bahng","slug":"hyojin-bahng","given":"Hyojin","family":"Bahng","affiliation":"† University of North Carolina at Charlotte {heykeetae ,"},{"name":"Jaegul Choo","slug":"jaegul-choo","given":"Jaegul","family":"Choo","affiliation":"† University of North Carolina at Charlotte {heykeetae ,"},{"name":"Noseong Park","slug":"noseong-park","given":"Noseong","family":"Park","affiliation":"‡ Korea University; † University of North Carolina at Charlotte {heykeetae ,"}],"abstract":"Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e.g., an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0.2470 for CelebA and a competitive unsupervised inception score of 8.33 in CIFAR-10.","doi":"10.24963/ijcai.2018/122","url":"https://ijcai.org/proceedings/2018/122","pdf":"https://ijcai.org/proceedings/2018/0122.pdf","markdown":"https://ijcai.org/proceedings/2018/122.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/122","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":122,"pages":{"first":878,"last":884},"keywords":["Machine Learning: Deep Learning","Computer Vision: Computer Vision","Machine Learning: Learning Generative Models"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.","title":"Wasserstein gan","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berthelot et al., 2017] David Berthelot, Tom Schumm, and Luke Metz. Began: Boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717, 2017.","title":"Began: Boundary equilibrium generative adversarial networks","authors":"David Berthelot, Tom Schumm, Luke Metz","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ghosh et al., 2017] Arnab Ghosh, Viveka Kulharia, Vinay Namboodiri, Philip HS Torr, and Puneet K Dokania. Multi-agent diverse generative adversarial networks. arXiv preprint arXiv:1704.02906, 2017.","title":"Multi-agent diverse generative adversarial networks","authors":"Arnab Ghosh, Viveka Kulharia, Vinay Namboodiri, Puneet K Philip Hs Torr, Dokania","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Grinblat et al., 2017] Guillermo L Grinblat, Lucas C Uzal, and Pablo M Granitto. Class-splitting generative adversar- ial networks. arXiv preprint arXiv:1709.07359, 2017.","title":"Class-splitting generative adversarial networks","authors":"Guillermo L Grinblat, Lucas C Uzal, Pablo M Granitto","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gumbel, 1954] Emil Julius Gumbel. Statistical theory of extreme valuse and some practical applications. Nat. Bur. Standards Appl. Math. Ser. 33, 1954.","title":"Statistical theory of extreme valuse and some practical applications","authors":"Emil Julius Gumbel, Gumbel","year":"1954","venue":"Nat. Bur. Standards Appl. Math. Ser","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hampshire and Waibel, 1990] John B Hampshire and Alex H Waibel. The meta-pi network: Connectionist rapid adaptation for high-performance multi-speaker phoneme recognition. In Acoustics, Speech, and Signal Processing, 1990. ICASSP-90., 1990 International Conference on, pages 165-168. IEEE, 1990.","title":"The meta-pi network: Connectionist rapid adaptation for high-performance multi-speaker phoneme recognition","authors":"B John, Alex H Hampshire, Waibel","year":"1990","venue":"Acoustics, Speech, and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hoang et al., 2017] Quan Hoang, Tu Dinh Nguyen, Trung Le, and Dinh Phung. Multi-generator gernerative adver- sarial nets. arXiv preprint arXiv:1708.02556, 2017.","title":"Multi-generator gernerative adversarial nets","authors":"Quan Hoang, Tu Dinh Nguyen, Trung Le, Dinh Phung","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2017] Xun Huang, Yixuan Li, Omid Pour- saeed, John Hopcroft, and Serge Belongie. Stacked gener- ative adversarial networks. In IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), volume 2, page 4, 2017.","title":"Stacked generative adversarial networks","authors":"Xun Huang, Yixuan Li, Omid Poursaeed, John Hopcroft, Serge Belongie","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jacobs et al., 1991] Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mix- tures of local experts. Neural computation, 3(1):79-87, 1991.","title":"Adaptive mixtures of local experts","authors":"Robert A Jacobs, Michael I Jordan, Steven J Nowlan, Geoffrey E Hinton","year":"1991","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jang et al., 2016] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.","title":"Categorical reparameterization with gumbel-softmax","authors":"Eric Jang, Shixiang Gu, Ben Poole","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Karras et al., 2017] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.","title":"Progressive growing of gans for improved quality, stability, and variation","authors":"Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kodali et al., 2017] Naveen Kodali, Jacob Abernethy, James Hays, and Zsolt Kira. How to train your dragan. arXiv preprint arXiv:1705.07215, 2017.","title":"How to train your dragan","authors":"Naveen Kodali, Jacob Abernethy, James Hays, Zsolt Kira","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Maddison et al., 2014] Chris J Maddison, Daniel Tarlow, and Tom Minka. A* sampling. In Advances in Neural Information Processing Systems, pages 3086-3094, 2014.","title":"A* sampling","authors":"Chris J Maddison, Daniel Tarlow, Tom Minka","year":"2014","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mao et al., 2017] Xudong Mao, Qing Li, Haoran Xie, Ray- mond YK Lau, Zhen Wang, and Stephen Paul Smol- ley. Least squares generative adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2813-2821. IEEE, 2017.","title":"Least squares generative adversarial networks","authors":"Xudong Mao, Qing Li, Haoran Xie, Raymond Yk Lau, Zhen Wang, Stephen Paul Smolley","year":"2017","venue":"IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Salimans et al., 2016] Tim Salimans, Ian Goodfellow, Woj- ciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, pages 2234- 2242, 2016.","title":"Improved techniques for training gans","authors":"Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Tolstikhin et al., 2017] Ilya O Tolstikhin, Sylvain Gelly, Olivier Bousquet, Carl-Johann Simon-Gabriel, and Bern- hard Schölkopf. Adagan: Boosting generative models. In Advances in Neural Information Processing Systems, pages 5430-5439, 2017.","title":"Adagan: Boosting generative models","authors":"Sylvain Ilya O Tolstikhin, Olivier Gelly, Carl-Johann Bousquet, Bernhard Simon-Gabriel, Schölkopf","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}