CR-GAN: Learning Complete Representations for Multi-view Generation

CR-GAN: Learning Complete Representations for Multi-view Generation

Yu Tian, Xi Peng, Long Zhao, Shaoting Zhang, Dimitris N. Metaxas

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 942-948. https://doi.org/10.24963/ijcai.2018/131

Generating multi-view images from a single-view input is an important yet challenging problem. It has broad applications in vision, graphics, and robotics. Our study indicates that the widely-used generative adversarial network (GAN) may learn ?incomplete? representations due to the single-pathway framework: an encoder-decoder network followed by a discriminator network.We propose CR-GAN to address this problem. In addition to the single reconstruction path, we introduce a generation sideway to maintain the completeness of the learned embedding space. The two learning paths collaborate and compete in a parameter-sharing manner, yielding largely improved generality to ?unseen? dataset. More importantly, the two-pathway framework makes it possible to combine both labeled and unlabeled data for self-supervised learning, which further enriches the embedding space for realistic generations. We evaluate our approach on a wide range of datasets. The results prove that CR-GAN significantly outperforms state-of-the-art methods, especially when generating from ?unseen? inputs in wild conditions.
Keywords:
Humans and AI: Human-Computer Interaction
Robotics: Robotics and Vision
Computer Vision: 2D and 3D Computer Vision
Computer Vision: Biometrics, Face and Gesture Recognition