{"type":"ScholarlyArticle","title":"Joint Generative Moment-Matching Network for Learning Structural Latent Code","authors":[{"name":"Hongchang Gao","slug":"hongchang-gao","given":"Hongchang","family":"Gao","affiliation":"Department of Electrical and Computer Engineering , University of Pittsburgh , USA"},{"name":"Heng Huang","slug":"heng-huang","given":"Heng","family":"Huang","affiliation":"Department of Electrical and Computer Engineering , University of Pittsburgh , USA"}],"abstract":"Generative Moment-Matching Network (GMMN) is a deep generative model, which employs maximum mean discrepancy as the objective to learn model parameters. However, this model can only generate samples, failing to infer the latent code from samples for downstream tasks. In this paper, we propose a novel Joint Generative Moment-Matching Network (JGMMN), which learns the structural latent code for unsupervised inference. Specifically, JGMMN has a generation network for the generation task and an inference network for the inference task. We first reformulate this model as the two joint distributions matching problem. To solve this problem, we propose to use the Joint Maximum Mean Discrepancy (JMMD) as the objective to learn these two networks simultaneously. Furthermore, to enforce the consistency between the sample distribution and the inferred latent code distribution, we propose a novel multi-modal regularization to enforce this consistency. At last, extensive experiments on both synthetic and real-world datasets have verified the effectiveness and correctness of our proposed JGMMN.","doi":"10.24963/ijcai.2018/293","url":"https://ijcai.org/proceedings/2018/293","pdf":"https://ijcai.org/proceedings/2018/0293.pdf","markdown":"https://ijcai.org/proceedings/2018/293.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/293","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":293,"pages":{"first":2121,"last":2127},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Learning Generative Models","Machine Learning: Neural Networks"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.","title":"Wasserstein gan","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2011] Deng Cai, Xiaofei He, Jiawei Han, and Thomas S Huang. Graph regularized nonnegative ma- trix factorization for data representation. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 33(8):1548-1560, 2011.","title":"Graph regularized nonnegative matrix factorization for data representation","authors":"Deng Cai, Xiaofei He, Jiawei Han, Thomas S Huang","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. In- fogan: Interpretable representation learning by informa- tion maximizing generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2172- 2180, 2016.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Denton et al., 2015] Emily L Denton, Soumith Chintala, Rob Fergus, et al. Deep generative image models using a laplacian pyramid of adversarial networks. In Advances in neural information processing systems, pages 1486-1494, 2015.","title":"Deep generative image models using a laplacian pyramid of adversarial networks","authors":"Soumith Emily L Denton, Rob Chintala, Fergus","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dumoulin et al., 2016] Vincent Dumoulin, Ishmael Belg- hazi, Ben Poole, Alex Lamb, Martin Arjovsky, Olivier Mastropietro, and Aaron Courville. Adversarially learned inference. arXiv preprint arXiv:1606.00704, 2016.","title":"Adversarially learned inference","authors":"Ishmael Vincent Dumoulin, Ben Belghazi, Alex Poole, Martin Lamb, Olivier Arjovsky, Aaron Mastropietro, Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gretton et al., 2012] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(Mar):723-773, 2012.","title":"A kernel two-sample test","authors":"Arthur Gretton, Karsten M Borgwardt, J Malte, Bernhard Rasch, Alexander Schölkopf, Smola","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gulrajani et al., 2017] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028, 2017.","title":"Improved training of wasserstein gans","authors":"Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Aaron Vincent Dumoulin, Courville","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim et al., 2017] Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jungkwon Lee, and Jiwon Kim. Learning to dis- cover cross-domain relations with generative adversarial networks. arXiv preprint arXiv:1703.05192, 2017.","title":"Learning to discover cross-domain relations with generative adversarial networks","authors":"Kim","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ledig et al., 2016] Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistic single image super- resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802, 2016.","title":"Photo-realistic single image superresolution using a generative adversarial network","authors":"Ledig","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2015] Yujia Li, Kevin Swersky, and Rich Zemel. Generative moment matching networks. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 1718-1727, 2015.","title":"Generative moment matching networks","authors":"Li","year":"2015","venue":"Proceedings of the 32nd International Conference on Machine Learning (ICML-15)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2017] Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. Mmd gan: Towards deeper understanding of moment matching network. In Advances in Neural Information Processing Systems, pages 2200-2210, 2017.","title":"Mmd gan: Towards deeper understanding of moment matching network","authors":"Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, Barnabás Póczos","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Long et al., 2016] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learn- ing with joint adaptation networks. arXiv preprint arXiv:1605.06636, 2016.","title":"Deep transfer learning with joint adaptation networks","authors":"Mingsheng Long, Han Zhu, Jianmin Wang, Michael I Jordan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nair and Hinton, 2010] Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann ma- chines. In Proceedings of the 27th international confer- ence on machine learning (ICML), pages 807-814, 2010.","title":"Rectified linear units improve restricted boltzmann machines","authors":"Vinod Nair, Geoffrey E Hinton","year":"2010","venue":"Proceedings of the 27th international conference on machine learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Reed et al., 2016] Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. arXiv preprint arXiv:1605.05396, 2016.","title":"Generative adversarial text to image synthesis","authors":"Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ren et al., 2016] Yong Ren, Jun Zhu, Jialian Li, and Yucen Luo. Conditional generative moment-matching networks. In Advances in Neural Information Processing Systems, pages 2928-2936, 2016.","title":"Conditional generative moment-matching networks","authors":"Yong Ren, Jun Zhu, Jialian Li, Yucen Luo","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Song et al., 2009] Le Song, Jonathan Huang, Alex Smola, and Kenji Fukumizu. Hilbert space embeddings of con- ditional distributions with applications to dynamical sys- tems. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 961-968, 2009.","title":"Hilbert space embeddings of conditional distributions with applications to dynamical systems","authors":"Le Song, Jonathan Huang, Alex Smola, Kenji Fukumizu","year":"2009","venue":"Proceedings of the 26th Annual International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Song et al., 2010] Le Song, Byron Boots, Sajid M Siddiqi, Geoffrey J Gordon, and Alex Smola. Hilbert space em- beddings of hidden markov models. 2010.","title":"Hilbert space embeddings of hidden markov models","authors":"Le Song, Byron Boots, M Sajid, Geoffrey J Siddiqi, Alex Gordon, Smola","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sriperumbudur et al., 2010] Bharath K Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, and Gert RG Lanckriet. Hilbert space embeddings and metrics on probability measures. Journal of Machine Learning Research, 11(Apr):1517-1561, 2010.","title":"Hilbert space embeddings and metrics on probability measures","authors":"Arthur Bharath K Sriperumbudur, Kenji Gretton, Bernhard Fukumizu, Gert Rg Schölkopf, Lanckriet","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xie et al., 2016] Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learn- ing, pages 478-487, 2016.","title":"Unsupervised deep embedding for clustering analysis","authors":"Junyuan Xie, Ross Girshick, Ali Farhadi","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2016] Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European Conference on Computer Vision, pages 649-666, 2016.","title":"Colorful image colorization","authors":"Richard Zhang, Phillip Isola, Alexei A Efros","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image transla- tion using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Jun-Yan Zhu, Taesung Park, Phillip Isola, Alexei A Efros","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}