{"type":"ScholarlyArticle","title":"Geometric Enclosing Networks","authors":[{"name":"Trung Le","slug":"trung-le","given":"Trung","family":"Le","affiliation":"Faculty of Information Technology , Monash University"},{"name":"Hung Vu","slug":"hung-vu","given":"Hung","family":"Vu","affiliation":"Center for Pattern Recognition and Data Analytics , Deakin University , Australia"},{"name":"Tu Dinh Nguyen","slug":"tu-dinh-nguyen","given":"Tu Dinh","family":"Nguyen","affiliation":"Faculty of Information Technology , Monash University"},{"name":"Dinh Phung","slug":"dinh-phung","given":"Dinh","family":"Phung","affiliation":"Faculty of Information Technology , Monash University"}],"abstract":"Training model to generate data has increasingly attracted research attention and become important in modern world applications. We propose in this paper a new geometry-based optimization approach to address this problem. Orthogonal to current state-of-the-art density-based approaches, most notably VAE and GAN, we present a fresh new idea that borrows the principle of minimal enclosing ball to train a generator G\\left(\\bz\\right) in such a way that both training and generated data, after being mapped to the feature space, are enclosed in the same sphere. We develop theory to guarantee that the mapping is bijective so that its inverse from feature space to data space results in expressive nonlinear contours to describe the data manifold, hence ensuring data generated are also lying on the data manifold learned from training data. Our model enjoys a nice geometric interpretation, hence termed Geometric Enclosing Networks (GEN), and possesses some key advantages over its rivals, namely simple and easy-to-control optimization formulation, avoidance of mode collapsing and efficiently learn data manifold representation in a completely unsupervised manner. We conducted extensive experiments on synthesis and real-world datasets to illustrate the behaviors, strength and weakness of our proposed GEN, in particular its ability to handle multi-modal data and quality of generated data.","doi":"10.24963/ijcai.2018/326","url":"https://ijcai.org/proceedings/2018/326","pdf":"https://ijcai.org/proceedings/2018/0326.pdf","markdown":"https://ijcai.org/proceedings/2018/326.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/326","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":326,"pages":{"first":2355,"last":2361},"keywords":["Machine Learning: Learning Generative Models","Machine Learning: Kernel Methods"],"videos":[],"references":[{"position":1,"raw":"[Arjovsky et al., 2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein gan. arXiv preprint arXiv:1701.07875, 2017.","title":"Wasserstein gan","authors":"Martin Arjovsky, Soumith Chintala, Léon Bottou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ben-Hur et al., 2001] Asa Ben-Hur, David Horn, Hava T. Siegelmann, and Vladimir Vapnik. Support vector clus- tering. Journal of Machine Learning Research (JMLR), 2:125-137, 2001.","title":"Support vector clustering","authors":"Asa Ben-Hur, David Horn, T Hava, Vladimir Siegelmann, Vapnik","year":"2001","venue":"Journal of Machine Learning Research (JMLR)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bochner, 1959] Salomon Bochner. Lectures on Fourier In- tegrals, volume 42. Princeton University Press, 1959.","title":"Salomon Bochner","authors":"Bochner","year":"1959","venue":"Lectures on Fourier Integrals","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. In- fogan: Interpretable representation learning by informa- tion maximizing generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS), pages 2172-2180.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":null,"venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Curran Associates, Inc., 2016.","title":null,"authors":"Curran Associates, Inc","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cortes and Vapnik, 1995] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning, 20(3):273-297, September 1995.","title":"Corinna Cortes and Vladimir Vapnik. Support-vector networks","authors":"Vapnik Cortes","year":"1995","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dziugaite et al., 2015] Gintare Karolina Dziugaite, Daniel M. Roy, and Zoubin Ghahramani. Training generative neural networks via maximum mean discrep- ancy optimization. arXiv preprint arXiv:1505.03906, 2015.","title":"Training generative neural networks via maximum mean discrepancy optimization","authors":"Gintare Karolina Dziugaite, Daniel M Roy, Zoubin Ghahramani","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in Neural Informa- tion Processing Systems (NIPS), pages 2672-2680. 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow, 2017] Ian Goodfellow. NIPS 2016 tutorial: Generative adversarial networks. CoRR, 2017.","title":"NIPS 2016 tutorial: Generative adversarial networks","authors":"Ian Goodfellow","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hoang et al., 2018] Quan Hoang, Tu Dinh Nguyen, Trung Le, and Dinh Phung. MGAN: Training generative adver- sarial nets with multiple generators. In International Con- ference on Learning Representations, 2018.","title":"MGAN: Training generative adversarial nets with multiple generators","authors":"Quan Hoang, Tu Dinh Nguyen, Trung Le, Dinh Phung","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Welling, 2014] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. In The Inter- national Conference on Learning Representations (ICLR), 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"The International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Technical report, Univer- sity of Toronto, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lecun et al., 1998] Yann Lecun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning applied to document recognition, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2015] Yujia Li, Kevin Swersky, and Richard Zemel. Generative moment matching networks. In Inter- national Conference on Machine Learning (ICML), pages 1718-1727, 2015.","title":"Generative moment matching networks","authors":"Yujia Li, Kevin Swersky, Richard Zemel","year":"2015","venue":"International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.","title":"Deep learning face attributes in the wild","authors":"Ziwei Liu, Ping Luo, Xiaogang Wang, Xiaoou Tang","year":"2015","venue":"Proceedings of International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Metz et al., 2016] Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. arXiv preprint arXiv:1611.02163, 2016.","title":"Unrolled generative adversarial networks","authors":"Metz","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nguyen et al., 2017] Tu Dinh Nguyen, Trung Le, Hung Vu, and Dinh Phung. Dual discriminator generative adversar- ial nets. In Advances in Neural Information Processing Systems, pages 2667-2677, 2017.","title":"Dual discriminator generative adversarial nets","authors":"Trung Tu Dinh Nguyen, Hung Le, Dinh Vu, Phung","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nowozin et al., 2016] Sebastian Nowozin, Boston Cseke, and Ryota Tomioka. f-gan: Training generative neu- ral samplers using variational divergence minimization. In Advances in Neural Information Processing Systems (NIPS), pages 271-279, 2016.","title":"f-gan: Training generative neural samplers using variational divergence minimization","authors":"Nowozin","year":"2016","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rahimi and Recht, 2007] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Ad- vances in Neural Information Processing Systems (NIPS), pages 1177-1184, 2007.","title":"Random features for large-scale kernel machines","authors":"Recht Rahimi, Ali Rahimi, Benjamin Recht","year":"2007","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Roweis and Saul, 2000] Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally lin- ear embedding. Science, 290:2323-2326, 2000.","title":"Nonlinear dimensionality reduction by locally linear embedding","authors":"Sam T Roweis, Lawrence K Saul","year":"2000","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Salimans et al., 2016] Tim Salimans, Ian Goodfellow, Woj- ciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. In Ad- vances in Neural Information Processing Systems (NIPS), pages 2226-2234, 2016.","title":"Improved techniques for training gans","authors":"Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, Xi Chen","year":"2016","venue":"Advances in Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vapnik, 1995] Vladimir Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag New York, Inc., 1995.","title":"The Nature of Statistical Learning Theory","authors":"Vladimir Vapnik","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}