{"type":"ScholarlyArticle","title":"Group-based Learning of Disentangled Representations with Generalizability for Novel Contents","authors":[{"name":"Haruo Hosoya","slug":"haruo-hosoya","given":"Haruo","family":"Hosoya","affiliation":"ATR International , Kyoto , Japan"}],"abstract":"Sensory data are often comprised of independent content and transformation factors.  For example, face images may have shapes as content and poses as transformation.  To infer separately these factors from given data, various ``disentangling'' models have been proposed.  However, many of these are supervised or semi-supervised, either requiring attribute labels that are often unavailable or disallowing for generalization over new contents.  In this study, we introduce a novel deep generative model, called group-based variational autoencoders.  In this, we assume no explicit labels, but a weaker form of structure that groups together data instances having the same content but transformed differently; we thereby separately estimate a group-common factor as content and an instance-specific factor as transformation.  This approach allows for learning to represent a general continuous space of contents, which can accommodate unseen contents.  Despite the simplicity, our model succeeded in learning, from five datasets, content representations that are highly separate from the transformation representation and generalizable to data with novel contents.  We further provide detailed analysis of the latent content code and show insight into how our model obtains the notable transformation invariance and content generalizability.","doi":"10.24963/ijcai.2019/348","url":"https://ijcai.org/proceedings/2019/348","pdf":"https://ijcai.org/proceedings/2019/0348.pdf","markdown":"https://ijcai.org/proceedings/2019/348.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/348","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":348,"pages":{"first":2506,"last":2513},"keywords":["Machine Learning: Deep Learning","Machine Learning: Learning Generative Models"],"videos":[],"references":[{"position":1,"raw":"[Bouchacourt et al., 2018] Diane Bouchacourt, Ryota Tomioka, and Sebastian Nowozin. Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations. In AAAI Conference on Artificial Intelligence, 2018.","title":"Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations","authors":"Diane Bouchacourt, Ryota Tomioka, Sebastian Nowozin","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. InfoGAN - Interpretable Representation Learning by Information Max- imizing Generative Adversarial Nets. Advances in neural information processing systems, 2016.","title":"InfoGAN -Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Mickaël Chen, Ludovic Denoyer, and Thierry Artières. Multi-View Data Generation Without View Supervision. In International Conference on Learn- ing Representations, November 2018.","title":"Multi-View Data Generation Without View Supervision","authors":"Mickaël Chen, Ludovic Denoyer, Thierry Artières","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheung et al., 2014] Brian Cheung, Jesse A Livezey, Ar- jun K Bansal, and Bruno A Olshausen. Discovering Hidden Factors of Variation in Deep Networks. In International Conference on Learning Representations, Workshop, De- cember 2014.","title":"Discovering Hidden Factors of Variation in Deep Networks","authors":"Brian Cheung, Jesse A Livezey, Arjun K Bansal, Bruno A Olshausen","year":"2014","venue":"International Conference on Learning Representations, Workshop","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Denton and Birodkar, 2017] Emily Denton and Vighnesh Birodkar. Unsupervised Learning of Disentangled Rep- resentations from Video. Advances in neural information processing systems, May 2017.","title":"Emily Denton and Vighnesh Birodkar. Unsupervised Learning of Disentangled Representations from Video","authors":"Birodkar Denton","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dosovitskiy and Springenberg, 2015] A Dosovitskiy and J Tobias Springenberg. Learning to generate chairs with convolutional neural networks. Computer Vision and Pat- tern Recognition, 2015.","title":"Learning to generate chairs with convolutional neural networks","authors":"A Dosovitskiy, J Tobias, Springenberg","year":"2015","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[El Shafey et al., 2013] Laurent El Shafey, Chris McCool, Roy Wallace, and Sébastien Marcel. A scalable formu- lation of probabilistic linear discriminant analysis: applied to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7):1788-1794, July 2013.","title":"A scalable formulation of probabilistic linear discriminant analysis: applied to face recognition","authors":"Laurent El Shafey, Chris Mccool, Roy Wallace, Sébastien Marcel","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Földiák, 1991] P Földiák. Learning invariance from trans- formation sequences. Neural Computation, 3(2):194-200, 1991.","title":"Learning invariance from transformation sequences","authors":"Földiák","year":"1991","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] I Goodfellow, J Pouget-Abadie, and M Mirza. Generative adversarial nets. Advances in neural information processing systems, 2014.","title":"Generative adversarial nets","authors":"Goodfellow","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gross et al., 2010] Ralph Gross, Iain Matthews, Jeff Cohn, Takeo Kanade, and Simon Baker. Multi-PIE. Proceedings of the International Conference on Automatic Face and Gesture Recognition. IEEE International Conference on Automatic Face & Gesture Recognition, 28(5):807-813, May 2010.","title":"Multi-PIE","authors":"Ralph Gross, Iain Matthews, Jeff Cohn, Takeo Kanade, Simon Baker","year":"2010","venue":"Proceedings of the International Conference on Automatic Face and Gesture Recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Higgins et al., 2016] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning Basic Visual Concepts with a Constrained Varia- tional Framework. In International Conference on Learning Representations, November 2016.","title":"beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework","authors":"Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hosoya and Hyvärinen, 2016] H Hosoya and A Hyvärinen. Learning Visual Spatial Pooling by Strong PCA Dimension Reduction. Neural Computation, 28:1249-1263, 2016.","title":"Learning Visual Spatial Pooling by Strong PCA Dimension Reduction","authors":"H Hosoya, Hyvärinen","year":"2016","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hosoya and Hyvärinen, 2017] Haruo Hosoya and Aapo Hyvärinen. A mixture of sparse coding models explaining properties of face neurons related to holistic and parts-based processing. PLoS Computational Biology, 13(7):e1005667, July 2017.","title":"A mixture of sparse coding models explaining properties of face neurons related to holistic and parts-based processing","authors":"Hyvärinen Hosoya, Haruo Hosoya, Aapo Hyvärinen","year":"2017","venue":"PLoS Computational Biology","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Ba, 2015] D Kingma and J Ba. Adam: A method for stochastic optimization. In International Con- ference on Learning Representations, 2015.","title":"Adam: A method for stochastic optimization","authors":"D Kingma, Ba","year":"2015","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kingma and Welling, 2014] D P Kingma and M Welling. Auto-encoding variational bayes. In International Con- ference on Learning Representations, 2014.","title":"Auto-encoding variational bayes","authors":"D P Kingma, M Welling","year":"2014","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kingma et al., 2014] D P Kingma, S Mohamed, and D J Rezende. Semi-supervised learning with deep generative models. Advances in neural information processing sys- tems, 2014.","title":"Semi-supervised learning with deep generative models","authors":"S D P Kingma, D J Mohamed, Rezende","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lample et al., 2017] Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, and Marc'Aurelio Ranzato. Fader Networks:Manipulating Im- ages by Sliding Attributes. Advances in neural information processing systems, pages 5967-5976, 2017.","title":"Fader Networks:Manipulating Images by Sliding Attributes","authors":"Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, Marc'aurelio Ranzato","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[LeCun et al., 2004] Yann LeCun, Fu Jie Huang, and Léon Bottou. Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting. Computer Vision and Pattern Recognition, 2004.","title":"Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting","authors":"Yann Lecun, Jie Fu, Léon Huang, Bottou","year":"2004","venue":"Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mathieu et al., 2016] Michaël Mathieu, Junbo Jake Zhao, Pablo Sprechmann, Aditya Ramesh, and Yann LeCun. Dis- entangling factors of variation in deep representation using adversarial training. Advances in neural information pro- cessing systems, 2016.","title":"Disentangling factors of variation in deep representation using adversarial training","authors":"Michaël Mathieu, Jake Junbo, Pablo Zhao, Aditya Sprechmann, Yann Ramesh, Lecun","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Reed et al., 2015] Scott E Reed, Yi Zhang, Yuting Zhang, and Honglak Lee. Deep Visual Analogy-Making. Advances in neural information processing systems, 2015.","title":"Deep Visual Analogy-Making","authors":"Yi Scott E Reed, Yuting Zhang, Honglak Zhang, Lee","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schüldt et al., 2004] Christian Schüldt, Ivan Laptev, and Bar- bara Caputo. Recognizing Human Actions -A Local SVM Approach. In International Conference on Pattern Recog- nition, pages 32-36 Vol.3. IEEE, 2004.","title":"Christian Schüldt, Ivan Laptev, and Barbara Caputo. Recognizing Human Actions -A Local SVM Approach","authors":"Schüldt","year":"2004","venue":"International Conference on Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Siddharth et al., 2017] N Siddharth, Brooks Paige, Jan- Willem van de Meent, Alban Desmaison, Frank Wood, Noah D Goodman, Pushmeet Kohli, and Philip H S Torr. Learning Disentangled Representations with Semi- Supervised Deep Generative Models. Advances in neural information processing systems, 2017.","title":"Learning Disentangled Representations with Semi-Supervised Deep Generative Models","authors":"Brooks Siddharth, Jan-Willem Paige, Alban Van De Meent, Frank Desmaison, Wood, Pushmeet Noah D Goodman, Philip H S Kohli, Torr","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tenenbaum and Freeman, 2000] J B Tenenbaum and W T Freeman. Separating style and content with bilinear models. Neural Computation, 12(6):1247-1283, June 2000.","title":"Separating style and content with bilinear models","authors":"J B Tenenbaum, W T Freeman","year":"2000","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2017] Chaoyue Wang, Chaohui Wang, Chang Xu, and Dacheng Tao. Tag Disentangled Generative Adver- sarial Networks for Object Image Re-rendering. Interna- tional Joint Conference on Artificial Intelligence, August 2017.","title":"Tag Disentangled Generative Adversarial Networks for Object Image Re-rendering","authors":"Wang","year":"2017","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2015] Jimei Yang, Scott Reed, Ming-Hsuan Yang, and Honglak Lee. Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis. Advances in neural information processing systems, 2015.","title":"Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis","authors":"Yang","year":"2015","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}