{"type":"ScholarlyArticle","title":"Unsupervised Disentangled Representation Learning with Analogical Relations","authors":[{"name":"Zejian Li","slug":"zejian-li","given":"Zejian","family":"Li","affiliation":"College of Computer Science , Zhejiang University , Hangzhou 310027 , China"},{"name":"Yongchuan Tang","slug":"yongchuan-tang","given":"Yongchuan","family":"Tang","affiliation":"College of Computer Science , Zhejiang University , Hangzhou 310027 , China"},{"name":"Yongxing He","slug":"yongxing-he","given":"Yongxing","family":"He","affiliation":"College of Computer Science , Zhejiang University , Hangzhou 310027 , China"}],"abstract":"Learning the disentangled representation of interpretable generative factors of data is one of the foundations to allow artificial intelligence to think like people. In this paper, we propose the analogical training strategy for the unsupervised disentangled representation learning in generative models. The analogy is one of the typical cognitive processes, and our proposed strategy is based on the observation that sample pairs in which one is different from the other in one specific generative factor show the same analogical relation. Thus, the generator is trained to generate sample pairs from which a designed classifier can identify the underlying analogical relation. In addition, we propose a disentanglement metric called the subspace score, which is inspired by subspace learning methods and does not require supervised information. Experiments show that our proposed training strategy allows the generative models to find the disentangled factors, and that our methods can give competitive performances as compared with the state-of-the-art methods.","doi":"10.24963/ijcai.2018/335","url":"https://ijcai.org/proceedings/2018/335","pdf":"https://ijcai.org/proceedings/2018/0335.pdf","markdown":"https://ijcai.org/proceedings/2018/335.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/335","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":335,"pages":{"first":2418,"last":2424},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Learning Generative Models","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Aubry et al., 2014] Mathieu Aubry, Daniel Maturana, Alexei A. Efros, Bryan C. Russell, and Josef Sivic. Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3762-3769. IEEE, 2014.","title":"Seeing 3d chairs: exemplar part-based 2d-3d alignment using a large dataset of cad models","authors":"Aubry Mathieu, Daniel Maturana, Alexei A Efros, Bryan C Russell, Josef Sivic","year":"2014","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1828, 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: In- terpretable representation learning by information maxi- mizing generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2172-2180, 2016.","title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","authors":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Denton and Birodkar, 2017] Emily Denton and Vighnesh Birodkar. Unsupervised learning of disentangled repre- sentations from video. In Advances in Neural Information Processing Systems, pages 4417-4426.","title":"Emily Denton and Vighnesh Birodkar. Unsupervised learning of disentangled representations from video","authors":"Birodkar Denton","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Elhamifar and Vidal, 2013] Ehsan Elhamifar and Rene Vi- dal. Sparse subspace clustering: Algorithm, theory, and applications. IEEE transactions on pattern analysis and machine intelligence, 35(11):2765-2781, 2013.","title":"Sparse subspace clustering: Algorithm, theory, and applications","authors":"Ehsan Elhamifar, Rene Vidal","year":"2013","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sher- jil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information pro- cessing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gulrajani et al., 2017] Ishaan Gulrajani, Faruk Ahmed, Mar- tin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. arXiv preprint arXiv:1704.00028, 2017.","title":"Improved training of wasserstein gans","authors":"Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Aaron Vincent Dumoulin, Courville","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gust et al., 2008] Helmar Gust, Ulf Krumnack, Kai-Uwe Kühnberger, and Angela Schwering. Analogical reasoning: A core of cognition. In Proceedings of the 31st Annual Ger- man Conference on AI, KI 2008, volume 22, pages 8-12. Springer, 2008.","title":"Analogical reasoning: A core of cognition","authors":"Gust","year":"2008","venue":"Proceedings of the 31st Annual German Conference on AI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Higgins et al., 2017] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. B-vae: Learn- ing basic visual concepts with a constrained variational framework. In International Conference on Learning Rep- resentation, 2017.","title":"B-vae: Learning basic visual concepts with a constrained variational framework","authors":"Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner","year":"2017","venue":"International Conference on Learning Representation","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Ba, 2014] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"Diederik Kingma, Jimmy Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Welling, 2013] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kulkarni et al., 2015] Tejas D. Kulkarni, William F. Whit- ney, Pushmeet Kohli, and Joshua B. Tenenbaum. Deep convolutional inverse graphics network. In Advances in Neural Information Processing Systems, pages 2539-2547.","title":"Deep convolutional inverse graphics network","authors":"D Tejas, William F Kulkarni, Pushmeet Whitney, Joshua B Kohli, Tenenbaum","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Curran Associates, Inc., 2015.","title":null,"authors":"Curran Associates, Inc","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kumar et al., 2017] Abhishek Kumar, Prasanna Sattigeri, and Avinash Balakrishnan. Variational inference of disen- tangled latent concepts from unlabeled observations. arXiv preprint arXiv:1711.00848, 2017.","title":"Variational inference of disentangled latent concepts from unlabeled observations","authors":"Abhishek Kumar, Prasanna Sattigeri, Avinash Balakrishnan","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), pages 3730-3738. IEEE, 2015.","title":"Deep learning face attributes in the wild","authors":"Ziwei Liu, Ping Luo, Xiaogang Wang, Xiaoou Tang","year":"2015","venue":"Proceedings of International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nilsback and Zisserman, 2008] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Com- puter Vision, Graphics & Image Processing, 2008., pages 722-729. IEEE, 2008.","title":"Automated flower classification over a large number of classes","authors":"Maria-Elena Nilsback, Andrew Zisserman","year":"2008","venue":"Indian Conference on Computer Vision, Graphics & Image Processing","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Odena et al., 2016] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with aux- iliary classifier gans. arXiv preprint arXiv:1610.09585, 2016.","title":"Conditional image synthesis with auxiliary classifier gans","authors":"Odena","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Peng et al., 2017] Xi Peng, Xiang Yu, Kihyuk Sohn, Dimitris N Metaxas, and Manmohan Chandraker. Reconstruction-based disentanglement for pose-invariant face recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 1632-1641, 2017.","title":"Reconstruction-based disentanglement for pose-invariant face recognition","authors":"Xi Peng, Xiang Yu, Kihyuk Sohn, Dimitris N Metaxas, Manmohan Chandraker","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tropp and Gilbert, 2007] Joel A. Tropp and Anna C. Gilbert. Signal recovery from random measurements via orthogo- nal matching pursuit. IEEE Transactions on Information Theory, 53(12):4655-4666, 2007.","title":"Signal recovery from random measurements via orthogonal matching pursuit","authors":"Joel A Tropp, Anna C Gilbert","year":"2007","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017] Chaoyue Wang, Chaohui Wang, Chang Xu, and Dacheng Tao. Tag disentangled generative adver- sarial network for object image re-rendering. In Proceed- ings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI, pages 2901-2907, 2017.","title":"Tag disentangled generative adversarial network for object image re-rendering","authors":"Wang","year":"2017","venue":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/404"},{"position":26,"raw":"[Worrall et al., 2017] Daniel E. Worrall, Stephan J. Garbin, Daniyar Turmukhambetov, and Gabriel J. Brostow. Inter- pretable transformations with encoder-decoder networks. In The IEEE International Conference on Computer Vision (ICCV), volume 4, pages 5726-5735, 2017.","title":"Interpretable transformations with encoder-decoder networks","authors":"E Daniel, Stephan J Worrall, Daniyar Garbin, Gabriel J Turmukhambetov, Brostow","year":"2017","venue":"The IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}