{"type":"ScholarlyArticle","title":"Toward a neuro-inspired creative decoder","authors":[{"name":"Payel Das","slug":"payel-das","given":"Payel","family":"Das","affiliation":"IBM Research , Yorktown Heights , NY , USA"},{"name":"Brian Quanz","slug":"brian-quanz","given":"Brian","family":"Quanz","affiliation":"IBM Research , Yorktown Heights , NY , USA"},{"name":"Pin-Yu Chen","slug":"pin-yu-chen","given":"Pin-Yu","family":"Chen","affiliation":"IBM Research , Yorktown Heights , NY , USA"},{"name":"Jae-wook Ahn","slug":"jae-wook-ahn","given":"Jae-wook","family":"Ahn","affiliation":"IBM Research , Yorktown Heights , NY , USA"},{"name":"Dhruv Shah","slug":"dhruv-shah","given":"Dhruv","family":"Shah","affiliation":"IBM Research , Yorktown Heights , NY , USA"}],"abstract":"Creativity, a process that generates novel and meaningful ideas, involves increased association between task-positive (control) and task-negative (default) networks in the human brain. Inspired by this seminal finding, in this study we propose a creative decoder within a deep generative framework, which involves direct modulation of the neuronal activation pattern after sampling from the learned latent space. The proposed approach is fully unsupervised and can be used off- the-shelf. Several novelty metrics and human evaluation were used to evaluate the creative capacity of the deep decoder. Our experiments on different image datasets (MNIST, FMNIST, MNIST+FMNIST, WikiArt and CelebA) reveal that atypical co-activation of highly activated and weakly activated neurons in a deep decoder promotes generation of novel and meaningful artifacts.","doi":"10.24963/ijcai.2020/381","url":"https://ijcai.org/proceedings/2020/381","pdf":"https://ijcai.org/proceedings/2020/0381.pdf","markdown":"https://ijcai.org/proceedings/2020/381.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/381","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":381,"pages":{"first":2746,"last":2753},"keywords":["Multidisciplinary Topics and Applications: Art and Music","Machine Learning: Deep Learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25234/file_25234.m3u8","length_sec":266,"provider":"hls"}],"references":[{"position":1,"raw":"[Anticevic et al., 2012] Alan Anticevic, Michael W Cole, John D Murray, Philip R Corlett, Xiao-Jing Wang, and John H Krystal. The role of default network deactivation in cognition and disease. Trends in cognitive sciences, 16(12):584-592, 2012.","title":"The role of default network deactivation in cognition and disease","authors":"Alan Anticevic, John D Michael W Cole, Philip R Murray, Xiao-Jing Corlett, John H Wang, Krystal","year":"2012","venue":"Trends in cognitive sciences","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.","title":"Neural machine translation by jointly learning to align and translate","authors":"Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beaty et al., 2016] Roger E Beaty, Mathias Benedek, Paul J Silvia, and Daniel L Schacter. Creative cognition and brain network dynamics. Trends in cognitive sciences, 20(2):87- 95, 2016.","title":"Creative cognition and brain network dynamics","authors":"Mathias Roger E Beaty, Paul J Benedek, Daniel L Silvia, Schacter","year":"2016","venue":"Trends in cognitive sciences","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Beaty et al., 2017] Roger E Beaty, Alexander P Christensen, Mathias Benedek, Paul J Silvia, and Daniel L Schacter. Creative constraints: Brain activity and network dynamics underlying semantic interference during idea production. Neuroimage, 148:189-196, 2017.","title":"Creative constraints: Brain activity and network dynamics underlying semantic interference during idea production","authors":"Alexander P Roger E Beaty, Mathias Christensen, Paul J Benedek, Daniel L Silvia, Schacter","year":"2017","venue":"Neuroimage","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Beaty et al., 2018] Roger E Beaty, Yoed N Kenett, Alexan- der P Christensen, Monica D Rosenberg, Mathias Benedek, Qunlin Chen, Andreas Fink, Jiang Qiu, Thomas R Kwapil, Michael J Kane, et al. Robust prediction of individual creative ability from brain functional connectivity. Pro- ceedings of the National Academy of Sciences, pages 1087- 1092, 2018.","title":"Robust prediction of individual creative ability from brain functional connectivity","authors":"Roger E Beaty, N Yoed, Alexander P Kenett, Monica D Christensen, Mathias Rosenberg, Qunlin Benedek, Andreas Chen, Jiang Fink, Qiu, Michael J Thomas R Kwapil, Kane","year":"2018","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Boden, 2004] Margaret A Boden. The creative mind: Myths and mechanisms. Routledge, 2004.","title":"The creative mind: Myths and mechanisms","authors":"Margaret A Boden","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cherti and Kégl, 2016] Akın Kazakçıand Mehdi Cherti and Balázs Kégl. Digits that are not: Generating new types through deep neural nets. arXiv preprint arXiv:1606.04345, 2016.","title":"Digits that are not: Generating new types through deep neural nets","authors":"Kégl Cherti, Akın Kazakçıand, Mehdi Cherti, Balázs Kégl","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"Cherti et al., ] Mehdi Cherti, Balázs Kégl, and Akin Kazakçı. Out-of-class novelty generation: an experimental founda- tion. In Tools with Artificial Intelligence (ICTAI), 2017 IEEE 29th International Conference on.","title":"Out-of-class novelty generation: an experimental foundation","authors":"Cherti","year":"2017","venue":"Tools with Artificial Intelligence (ICTAI)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Clouâtre and Demers, 2019] Louis Clouâtre and Marc De- mers. Figr: Few-shot image generation with reptile. arXiv preprint arXiv:1901.02199, 2019.","title":"Louis Clouâtre and Marc Demers. Figr: Few-shot image generation with reptile","authors":"Demers Clouâtre","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Diedrich et al., 2015] Jennifer Diedrich, Mathias Benedek, Emanuel Jauk, and Aljoscha C Neubauer. Are creative ideas novel and useful? Psychology of Aesthetics, Creativity, and the Arts, 9(1):35, 2015.","title":"Are creative ideas novel and useful?","authors":"Jennifer Diedrich, Mathias Benedek, Emanuel Jauk, Aljoscha C Neubauer","year":"2015","venue":"Psychology of Aesthetics, Creativity, and the Arts","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ding et al., 2014] Xuemei Ding, Yuhua Li, Ammar Bela- treche, and Liam P Maguire. An experimental evaluation of novelty detection methods. Neurocomputing, 135:313-327, 2014.","title":"An experimental evaluation of novelty detection methods","authors":"Xuemei Ding, Yuhua Li, Ammar Belatreche, Liam P Maguire","year":"2014","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[DiPaola and Gabora, 2009] Steve DiPaola and Liane Gab- ora. Incorporating characteristics of human creativity into an evolutionary art algorithm. Genetic Programming and Evolvable Machines, 10(2):97-110, 2009.","title":"Incorporating characteristics of human creativity into an evolutionary art algorithm","authors":"Gabora Dipaola","year":"2009","venue":"Genetic Programming and Evolvable Machines","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dosovitskiy et al., 2016] Alexey Dosovitskiy, Jost Tobias Springenberg, Maxim Tatarchenko, and Thomas Brox. Learning to generate chairs, tables and cars with convo- lutional networks. IEEE transactions on pattern analysis and machine intelligence, 39(4):692-705, 2016.","title":"Learning to generate chairs, tables and cars with convolutional networks","authors":"Alexey Dosovitskiy, Jost Tobias Springenberg, Maxim Tatarchenko, Thomas Brox","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Elgammal et al., 2017] Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. Can: Cre- ative adversarial networks, generating\" art\" by learning about styles and deviating from style norms. arXiv preprint arXiv:1706.07068, 2017.","title":"Can: Creative adversarial networks, generating\" art\" by learning about styles and deviating from style norms","authors":"Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, Marian Mazzone","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Frankle and Carbin, 2018] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, train- able neural networks. arXiv preprint arXiv:1803.03635, 2018.","title":"The lottery ticket hypothesis: Finding sparse, trainable neural networks","authors":"Carbin Frankle, Jonathan Frankle, Michael Carbin","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gao et al., 2017] Zhenni Gao, Delong Zhang, Aiying Liang, Bishan Liang, Zengjian Wang, Yuxuan Cai, Junchao Li, Mengxia Gao, Xiaojin Liu, Song Chang, et al. Exploring the associations between intrinsic brain connectivity and creative ability using functional connectivity strength and connectome analysis. Brain connectivity, 7(9):590-601, 2017.","title":"Exploring the associations between intrinsic brain connectivity and creative ability using functional connectivity strength and connectome analysis","authors":"Gao","year":"2017","venue":"Brain connectivity","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gatys et al., 2015] Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015.","title":"A neural algorithm of artistic style","authors":"Leon A Gatys, Alexander S Ecker, Matthias Bethge","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sher- jil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information pro- cessing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Graf and Banzhaf, 1995] Jeanine Graf and Wolfgang Banzhaf. Interactive evolution of images. In Evolutionary Programming, pages 53-65, 1995.","title":"Interactive evolution of images","authors":"Banzhaf Graf, Jeanine Graf, Wolfgang Banzhaf","year":"1995","venue":"Evolutionary Programming","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hendrycks and Gimpel, 2016] Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out- of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016.","title":"A baseline for detecting misclassified and outof-distribution examples in neural networks","authors":"Gimpel Hendrycks, Dan Hendrycks, Kevin Gimpel","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Jennings, 2010] Kyle E Jennings. Developing creativity: Ar- tificial barriers in artificial intelligence. Minds and Ma- chines, 20(4):489-501, 2010.","title":"Developing creativity: Artificial barriers in artificial intelligence","authors":"Jennings Kyle","year":"2010","venue":"Minds and Machines","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kégl et al., 2018] Balázs Kégl, Mehdi Cherti, and Akın Kazakçı. Spurious samples in deep generative models: bug or feature? arXiv preprint arXiv:1810.01876, 2018.","title":"and Akın Kazakçı. Spurious samples in deep generative models: bug or feature","authors":"Kégl","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kingma et al., 2016] Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow. In Advances in neural information processing systems, pages 4743-4751, 2016.","title":"Improved variational inference with inverse autoregressive flow","authors":"Tim Durk P Kingma, Rafal Salimans, Xi Jozefowicz, Ilya Chen, Max Sutskever, Welling","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Kliger and Fleishman, 2018] Mark Kliger and Shachar Fleishman. Novelty detection with gan. arXiv preprint arXiv:1802.10560, 2018.","title":"Novelty detection with gan","authors":"Fleishman Kliger, Mark Kliger, Shachar Fleishman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lake et al., 2015] Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learn- ing through probabilistic program induction. Science, 350(6266):1332-1338, 2015.","title":"Human-level concept learning through probabilistic program induction","authors":"Ruslan Brenden M Lake, Joshua B Salakhutdinov, Tenenbaum","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Lopez and Tucker, 2018] CS Lopez and CE Tucker. Human validation of computer vs human generated design sketches. ASME Paper No. DETC2018-85698, 2018.","title":"Human validation of computer vs human generated design sketches","authors":"Tucker ; Cs Lopez, C E Lopez, Tucker","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Nguyen et al., 2015] Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Innovation engines: Automated creativity and improved stochastic optimization via deep learning. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pages 959-966. ACM, 2015.","title":"Innovation engines: Automated creativity and improved stochastic optimization via deep learning","authors":"Anh Mai Nguyen, Jason Yosinski, Jeff Clune","year":"2015","venue":"Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Nichol et al., 2018] Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018.","title":"On first-order meta-learning algorithms","authors":"Alex Nichol, Joshua Achiam, John Schulman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Rezende and Mohamed, 2015] Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770, 2015.","title":"Variational inference with normalizing flows","authors":"Mohamed ; Rezende, Danilo Jimenez, Rezende, Shakir Mohamed","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Rezende et al., 2016] Danilo Jimenez Rezende, Shakir Mo- hamed, Ivo Danihelka, Karol Gregor, and Daan Wierstra. One-shot generalization in deep generative models. arXiv preprint arXiv:1603.05106, 2016.","title":"One-shot generalization in deep generative models","authors":"Danilo Jimenez Rezende, Shakir Mohamed, Ivo Danihelka, Karol Gregor, Daan Wierstra","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"Salimans et al., ] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Im- proved techniques for training gans. In Advances in neural information processing systems.","title":"Improved techniques for training gans","authors":"Salimans","year":null,"venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Sbai et al., 2018] Othman Sbai, Mohamed Elhoseiny, An- toine Bordes, Yann LeCun, and Camille Couprie. De- sign: Design inspiration from generative networks. In Pro- ceedings of the European Conference on Computer Vision (ECCV), pages 0-0, 2018.","title":"Design: Design inspiration from generative networks","authors":"Othman Sbai, Mohamed Elhoseiny, Antoine Bordes, Yann Lecun, Camille Couprie","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Shi et al., 2018] Liang Shi, Jiangzhou Sun, Yunman Xia, Zhiting Ren, Qunlin Chen, Dongtao Wei, Wenjing Yang, and Jiang Qiu. Large-scale brain network connectivity underlying creativity in resting-state and task fmri: cooper- ation between default network and frontal-parietal network. Biological psychology, 135:102-111, 2018.","title":"Large-scale brain network connectivity underlying creativity in resting-state and task fmri: cooperation between default network and frontal-parietal network","authors":"Liang Shi, Jiangzhou Sun, Yunman Xia, Zhiting Ren, Qunlin Chen, Dongtao Wei, Wenjing Yang, Jiang Qiu","year":"2018","venue":"Biological psychology","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Spreng et al., 2015] R Nathan Spreng, Kathy D Gerlach, Gary R Turner, and Daniel L Schacter. Autobiographi- cal planning and the brain: activation and its modulation by qualitative features. Journal of cognitive neuroscience, 27(11):2147-2157, 2015.","title":"Autobiographical planning and the brain: activation and its modulation by qualitative features","authors":"Nathan Spreng, Kathy D Gerlach, Gary R Turner, Daniel L Schacter","year":"2015","venue":"Journal of cognitive neuroscience","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Szegedy et al., 2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 1-9, 2015.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2015","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wang et al., 2018] Huan-gang Wang, Xin Li, and Tao Zhang. Generative adversarial network based novelty detection us- ing minimized reconstruction error. Frontiers of Informa- tion Technology & Electronic Engineering, 19(1):116-125, 2018.","title":"Generative adversarial network based novelty detection using minimized reconstruction error","authors":"Huan-Gang Wang, Xin Li, Tao Zhang","year":"2018","venue":"Frontiers of Information Technology & Electronic Engineering","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}