{"type":"ScholarlyArticle","title":"Towards Creativity Characterization of Generative Models via Group-Based Subset Scanning","authors":[{"name":"Celia Cintas","slug":"celia-cintas","given":"Celia","family":"Cintas","affiliation":"IBM Research Africa , Nairobi , Kenya"},{"name":"Payel Das","slug":"payel-das","given":"Payel","family":"Das","affiliation":"IBM Research , Yorktown Heights , NY , USA"},{"name":"Brian Quanz","slug":"brian-quanz","given":"Brian","family":"Quanz","affiliation":"IBM Research , Yorktown Heights , NY , USA"},{"name":"Girmaw Abebe Tadesse","slug":"girmaw-abebe-tadesse","given":"Girmaw Abebe","family":"Tadesse","affiliation":"IBM Research Africa , Nairobi , Kenya"},{"name":"Skyler Speakman","slug":"skyler-speakman","given":"Skyler","family":"Speakman","affiliation":"IBM Research Africa , Nairobi , Kenya"},{"name":"Pin-Yu Chen","slug":"pin-yu-chen","given":"Pin-Yu","family":"Chen","affiliation":"IBM Research , Yorktown Heights , NY , USA"}],"abstract":"Deep generative models, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), have been employed widely in computational creativity research. However, such models discourage out-of-distribution generation to avoid spurious sample generation, thereby limiting their creativity. Thus, incorporating research on human creativity into generative deep learning techniques presents an opportunity to make their outputs more compelling and human-like.  As we see the emergence of generative models directed toward creativity research, a need for machine learning-based surrogate metrics to characterize creative output from these models is imperative. \r\nWe propose group-based subset scanning to identify, quantify,  and characterize creative processes by detecting a subset of anomalous node-activations in the hidden layers of the generative models. \r\nOur experiments on the standard image benchmarks and their ``creatively generated'' variants reveal that the proposed subset scores distribution is more useful for detecting novelty in creative processes in the activation space rather than the pixel space. Further, we found that creative samples generate larger subsets of anomalies than normal or non-creative samples across datasets. The node activations highlighted during the creative decoding process are different from those responsible for the normal sample generation. Lastly, we assess if the images from the subsets selected by our method were also found creative by human evaluators, presenting a link between creativity perception in humans and node activations within deep neural nets.","doi":"10.24963/ijcai.2022/683","url":"https://ijcai.org/proceedings/2022/683","pdf":"https://ijcai.org/proceedings/2022/0683.pdf","markdown":"https://ijcai.org/proceedings/2022/683.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/683","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Research Papers","track":"AI and Arts","number":683,"pages":{"first":4929,"last":4935},"keywords":["Methods and resources: Machine learning, deep learning, neural models, reinforcement learning","Theory and philosophy of arts and creativity in AI systems: Evaluation of artistic or creative outputs, or of systems","Theory and philosophy of arts and creativity in AI systems: Support of human creativity","Theory and philosophy of arts and creativity in AI systems: Autonomous creative or artistic AI"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987163","length_sec":94,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985306","length_sec":829,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Amabile, 1982] Teresa M Amabile. Social psychology of creativity: A consensual assessment technique. Journal of personality and social psychology, 43(5):997, 1982.","title":"Social psychology of creativity: A consensual assessment technique","authors":"Teresa M Amabile","year":"1982","venue":"Journal of personality and social psychology","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boden, 2004] Margaret A Boden. The creative mind: Myths and mechanisms. Routledge, 2004.","title":"The creative mind: Myths and mechanisms","authors":"Margaret A Boden","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and Neill, 2014] Feng Chen and Daniel B. Neill. Non- parametric scan statistics for event detection and forecast- ing in heterogeneous social media graphs. In KDD '14, pages 1166-1175, 2014.","title":"Nonparametric scan statistics for event detection and forecasting in heterogeneous social media graphs","authors":"Chen, Neill ; ] Feng Chen, Daniel B Neill","year":"2014","venue":"KDD '14","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cherti et al., 2017] Mehdi Cherti, Balázs Kégl, and Akin Kazakc ¸ı. Out-of-class novelty generation: an experimental foundation. In Tools with Artificial Intelligence (ICTAI), 2017 IEEE 29th International Conference on, 2017.","title":"Out-of-class novelty generation: an experimental foundation","authors":"Mehdi Cherti, Balázs Kégl, Akin Kazakc ¸ı","year":"2017","venue":"Tools with Artificial Intelligence (ICTAI)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cintas et al., 2020] Celia Cintas, Skyler Speakman, Victor Akinwande, William Ogallo, Komminist Weldemariam, Srihari Sridharan, and Edward McFowland. Detecting ad- versarial attacks via subset scanning of autoencoder acti- vations and reconstruction error. In IJCAI, pages 876-882, 2020.","title":"Detecting adversarial attacks via subset scanning of autoencoder activations and reconstruction error","authors":"Celia Cintas, Skyler Speakman, Victor Akinwande, William Ogallo, Komminist Weldemariam, Srihari Sridharan, Edward Mcfowland","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/122"},{"position":6,"raw":"[Das et al., 2020] Payel Das, Brian Quanz, Pin-Yu Chen, Jae-wook Ahn, and Dhruv Shah. Toward a neuro-inspired creative decoder. In Christian Bessiere, editor, Proceed- ings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 2746-2753. In- ternational Joint Conferences on Artificial Intelligence Or- ganization, 7 2020. Main track.","title":"Toward a neuro-inspired creative decoder","authors":"Payel Das, Brian Quanz, Pin-Yu Chen, Jae-Wook Ahn, Dhruv Shah","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20","doi":null,"url":null,"internal_url":"/proceedings/2020/381"},{"position":7,"raw":"[Ding et al., 2014] Xuemei Ding, Yuhua Li, Ammar Bela- treche, and Liam P Maguire. An experimental evaluation of novelty detection methods. Neurocomputing, 135:313- 327, 2014.","title":"An experimental evaluation of novelty detection methods","authors":"Xuemei Ding, Yuhua Li, Ammar Belatreche, Liam P Maguire","year":"2014","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Franceschelli and Musolesi, 2021] Giorgio Franceschelli and Mirco Musolesi. Creativity and machine learning: A survey. arXiv preprint arXiv:2104.02726, 2021.","title":"Creativity and machine learning: A survey","authors":"Musolesi Franceschelli","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grace and Maher, 2019] Kazjon Grace and Mary Lou Ma- her. Expectation-Based Models of Novelty for Evaluating Computational Creativity, pages 195-209. Springer Inter- national Publishing, Cham, 2019.","title":"Kazjon Grace and Mary Lou Maher. Expectation-Based Models of Novelty for Evaluating Computational Creativity","authors":"Maher Grace","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Grace et al., 2015] Kazjon Grace, Mary Lou Maher, Dou- glas Fisher, and Katherine Brady. Data-intensive evalua- tion of design creativity using novelty, value, and surprise. International Journal of Design Creativity and Innovation, 3(3-4):125-147, 2015.","title":"Data-intensive evaluation of design creativity using novelty, value, and surprise","authors":"Grace","year":"2015","venue":"International Journal of Design Creativity and Innovation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma et al., 2016] Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow. In Advances in neural information processing sys- tems, pages 4743-4751, 2016. Kliger and Fleishman, 2018] Mark Kliger and Shachar Fleishman. Novelty detection with gan. arXiv preprint arXiv:1802.10560, 2018.","title":"Improved variational inference with inverse autoregressive flow","authors":"Tim Durk P Kingma, Rafal Salimans, Xi Jozefowicz, Ilya Chen, Max Sutskever, Welling","year":"2016","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lake et al., 2015] Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learn- ing through probabilistic program induction. Science, 350(6266):1332-1338, 2015.","title":"Human-level concept learning through probabilistic program induction","authors":"Ruslan Brenden M Lake, Joshua B Salakhutdinov, Tenenbaum","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lamb et al., 2018] Carolyn Lamb, Daniel G Brown, and Charles LA Clarke. Evaluating computational creativity: An interdisciplinary tutorial. ACM Computing Surveys (CSUR), 51(2):1-34, 2018.","title":"Evaluating computational creativity: An interdisciplinary tutorial","authors":"Carolyn Lamb, Daniel G Brown, Charles La Clarke","year":"2018","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2008] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In 2008 eighth ieee international conference on data mining, pages 413-422. IEEE, 2008.","title":"Isolation forest","authors":"Tony Fei, Kai Liu, Ming Ting, Zhi-Hua Zhou","year":"2008","venue":"2008 eighth ieee international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lopez and Tucker, 2018] CS Lopez and CE Tucker. Hu- man validation of computer vs human generated design sketches. ASME Paper No. DETC2018-85698, 2018.","title":"CS Lopez and CE Tucker. Human validation of computer vs human generated design sketches","authors":"Tucker Lopez","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[McFowland III et al., 2013] Edward McFowland III, Skyler D Speakman, and Daniel B Neill. Fast generalized subset scan for anomalous pattern detection. The Journal of Machine Learning Research, 14(1):1533-1561, Jun 2013.","title":"Fast generalized subset scan for anomalous pattern detection","authors":"Edward Mcfowland, Iii, Skyler D Speakman, Daniel B Neill","year":"2013","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Neill, 2012] Daniel B. Neill. Fast subset scan for spatial pattern detection. Journal of the Royal Statistical Society (Series B: Statistical Methodology), 74(2):337-360, 2012.","title":"Fast subset scan for spatial pattern detection","authors":"B Daniel, Neill","year":"2012","venue":"Journal of the Royal Statistical Society (Series B: Statistical Methodology)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rezende and Mohamed, 2015] Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770, 2015.","title":"Variational inference with normalizing flows","authors":"Mohamed ; Rezende, Danilo Jimenez, Rezende, Shakir Mohamed","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Said-Metwaly et al., 2017] Sameh Said-Metwaly, Wim Van den Noortgate, and Eva Kyndt. Approaches to measuring creativity: A systematic literature review. Cre- ativity. Theories-Research-Applications, 4(2):238-275, 2017.","title":"Approaches to measuring creativity: A systematic literature review","authors":"Sameh Said-Metwaly, Wim Van Den Noortgate, Eva Kyndt","year":"2017","venue":"Creativity. Theories-Research-Applications","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Salimans et al., 2016] Tim Salimans, Ian Goodfellow, Woj- ciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NIPS, pages 2234-2242, 2016.","title":"Improved techniques for training gans","authors":"Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Speakman et al., 2016] Skyler Speakman, Sriram So- manchi, Edward McFowland III, and Daniel B. Neill. Penalized fast subset scanning. Journal of Computational and Graphical Statistics, 25(2):382-404, 2016.","title":"Penalized fast subset scanning","authors":"Skyler Speakman, Sriram Somanchi, Edward Mcfowland, Iii, Daniel B Neill","year":"2016","venue":"Journal of Computational and Graphical Statistics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Tan et al., 2019] Wei Ren Tan, Chee Seng Chan, Hernan Aguirre, and Kiyoshi Tanaka. Improved artgan for condi- tional synthesis of natural image and artwork. IEEE Trans- actions on Image Processing, 28(1):394-409, 2019.","title":"Improved artgan for conditional synthesis of natural image and artwork","authors":"Chee Wei Ren Tan, Hernan Seng Chan, Kiyoshi Aguirre, Tanaka","year":"2019","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2018] Huan-gang Wang, Xin Li, and Tao Zhang. Generative adversarial network based novelty detection using minimized reconstruction error. Fron- tiers of Information Technology & Electronic Engineering, 19(1):116-125, 2018.","title":"Generative adversarial network based novelty detection using minimized reconstruction error","authors":"Huan-Gang Wang, Xin Li, Tao Zhang","year":"2018","venue":"Frontiers of Information Technology & Electronic Engineering","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}