{"type":"ScholarlyArticle","title":"Adapting Deep Network Features to Capture Psychological Representations: An Abridged Report","authors":[{"name":"Joshua C. Peterson","slug":"joshua-c-peterson","given":"Joshua C.","family":"Peterson","affiliation":"UC Berkeley"},{"name":"Joshua T. Abbott","slug":"joshua-t-abbott","given":"Joshua T.","family":"Abbott","affiliation":"UC Berkeley"},{"name":"Thomas L. Griffiths","slug":"thomas-l-griffiths","given":"Thomas L.","family":"Griffiths","affiliation":"UC Berkeley"}],"abstract":"Deep neural networks have become increasingly successful at solving classic perception problems (e.g., recognizing objects), often reaching or surpassing human-level accuracy. In this abridged report of Peterson et al. [2016], we examine the relationship between the image representations learned by these networks and those of humans. We find that deep features learned in service of object classification account for a significant amount of the variance in human similarity judgments for a set of animal images. However, these features do not appear to capture some key qualitative aspects of human representations. To close this gap, we present a method for adapting deep features to align with human similarity judgments, resulting in image representations that can potentially be used to extend the scope of psychological experiments and inform human-centric AI.","doi":"10.24963/ijcai.2017/697","url":"https://ijcai.org/proceedings/2017/697","pdf":"https://ijcai.org/proceedings/2017/0697.pdf","markdown":"https://ijcai.org/proceedings/2017/697.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/697","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Best Sister Conferences","track":"Best Sister Conferences","number":697,"pages":{"first":4934,"last":4938},"keywords":["Artificial Intelligence: knowledge representation and reasoning","Artificial Intelligence: computer vision","Artificial Intelligence: cognitive science","Artificial Intelligence: artificial intelligence"],"videos":[],"references":[{"position":1,"raw":"Pulkit Agrawal, Dustin Stansbury, Jitendra Malik, and Jack L. Gallant. Pixels to Voxels: Modeling Visual Representa- tion in the Human Brain. arXiv preprint arXiv:1407.5104, 2014.","title":"Pixels to Voxels: Modeling Visual Representation in the Human Brain","authors":"Pulkit Agrawal, Dustin Stansbury, Jitendra Malik, Jack L Gallant","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Joseph L. Austerweil and Thomas L. Griffiths. A nonpara- metric Bayesian framework for constructing flexible fea- ture representations. Psychological Review, 120(4):817, 2013.","title":"A nonparametric Bayesian framework for constructing flexible feature representations","authors":"Joseph L Austerweil, Thomas L Griffiths","year":"2013","venue":"Psychological Review","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Yoshua Bengio. Learning deep architectures for AI. Founda- tions and Trends in Machine Learning, 2(1):1-127, 2009.","title":"Learning deep architectures for AI","authors":"Yoshua Bengio","year":"2009","venue":"Foundations and Trends in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Rachit Dubey, Joshua Peterson, Aditya Khosla, Ming-Hsuan Yang, and Bernard Ghanem. What makes an object mem- orable? In International Conference on Computer Vision (ICCV), 2015.","title":"What makes an object memorable?","authors":"Rachit Dubey, Joshua Peterson, Aditya Khosla, Ming-Hsuan Yang, Bernard Ghanem","year":"2015","venue":"International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Thomas L Griffiths, Nick Chater, Charles Kemp, Amy Per- fors, and Joshua B Tenenbaum. Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences, 14(8):357-364, 2010.","title":"Probabilistic models of cognition: Exploring representations and inductive biases","authors":"Nick Thomas L Griffiths, Charles Chater, Amy Kemp, Joshua B Perfors, Tenenbaum","year":"2010","venue":"Trends in Cognitive Sciences","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv preprint arXiv:1502.01852, 2015.","title":"Delving deep into rectifiers: Surpassing human-level performance on imagenet classification","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadar- rama, and Trevor Darrell. Caffe: Convolutional ar- chitecture for fast feature embedding. arXiv preprint arXiv:1408.5093, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, Trevor Darrell","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"Seyed-Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsupervised, models may ex- plain it cortical representation. PLoS Comput Biol, 10(11), 2014.","title":"Deep supervised, but not unsupervised, models may explain it cortical representation","authors":"Seyed-Mahdi Khaligh-Razavi, Nikolaus Kriegeskorte","year":"2014","venue":"PLoS Comput Biol","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Im- agenet classification with deep convolutional neural net- works. In Advances in Neural Information Processing Sys- tems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Brenden M Lake, Wojciech Zaremba, Rob Fergus, and Todd M. Gureckis. Deep neural networks predict category typicality ratings for images. In Proceedings of the 37th Annual Cognitive Science Society, 2015.","title":"Deep neural networks predict category typicality ratings for images","authors":"Wojciech Brenden M Lake, Rob Zaremba, Todd M Fergus, Gureckis","year":"2015","venue":"Proceedings of the 37th Annual Cognitive Science Society","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwrit- ten zip code recognition. Neural computation, 1(4):541- 551, 1989.","title":"Backpropagation applied to handwritten zip code recognition","authors":"Yann Lecun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, Lawrence D Jackel","year":"1989","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436-444, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"Marieke Mur, Mirjam Meys, Jerzy Bodurka, Rainer Goebel, Peter A. Bandettini, and Nikolaus Kriegeskorte. Hu- man object-similarity judgments reflect and transcend the primate-IT object representation. Frontiers in Psychology, 4, 2013.","title":"Human object-similarity judgments reflect and transcend the primate-IT object representation","authors":"Marieke Mur, Mirjam Meys, Jerzy Bodurka, Rainer Goebel, Peter A Bandettini, Nikolaus Kriegeskorte","year":"2013","venue":"Frontiers in Psychology","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"Joshua C Peterson, Joshua T Abbott, and Thomas L Griffiths. Adapting deep network features to capture psychological representations. arXiv preprint arXiv:1608.02164, 2016.","title":"Adapting deep network features to capture psychological representations","authors":"Joshua T Joshua C Peterson, Thomas L Abbott, Griffiths","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Roger N Shepard and Phipps Arabie. Additive clustering: Representation of similarities as combinations of discrete overlapping properties. Psychological Review, 86(2):87, 1979.","title":"Additive clustering: Representation of similarities as combinations of discrete overlapping properties","authors":"N Roger, Phipps Shepard, Arabie","year":"1979","venue":"Psychological Review","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"Roger N Shepard. Multidimensional scaling, tree-fitting, and clustering. Science, 210(4468):390-398, 1980.","title":"Multidimensional scaling, tree-fitting, and clustering","authors":"N Roger, Shepard","year":"1980","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014.","title":"Going deeper with convolutions","authors":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, and Noah D Goodman. How to grow a mind: Statistics, structure, and abstraction. Science, 331(6022):1279-1285, 2011.","title":"How to grow a mind: Statistics, structure, and abstraction","authors":"Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, Noah D Goodman","year":"2011","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, 111(23):8619-8624, 2014.","title":"Performance-optimized hierarchical models predict neural responses in higher visual cortex","authors":"Ha Daniel Lk Yamins, Charles F Hong, Ethan A Cadieu, Darren Solomon, James J Seibert, Dicarlo","year":"2014","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}