{"type":"ScholarlyArticle","title":"Characterizing Similarity of Visual Stimulus from Associated Neuronal Response","authors":[{"name":"Vikram Ravindra","slug":"vikram-ravindra","given":"Vikram","family":"Ravindra","affiliation":"Department of Computer Science , Purdue University , West Lafayette , IN"},{"name":"Ananth Grama","slug":"ananth-grama","given":"Ananth","family":"Grama","affiliation":"Department of Computer Science , Purdue University , West Lafayette , IN"}],"abstract":"The problem of characterizing brain functions such as memory, perception, and processing of stimuli has received significant attention in neuroscience literature. These experiments rely on carefully calibrated, albeit complex inputs, to record brain response to signals. A major problem in analyzing brain response to common stimuli such as audio-visual input from videos (e.g., movies) or story narration through audio books, is that  observed neuronal responses are due to combinations of ``pure'' factors, many of which may be latent. In this paper, we present a novel methodological framework for deconvolving the brain's response to mixed stimuli into its constituent responses to underlying pure factors. This framework, based on archetypal analysis, is applied to the analysis of imaging data from an adult cohort watching the BBC show, Sherlock. By focusing on visual stimulus, we show strong correlation between our observed deconvolved response and third-party textual video annotations -- demonstrating the significant power of our analyses techniques. Building on these results, we show that our techniques can be used to predict neuronal responses in new subjects (how other individuals react to Sherlock), as well as to new visual content (how individuals react to other videos with known annotations). This paper reports on the first study that relates video features with neuronal responses in a rigorous algorithmic and statistical framework based on deconvolution of observed mixed imaging signals using archetypal analysis.","doi":"10.24963/ijcai.2020/85","url":"https://ijcai.org/proceedings/2020/85","pdf":"https://ijcai.org/proceedings/2020/0085.pdf","markdown":"https://ijcai.org/proceedings/2020/85.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/85","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":85,"pages":{"first":608,"last":614},"keywords":["Machine Learning: Feature Selection; Learning Sparse Models","Humans and AI: Brain Sciences","Computer Vision: Biomedical Image Understanding"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24640/file_24640.m3u8","length_sec":267,"provider":"hls"}],"references":[{"position":1,"raw":"[Chan et al., 2009] T. Chan, C. Chi, Y. Huang, and W. Ma. A convex analysis-based minimum-volume enclosing sim- plex algorithm for hyperspectral unmixing. IEEE Transac- tions on Signal Processing, 57(11):4418-4432, Nov 2009.","title":"A convex analysis-based minimum-volume enclosing simplex algorithm for hyperspectral unmixing","authors":"T Chan, C Chi, Y Huang, W Ma","year":"2009","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chan et al., 2012] T. Chan, J. Liou, A. Ambikapathi, W. Ma, and C. Chi. Fast algorithms for robust hyper- spectral endmember extraction based on worst-case sim- plex volume maximization. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1237-1240, March 2012.","title":"Fast algorithms for robust hyperspectral endmember extraction based on worst-case simplex volume maximization","authors":"T Chan, J Liou, A Ambikapathi, W Ma, C Chi","year":"2012","venue":"2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2017] Janice Chen, Yuan Chang Leong, Christopher J Honey, Chung H Yong, Kenneth A Norman, and Uri Hasson. Shared memories reveal shared structure in neural activity across individuals. Nature Neuroscience, 20(1):115, 2017.","title":"Shared memories reveal shared structure in neural activity across individuals","authors":"Janice Chen, Chang Yuan, Christopher J Leong, Chung H Honey, Kenneth A Yong, Uri Norman, Hasson","year":"2017","venue":"Nature Neuroscience","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cutler and Breiman, 1994] Adele Cutler and Leo Breiman. Archetypal analysis. Technometrics, 36(4):338-347, 1994.","title":"Adele Cutler and Leo Breiman","authors":"Breiman Cutler","year":"1994","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Damle and Sun, 2017] Anil Damle and Yuekai Sun. A ge- ometric approach to archetypal analysis and nonnegative matrix factorization. Technometrics, 59(3):361-370, 2017.","title":"A geometric approach to archetypal analysis and nonnegative matrix factorization","authors":"Anil Damle, Yuekai Sun","year":"2017","venue":"Technometrics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Desikan et al., 2006] Rahul S Desikan, Florent Ségonne, Bruce Fischl, Brian T Quinn, Bradford C Dickerson, Deborah Blacker, Randy L Buckner, Anders M Dale, R. Paul Maguire, Bradley T Hyman, Marilyn S Albert, and Ronald J Killiany. An automated labeling system for sub- dividing the human cerebral cortex on mri scans into gyral based regions of interest. 31(3):968-980, 2006.","title":"An automated labeling system for subdividing the human cerebral cortex on mri scans into gyral based regions of interest","authors":"Florent Rahul S Desikan, Bruce Ségonne, Brian T Fischl, Bradford C Quinn, Deborah Dickerson, Randy L Blacker, Anders M Buckner, R Paul Dale, Bradley T Maguire, Marilyn S Hyman, Ronald J Albert, Killiany","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hasson et al., 2004] Uri Hasson, Yuval Nir, Ifat Levy, Galit Fuhrmann, and Rafael Malach. Intersubject synchroniza- tion of cortical activity during natural vision.(research ar- ticle). Science, 303(5664):1634, 2004.","title":"Intersubject synchronization of cortical activity during natural vision","authors":"Uri Hasson, Yuval Nir, Ifat Levy, Galit Fuhrmann, Rafael Malach","year":"2004","venue":"research article). Science","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hinrich et al., 2016] Jesper Love Hinrich, Sophia Eliza- beth Bardenfleth, Rasmus Erbou Roge, Nathan William Churchill, Kristoffer Hougaard Madsen, and Morten Morup. Archetypal analysis for modeling multisubject fmri data. 10(7):1160-1171, 2016.","title":"Archetypal analysis for modeling multisubject fmri data","authors":"Hinrich","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Honey et al., 2012] Christopher J Honey, Christopher R Thompson, Yulia Lerner, and Uri Hasson. Not lost in translation: neural responses shared across languages. 32(44):15277, 2012.","title":"Not lost in translation: neural responses shared across languages","authors":"J Christopher, Honey, Yulia Christopher R Thompson, Uri Lerner, Hasson","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Javadi and Montanari, 2019] H. Javadi and A. Montanari. Nonnegative matrix factorization via archetypal analysis. Journal of the American Statistical Association, 2019.","title":"Nonnegative matrix factorization via archetypal analysis","authors":"H Javadi, A Montanari","year":"2019","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lahnakoski et al., 2014] Juha M. Lahnakoski, Enrico Glerean, Iiro P. Jääskeläinen, Jukka Hyönä, Riitta Hari, Mikko Sams, and Lauri Nummenmaa. Synchronous brain activity across individuals underlies shared psychological perspectives. NeuroImage, 100:316 -324, 2014.","title":"Synchronous brain activity across individuals underlies shared psychological perspectives","authors":"M Juha, Enrico Lahnakoski, Glerean, P Iiro, Jukka Jääskeläinen, Riitta Hyönä, Mikko Hari, Lauri Sams, Nummenmaa","year":"2014","venue":"NeuroImage","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Bioucas-Dias, 2008] Jun Li and J.M Bioucas-Dias. Minimum volume simplex analysis: A fast algorithm to unmix hyperspectral data. In IGARSS 2008 -2008 IEEE International Geoscience and Remote Sensing Sym- posium, volume 3, pages III -250-III -253. IEEE, 2008.","title":"Minimum volume simplex analysis: A fast algorithm to unmix hyperspectral data","authors":"Jun Li, J M Bioucas-Dias","year":"2008","venue":"IGARSS 2008 -2008 IEEE International Geoscience and Remote Sensing Symposium","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mørup and Hansen, 2012] Morten Mørup and Lars Kai Hansen. Archetypal analysis for machine learning and data mining. Neurocomputing, 80:54, 2012.","title":"Archetypal analysis for machine learning and data mining","authors":"Hansen Mørup, Morten Mørup, Lars Kai Hansen","year":"2012","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Seth and Eugster, 2016] Sohan Seth and Manuel J. A. Eug- ster. Probabilistic archetypal analysis. Machine Learning, 102(1):85-113, Jan 2016.","title":"Probabilistic archetypal analysis","authors":"Sohan Seth, J A Manuel, Eugster","year":"2016","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Simony et al., 2016] Erez Simony, Christopher J Honey, Janice Chen, Olga Lositsky, Yaara Yeshurun, Ami Wiesel, and Uri Hasson. Dynamic reconfiguration of the default mode network during narrative comprehension. Nature Communications, 7(1), 2016.","title":"Dynamic reconfiguration of the default mode network during narrative comprehension","authors":"Erez Simony, J Christopher, Janice Honey, Olga Chen, Yaara Lositsky, Ami Yeshurun, Uri Wiesel, Hasson","year":"2016","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wilson et al., 2008] Stephen M. Wilson, Istvan Molnar- Szakacs, and Marco Iacoboni. Beyond superior tempo- ral cortex: Intersubject correlations in narrative speech comprehension. Cerebral Cortex, 18(1):230-242, January 2008.","title":"Beyond superior temporal cortex: Intersubject correlations in narrative speech comprehension","authors":"Stephen M Wilson, Istvan Molnar-Szakacs, Marco Iacoboni","year":"2008","venue":"Cerebral Cortex","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}