{"type":"ScholarlyArticle","title":"Trends, Applications, and Challenges in Human Attention Modelling","authors":[{"name":"Giuseppe Cartella","slug":"giuseppe-cartella","given":"Giuseppe","family":"Cartella","affiliation":"University of Modena and Reggio Emilia , Italy"},{"name":"Marcella Cornia","slug":"marcella-cornia","given":"Marcella","family":"Cornia","affiliation":"University of Modena and Reggio Emilia , Italy"},{"name":"Vittorio Cuculo","slug":"vittorio-cuculo","given":"Vittorio","family":"Cuculo","affiliation":"University of Modena and Reggio Emilia , Italy"},{"name":"Alessandro D'Amelio","slug":"alessandro-d-amelio","given":"Alessandro","family":"D'Amelio","affiliation":"University of Milan , Italy"},{"name":"Dario Zanca","slug":"dario-zanca","given":"Dario","family":"Zanca","affiliation":"Friedrich-Alexander-Universitat Erlangen-Nurnberg , Germany"},{"name":"Giuseppe Boccignone","slug":"giuseppe-boccignone","given":"Giuseppe","family":"Boccignone","affiliation":"University of Milan , Italy"},{"name":"Rita Cucchiara","slug":"rita-cucchiara","given":"Rita","family":"Cucchiara","affiliation":"University of Modena and Reggio Emilia , Italy"}],"abstract":"Human attention modelling has proven, in recent years, to be particularly useful not only for understanding the cognitive processes underlying visual exploration, but also for providing support to artificial intelligence models that aim to solve problems in various domains, including image and video processing, vision-and-language applications, and language modelling. This survey offers a reasoned overview of recent efforts to integrate human attention mechanisms into contemporary deep learning models and discusses future research directions and challenges.\r\nFor a comprehensive overview of the ongoing research, refer to our dedicated repository available at https://github.com/aimagelab/awesome-human-visual-attention.","doi":"10.24963/ijcai.2024/882","url":"https://ijcai.org/proceedings/2024/882","pdf":"https://ijcai.org/proceedings/2024/0882.pdf","markdown":"https://ijcai.org/proceedings/2024/882.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/882","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Survey Track","track":"Survey Track","number":882,"pages":{"first":7971,"last":7979},"keywords":["Humans and AI: General","Humans and AI: HAI: Applications"],"videos":[],"references":[{"position":1,"raw":"[Aberman et al., 2022] Kfir Aberman, Junfeng He, Yossi Gandelsman, et al. Deep saliency prior for reducing vi- sual distraction. In CVPR, 2022.","title":"Deep saliency prior for reducing visual distraction","authors":"Junfeng Kfir Aberman, Yossi He, Gandelsman","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aronson and Admoni, 2022] Reuben M Aronson and Henny Admoni. Gaze complements control input for goal prediction during assisted teleoperation. In RSS, 2022.","title":"Gaze complements control input for goal prediction during assisted teleoperation","authors":"M Reuben, Henny Aronson, Admoni","year":"2022","venue":"RSS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aydemir et al., 2023] Bahar Aydemir, Ludo Hoffstetter, Tong Zhang, et al. Tempsal-uncovering temporal infor- mation for deep saliency prediction. In CVPR, 2023.","title":"Tempsal-uncovering temporal information for deep saliency prediction","authors":"Bahar Aydemir, Ludo Hoffstetter, Tong Zhang","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Baee et al., 2021] Sonia Baee, Erfan Pakdamanian, Inki Kim, Lu Feng, Vicente Ordonez, et al. MEDIRL: Predict- ing the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning. In ICCV, 2021.","title":"MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning","authors":"Sonia Baee, Erfan Pakdamanian, Inki Kim, Lu Feng, Vicente Ordonez","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Berzak et al., 2017] Yevgeni Berzak, Chie Nakamura, Suzanne Flynn, and Boris Katz. Predicting Native Language from Gaze. In ACL, 2017.","title":"Predicting Native Language from Gaze","authors":"Yevgeni Berzak, Chie Nakamura, Suzanne Flynn, Boris Katz","year":"2017","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bhattacharya et al., 2022] Moinak Bhattacharya, Shubham Jain, and Prateek Prasanna. RadioTransformer: A Cascaded Global-Focal Transformer for Visual Atten- tion-Guided Disease Classification. In ECCV, 2022.","title":"RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-Guided Disease Classification","authors":"Bhattacharya","year":"2022","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bylinskii et al., 2017] Zoya Bylinskii, Nam Wook Kim, Pe- ter O'Donovan, Sami Alsheikh, Spandan Madan, et al. Learning visual importance for graphic designs and data visualizations. In ACM UIST, 2017.","title":"Learning visual importance for graphic designs and data visualizations","authors":"Zoya Bylinskii, Nam Wook Kim, O' Peter, Sami Donovan, Spandan Alsheikh, Madan","year":"2017","venue":"ACM UIST","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bylinskii et al., 2019] Zoya Bylinskii, Tilke Judd, Aude Oliva, et al. What do different evaluation metrics tell us about saliency models? IEEE TPAMI, 41(3), 2019.","title":"What do different evaluation metrics tell us about saliency models","authors":"Zoya Bylinskii, Tilke Judd, Aude Oliva","year":"2019","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2020] Shi Chen, Ming Jiang, Jinhui Yang, and Qi Zhao. AiR: Attention with Reasoning Capability. In ECCV, 2020.","title":"AiR: Attention with Reasoning Capability","authors":"Ming Shi Chen, Jinhui Jiang, Qi Yang, Zhao","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chen et al., 2023] Shi Chen, Nachiappan Valliappan, Shaolei Shen, et al. Learning from unique perspectives: User-aware saliency modeling. In CVPR, 2023.","title":"Learning from unique perspectives: User-aware saliency modeling","authors":"Nachiappan Shi Chen, Shaolei Valliappan, Shen","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Cornia et al., 2018a] Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, and Rita Cucchiara. Paying More At- tention to Saliency: Image Captioning with Saliency and Context Attention. ACM TOMM, 14(2), 2018.","title":"Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention","authors":"Cornia","year":"2018","venue":"ACM TOMM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Cornia et al., 2018b] Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, and Rita Cucchiara. Predicting Hu- man Eye Fixations via an LSTM-Based Saliency Attentive Model. IEEE TIP, 27(10), 2018.","title":"Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model","authors":"Cornia","year":"2018","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Das et al., 2016] Abhishek Das, Harsh Agrawal, Larry Zit- nick, Devi Parikh, and Dhruv Batra. Human Attention in Visual Question Answering: Do Humans and Deep Net- works Look at the Same Regions? In EMNLP, 2016.","title":"Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?","authors":"Abhishek Das, Harsh Agrawal, Larry Zitnick, Devi Parikh, Dhruv Batra","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Deng et al., 2023] Shuwen Deng, Paul Prasse, David R Re- ich, Tobias Scheffer, and Lena A Jäger. Pre-Trained Lan- guage Models Augmented with Synthetic Scanpaths for Natural Language Understanding. In EMNLP, 2023.","title":"Pre-Trained Language Models Augmented with Synthetic Scanpaths for Natural Language Understanding","authors":"Shuwen Deng, Paul Prasse, Tobias David R Reich, Lena A Scheffer, Jäger","year":"2023","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Fan et al., 2019] Deng-Ping Fan, Wenguan Wang, Ming- Ming Cheng, and Jianbing Shen. Shifting more attention to video salient object detection. In CVPR, 2019. [Fosco et al., 2020]","title":"Shifting more attention to video salient object detection","authors":"Deng-Ping Fan, Wenguan Wang, Ming-Ming Cheng, Jianbing Shen","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"Camilo Fosco, Vincent Casser, Amish Kumar Bedi, Peter O'Donovan, Aaron Hertz- mann, and Zoya Bylinskii. Predicting visual importance across graphic design types. In ACM UIST, 2020.","title":"Predicting visual importance across graphic design types","authors":"Camilo Fosco, Vincent Casser, Amish Kumar Bedi, O' Peter, Aaron Donovan, Zoya Hertzmann, Bylinskii","year":"2020","venue":"ACM UIST","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gatys et al., 2017] Leon A Gatys, Matthias Kümmerer, Thomas SA Wallis, et al. Guiding human gaze with con- volutional neural networks. arXiv:1712.06492, 2017.","title":"Guiding human gaze with convolutional neural networks","authors":"Leon A Gatys, Matthias Kümmerer, S A Thomas, Wallis","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ghosh et al., 2023] Shreya Ghosh, Abhinav Dhall, et al. Automatic gaze analysis: A survey of deep learning based approaches. IEEE TPAMI, 46(1), 2023.","title":"Automatic gaze analysis: A survey of deep learning based approaches","authors":"Shreya Ghosh, Abhinav Dhall","year":"2023","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Grauman et al., 2022] Kristen Grauman, Andrew Westbury, Eugene Byrne, et al. Ego4D: Around the world in 3,000 hours of egocentric video. In CVPR, 2022.","title":"Ego4D: Around the world in 3,000 hours of egocentric video","authors":"Kristen Grauman, Andrew Westbury, Eugene Byrne","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Gressel et al., 2023] Céline Gressel, Rebekah Overdorf, Inken Hagenstedt, Murat Karaboga, Helmut Lurtz, et al. Privacy-Aware Eye Tracking: Challenges and Future Di- rections. IEEE Pervasive Computing, 22(1), 2023.","title":"Privacy-Aware Eye Tracking: Challenges and Future Directions","authors":"Gressel","year":"2023","venue":"IEEE Pervasive Computing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[He et al., 2019] Sen He, Hamed R Tavakoli, Ali Borji, and Nicolas Pugeault. Human Attention in Image Captioning: Dataset and Analysis. In ICCV, 2019.","title":"Human Attention in Image Captioning: Dataset and Analysis","authors":"Sen He, Ali Hamed R Tavakoli, Nicolas Borji, Pugeault","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Hou et al., 2024] Yuqi Hou, Zhongqun Zhang, Nora Ho- ranyi, Jaewon Moon, et al. Multi-Modal Gaze Following in Conversational Scenarios. In WACV, 2024.","title":"Multi-Modal Gaze Following in Conversational Scenarios","authors":"Yuqi Hou, Zhongqun Zhang, Nora Horanyi, Jaewon Moon","year":"2024","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Huang et al., 2019] Sandy H Huang, Isabella Huang, Ravi Pandya, and Anca D Dragan. Nonverbal Robot Feedback for Human Teachers. In CoRL, 2019.","title":"Nonverbal Robot Feedback for Human Teachers","authors":"H Sandy, Isabella Huang, Ravi Huang, Anca D Pandya, Dragan","year":"2019","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ilaslan et al., 2023] Muhammet Ilaslan, Chenan Song, Joya Chen, Difei Gao, Weixian Lei, et al. GazeVQA: A video question answering dataset for multiview eye-gaze task- oriented collaborations. In EMNLP, 2023.","title":"GazeVQA: A video question answering dataset for multiview eye-gaze taskoriented collaborations","authors":"Muhammet Ilaslan, Chenan Song, Joya Chen, Difei Gao, Weixian Lei","year":"2023","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Itti et al., 1998] Laurent Itti, Christof Koch, and Ernst Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE TPAMI, 20(11), 1998.","title":"A model of saliency-based visual attention for rapid scene analysis","authors":"Laurent Itti, Christof Koch, Ernst Niebur","year":"1998","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Jiang et al., 2015] Ming Jiang, Shengsheng Huang, Juany- ong Duan, and Qi Zhao. SALICON: Saliency in Context. In CVPR, 2015.","title":"SALICON: Saliency in Context","authors":"Ming Jiang, Shengsheng Huang, Juanyong Duan, Qi Zhao","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Jiang et al., 2021] Lai Jiang, Mai Xu, Xiaofei Wang, et al. Saliency-guided image translation. In CVPR, 2021.","title":"Saliency-guided image translation","authors":"Lai Jiang, Mai Xu, Xiaofei Wang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Judd et al., 2009] Tilke Judd, Krista Ehinger, Frédo Durand, and Antonio Torralba. Learning to predict where humans look. In ICCV, 2009.","title":"Learning to predict where humans look","authors":"Tilke Judd, Krista Ehinger, Frédo Durand, Antonio Torralba","year":"2009","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Karargyris et al., 2021] Alexandros Karargyris, Satyananda Kashyap, Ismini Lourentzou, et al. Creation and valida- tion of a chest X-ray dataset with eye-tracking and report dictation for AI development. Scient. Data, 8(1), 2021.","title":"Creation and validation of a chest X-ray dataset with eye-tracking and report dictation for AI development","authors":"Alexandros Karargyris, Satyananda Kashyap, Ismini Lourentzou","year":"2021","venue":"Scient. Data","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Khurana et al., 2023] Varun Khurana, Yaman Kumar Singla, et al. Synthesizing Human Gaze Feedback for Improved NLP Performance. In EACL, 2023.","title":"Synthesizing Human Gaze Feedback for Improved NLP Performance","authors":"Varun Khurana, Yaman Kumar Singla","year":"2023","venue":"EACL","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Kümmerer and Bethge, 2023] Matthias Kümmerer and Matthias Bethge. Predicting visual fixations. Annual Review of Vision Science, 9, 2023.","title":"Predicting visual fixations","authors":"Bethge Kümmerer, Matthias Kümmerer, Matthias Bethge","year":"2023","venue":"Annual Review of Vision Science","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Kümmerer et al., 2022] Matthias Kümmerer, Matthias Bethge, et al. Deepgaze iii: Modeling free-viewing human scanpaths with deep learning. J. of Vision, 22(5), 2022.","title":"Deepgaze iii: Modeling free-viewing human scanpaths with deep learning","authors":"Matthias Kümmerer, Matthias Bethge","year":"2022","venue":"J. of Vision","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Le Meur and Liu, 2015] Olivier Le Meur and Zhi Liu. Sac- cadic model of eye movements for free-viewing condition. Vision Research, 116, 2015.","title":"Saccadic model of eye movements for free-viewing condition","authors":"Le Meur, ; Liu, Le Olivier, Zhi Meur, Liu","year":"2015","venue":"Vision Research","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Leiva et al., 2020] Luis A Leiva, Yunfei Xue, Avya Bansal, Hamed R Tavakoli, et al. Understanding visual saliency in mobile user interfaces. In ACM MobileHCI, 2020.","title":"Understanding visual saliency in mobile user interfaces","authors":"Yunfei Luis A Leiva, Avya Xue, Bansal, Hamed R Tavakoli","year":"2020","venue":"ACM MobileHCI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Lou et al., 2022] Jianxun Lou, Hanhe Lin, David Marshall, et al. Transalnet: Towards perceptually relevant visual saliency prediction. Neurocomputing, 494, 2022.","title":"Transalnet: Towards perceptually relevant visual saliency prediction","authors":"Lou","year":"2022","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ma et al., 2023] Chong Ma, Lin Zhao, Yuzhong Chen, et al. Eye-gaze-guided vision transformer for rectifying shortcut learning. IEEE Trans. Med. Imaging, 41(11), 2023.","title":"Eye-gaze-guided vision transformer for rectifying shortcut learning","authors":"Chong Ma, Lin Zhao, Yuzhong Chen","year":"2023","venue":"IEEE Trans. Med. Imaging","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Miangoleh et al., 2023] S Mahdi H Miangoleh, Zoya Bylin- skii, Eric Kee, Eli Shechtman, and Yagiz Aksoy. Realistic saliency guided image enhancement. In CVPR, 2023.","title":"Realistic saliency guided image enhancement","authors":"H Mahdi, Zoya Miangoleh, Eric Bylinskii, Eli Kee, Yagiz Shechtman, Aksoy","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Min and Corso, 2021] Kyle Min and Jason J Corso. Inte- grating human gaze into attention for egocentric activity recognition. In WACV, 2021.","title":"Integrating human gaze into attention for egocentric activity recognition","authors":"Kyle Min, Jason J Corso","year":"2021","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Mondal et al., 2023] Sounak Mondal, Zhibo Yang, Seoy- oung Ahn, Dimitris Samaras, Gregory Zelinsky, and Minh Hoai. Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention. In CVPR, 2023.","title":"Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention","authors":"Mondal","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Pal et al., 2020] Anwesan Pal, Sayan Mondal, and Henrik I Christensen. Looking at the right stuff -guided semantic- gaze for autonomous driving. In CVPR, 2020.","title":"Looking at the right stuff -guided semanticgaze for autonomous driving","authors":"Pal","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Palazzi et al., 2018] Andrea Palazzi, Davide Abati, Francesco Solera, Simone Calderara, and Rita Cuc- chiara. Predicting the Driver's Focus of Attention: The DR(eye)VE Project. IEEE TPAMI, 41(7), 2018.","title":"Predicting the Driver's Focus of Attention: The DR(eye)VE Project","authors":"Andrea Palazzi, Davide Abati, Francesco Solera, Simone Calderara, Rita Cucchiara","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Qian et al., 2023] Kun Qian, Zhuoyang Zhang, Wei Song, and Jianfeng Liao. Gvgnet: Gaze-directed visual ground- ing for learning under-specified object referring intention. IEEE RA-L, 8(9), 2023.","title":"Gvgnet: Gaze-directed visual grounding for learning under-specified object referring intention","authors":"Zhuoyang Kun Qian, Wei Zhang, Jianfeng Song, Liao","year":"2023","venue":"IEEE RA-L","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Qiao et al., 2018] Tingting Qiao, Jianfeng Dong, and Duan- qing Xu. Exploring Human-like Attention Supervision in Visual Question Answering. In AAAI, 2018.","title":"Exploring Human-like Attention Supervision in Visual Question Answering","authors":"Tingting Qiao, Jianfeng Dong, Duanqing Xu","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Qiao et al., 2023] Minglang Qiao, Yufan Liu, Mai Xu, Xin Deng, Bing Li, Weiming Hu, and Ali Borji. Joint learn- ing of audio-visual saliency prediction and sound source localization on multi-face videos. IJCV, 2023.","title":"Joint learning of audio-visual saliency prediction and sound source localization on multi-face videos","authors":"Qiao","year":"2023","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Reich et al., 2022] David Robert Reich, Paul Prasse, Chiara Tschirner, Patrick Haller, et al. Inferring native and non- native human reading comprehension and subjective text difficulty from scanpaths in reading. In ETRA, 2022.","title":"Inferring native and nonnative human reading comprehension and subjective text difficulty from scanpaths in reading","authors":"David Robert Reich, Paul Prasse, Chiara Tschirner, Patrick Haller","year":"2022","venue":"ETRA","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Saran et al., 2019] Akanksha Saran, Elaine Schaertl Short, Andrea Thomaz, and Scott Niekum. Understanding teacher gaze patterns for robot learning. In CoRL, 2019.","title":"Understanding teacher gaze patterns for robot learning","authors":"Akanksha Saran, Elaine Schaertl Short, Andrea Thomaz, Scott Niekum","year":"2019","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Skerath et al., 2023] Lina Skerath, Paulina Toborek, Anita Zielińska, et al. Native Language Prediction from Gaze: a Reproducibility Study. In ACL Workshops, 2023.","title":"Native Language Prediction from Gaze: a Reproducibility Study","authors":"Lina Skerath, Paulina Toborek, Anita Zielińska","year":"2023","venue":"ACL Workshops","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Sood et al., 2020] Ekta Sood, Simon Tannert, Philipp Müller, and Andreas Bulling. Improving Natural Lan- guage Processing Tasks with Human Gaze-Guided Neural Attention. In NeurIPS, 2020.","title":"Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention","authors":"Ekta Sood, Simon Tannert, Philipp Müller, Andreas Bulling","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Sood et al., 2021] Ekta Sood, Fabian Kögel, Florian Strohm, Prajit Dhar, and Andreas Bulling. VQA-MHUG: A gaze dataset to study multimodal neural attention in visual question answering. In CoNLL, 2021.","title":"VQA-MHUG: A gaze dataset to study multimodal neural attention in visual question answering","authors":"Ekta Sood, Fabian Kögel, Florian Strohm, Prajit Dhar, Andreas Bulling","year":"2021","venue":"CoNLL","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Sood et al., 2023] Ekta Sood, Fabian Kögel, Philipp Müller, et al. Multimodal Integration of Human-Like Attention in Visual Question Answering. In CVPR Workshops, 2023.","title":"Multimodal Integration of Human-Like Attention in Visual Question Answering","authors":"Ekta Sood, Fabian Kögel, Philipp Müller","year":"2023","venue":"CVPR Workshops","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Sugano and Bulling, 2016] Yusuke Sugano and Andreas Bulling. Seeing with Humans: Gaze-Assisted Neural Im- age Captioning. arXiv:1608.05203, 2016.","title":"Seeing with Humans: Gaze-Assisted Neural Image Captioning","authors":"Bulling Sugano, Yusuke Sugano, Andreas Bulling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Sui et al., 2023] Xiangjie Sui, Yuming Fang, Hanwei Zhu, Shiqi Wang, et al. ScanDMM: A Deep Markov Model of Scanpath Prediction for 360 Images. In CVPR, 2023.","title":"ScanDMM: A Deep Markov Model of Scanpath Prediction for 360 Images","authors":"Sui","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Takmaz et al., 2020] Ece Takmaz, Sandro Pezzelle, Lisa Beinborn, and Raquel Fernández. Generating Image De- scriptions via Sequential Cross-Modal Alignment Guided by Human Gaze. In EMNLP, 2020.","title":"Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze","authors":"Ece Takmaz, Sandro Pezzelle, Lisa Beinborn, Raquel Fernández","year":"2020","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Tiezzi et al., 2022] Matteo Tiezzi, Simone Marullo, Alessandro Betti, Enrico Meloni, Lapo Faggi, et al. Foveated Neural Computation. In ECML PKDD, 2022.","title":"Foveated Neural Computation","authors":"Matteo Tiezzi, Simone Marullo, Alessandro Betti, Enrico Meloni, Lapo Faggi","year":"2022","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Wang et al., 2019] Wenguan Wang, Hongmei Song, Shuyang Zhao, et al. Learning unsupervised video object segmentation through visual attention. In CVPR, 2019.","title":"Learning unsupervised video object segmentation through visual attention","authors":"Wang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Wang et al., 2022] Sheng Wang, Xi Ouyang, Tianming Liu, et al. Follow my eye: Using gaze to supervise computer- aided diagnosis. IEEE Trans. Med. Imaging, 41(7), 2022.","title":"Follow my eye: Using gaze to supervise computeraided diagnosis","authors":"Sheng Wang, Xi Ouyang, Tianming Liu","year":"2022","venue":"IEEE Trans. Med. Imaging","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Wang et al., 2024] Bin Wang, Hongyi Pan, Armstrong Aboah, et al. GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-Ray Classification. In WACV, 2024.","title":"GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-Ray Classification","authors":"Wang","year":"2024","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Xia et al., 2018] Ye Xia, Danqing Zhang, Jinkyu Kim, Ken Nakayama, Karl Zipser, and David Whitney. Predicting driver attention in critical situations. In ACCV, 2018.","title":"Predicting driver attention in critical situations","authors":"Ye Xia, Danqing Zhang, Jinkyu Kim, Ken Nakayama, Karl Zipser, David Whitney","year":"2018","venue":"ACCV","doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Yan et al., 2023] Kun Yan, Lei Ji, Zeyu Wang, et al. Voila- A: Aligning Vision-Language Models with User's Gaze Attention. arXiv:2401.09454, 2023.","title":"Voila-A: Aligning Vision-Language Models with User's Gaze Attention","authors":"Kun Yan, Lei Ji, Zeyu Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Yang et al., 2019] Sheng Yang, Qiuping Jiang, Weisi Lin, and Yongtao Wang. SGDNet: an end-to-end saliency- guided deep neural network for no-reference image quality assessment. In ACM MM, 2019.","title":"SGDNet: an end-to-end saliencyguided deep neural network for no-reference image quality assessment","authors":"Sheng Yang, Qiuping Jiang, Weisi Lin, Yongtao Wang","year":"2019","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Yu et al., 2017] Youngjae Yu, Jongwook Choi, Yeonhwa Kim, et al. Supervising Neural Attention Models for Video Captioning by Human Gaze Data. In CVPR, 2017.","title":"Supervising Neural Attention Models for Video Captioning by Human Gaze Data","authors":"Youngjae Yu, Jongwook Choi, Yeonhwa Kim","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Zanca et al., 2020] Dario Zanca, Stefano Melacci, and Marco Gori. Gravitational laws of focus of attention. IEEE TPAMI, 42(12), 2020.","title":"Gravitational laws of focus of attention","authors":"Dario Zanca, Stefano Melacci, Marco Gori","year":"2020","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Zhang et al., 2020] Ruohan Zhang, Akanksha Saran, Bo Liu, Yifeng Zhu, Sihang Guo, et al. Human gaze assisted artificial intelligence: A review. In IJCAI, 2020.","title":"Human gaze assisted artificial intelligence: A review","authors":"Zhang","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/689"},{"position":64,"raw":"[Zhu et al., 2021] Mengmeng Zhu, Guanqun Hou, Xinjia Chen, et al. Saliency-guided Transformer network com- bined with local embedding for no-reference image quality assessment. In ICCV, 2021.","title":"Saliency-guided Transformer network combined with local embedding for no-reference image quality assessment","authors":"Zhu","year":"2021","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}