{"type":"ScholarlyArticle","title":"Retrieval Guided Music Captioning via Multimodal Prefixes","authors":[{"name":"Nikita Srivatsan","slug":"nikita-srivatsan","given":"Nikita","family":"Srivatsan","affiliation":"Carnegie Mellon University"},{"name":"Ke Chen","slug":"ke-chen","given":"Ke","family":"Chen","affiliation":"UC San Diego"},{"name":"Shlomo Dubnov","slug":"shlomo-dubnov","given":"Shlomo","family":"Dubnov","affiliation":"UC San Diego"},{"name":"Taylor Berg-Kirkpatrick","slug":"taylor-berg-kirkpatrick","given":"Taylor","family":"Berg-Kirkpatrick","affiliation":"UC San Diego"}],"abstract":"In this paper we put forward a new approach to music captioning, the task of automatically generating natural language descriptions for songs. These descriptions are useful both for categorization and analysis, and also from an accessibility standpoint as they form an important component of closed captions for video content. Our method supplements an audio encoding with a retriever, allowing the decoder to condition on multimodal signal both from the audio of the song itself as well as a candidate caption identified by a nearest neighbor system. This lets us retain the advantages of a retrieval based approach while also allowing for the flexibility of a generative one. We evaluate this system on a dataset of 200k music-caption pairs scraped from Audiostock, a royalty-free music platform, and on MusicCaps, a dataset of 5.5k pairs. We demonstrate significant improvements over prior systems across both automatic metrics and human evaluation.","doi":"10.24963/ijcai.2024/859","url":"https://ijcai.org/proceedings/2024/859","pdf":"https://ijcai.org/proceedings/2024/0859.pdf","markdown":"https://ijcai.org/proceedings/2024/859.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/859","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI, Arts & Creativity","track":"AI, Arts & Creativity","number":859,"pages":{"first":7762,"last":7770},"keywords":["Methods and resources: Machine learning, deep learning, neural models, reinforcement learning","Application domains: Music and sound"],"videos":[],"references":[{"position":1,"raw":"[Agostinelli et al., 2023] Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Cail- lon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matthew Sharifi, Neil Zeghidour, and Chris- tian Havnø Frank. Musiclm: Generating music from text. ArXiv preprint 2301.11325, 2023.","title":"Musiclm: Generating music from text","authors":"Andrea Agostinelli, I Timo, Zalán Denk, Jesse Borsos, Mauro Engel, Antoine Verzetti, Qingqing Caillon, Aren Huang, Adam Jansen, Marco Roberts, Matthew Tagliasacchi, Neil Sharifi, Christian Zeghidour, Frank Havnø","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aleksandrowicz, 2020] Paweł Aleksandrowicz. Can subti- tles for the deaf and hard-of-hearing convey the emotions of film music? a reception study. Perspectives, 28(1):58- 72, 2020.","title":"Can subtitles for the deaf and hard-of-hearing convey the emotions of film music? a reception study","authors":"Paweł Aleksandrowicz","year":"2020","venue":"Perspectives","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Borgeaud et al., 2021] Sebastian Borgeaud, Arthur Men- sch, and Jordan Hoffmann et al. Improving language mod- els by retrieving from trillions of tokens. In Proceedings of International Conference on Machine Learning (ICML), 2021.","title":"Improving language models by retrieving from trillions of tokens","authors":"Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann","year":"2021","venue":"Proceedings of International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2022] Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Hts-at: A hierarchical token-semantic audio transformer for sound classification and detection. In Proceedings of IEEE In- ternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 646-650, 2022.","title":"Hts-at: A hierarchical token-semantic audio transformer for sound classification and detection","authors":"Ke Chen, Xingjian Du, Bilei Zhu, Zejun Ma, Taylor Berg-Kirkpatrick, Shlomo Dubnov","year":"2022","venue":"Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen* et al., 2024] Ke Chen*, Yusong Wu*, Haohe Liu*, Marianna Nezhurina, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Musicldm: Enhancing novelty in text- to-music generation using beat-synchronous mixup strate- gies. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024.","title":"Musicldm: Enhancing novelty in textto-music generation using beat-synchronous mixup strategies","authors":"Ke Chen, *, Yusong Wu, *, Haohe Liu, *, Marianna Nezhurina, Taylor Berg-Kirkpatrick, Shlomo Dubnov","year":"2024","venue":"Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Choi et al., 2016] Keunwoo Choi, György Fazekas, and Mark B. Sandler. Towards music captioning: Generating music playlist descriptions. ArXiv preprint 1608.04868, 2016.","title":"Towards music captioning: Generating music playlist descriptions","authors":"Choi","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Davidson, 2022] Michael Davidson. 7. a captioned life. In Distressing Language, pages 157-182. New York Univer- sity Press, 2022.","title":"7. a captioned life","authors":"Michael Davidson","year":"2022","venue":"Distressing Language","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Deng et al., 2023] Zihao Deng, Yi Ma, Yudong Liu, Rongchen Guo, Ge Zhang, Wenhu Chen, Wenhao Huang, and Emmanouil Benetos. Musilingo: Bridging music and text with pre-trained language models for music caption- ing and query response. ArXiv preprint 2309.08730, 2023.","title":"Musilingo: Bridging music and text with pre-trained language models for music captioning and query response","authors":"Zihao Deng, Yi Ma, Yudong Liu, Rongchen Guo, Ge Zhang, Wenhu Chen, Wenhao Huang, Emmanouil Benetos","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Denkowski and Lavie, 2014] Michael Denkowski and Alon Lavie. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Workshop on Statistical Machine Translation, pages 376- 380, 2014.","title":"Meteor universal: Language specific translation evaluation for any target language","authors":"Lavie Denkowski, Michael Denkowski, Alon Lavie","year":"2014","venue":"Proceedings of the Workshop on Statistical Machine Translation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Doh et al., 2021] Seungheon Doh, Junwon Lee, and Juhan Nam. Music playlist title generation: A machine- translation approach. In Proceedings of the Workshop on NLP for Music and Spoken Audio (NLP4MusA), 2021.","title":"Music playlist title generation: A machinetranslation approach","authors":"Doh","year":"2021","venue":"Proceedings of the Workshop on NLP for Music and Spoken Audio (NLP4MusA)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Doh et al., 2023] Seungheon Doh, Keunwoo Choi, Jongpil Lee, and Juhan Nam. Lp-musiccaps: Llm-based pseudo music captioning. In Proceedings of International Soci- ety for Music Information Retrieval Conference (ISMIR), 2023.","title":"Lp-musiccaps: Llm-based pseudo music captioning","authors":"Doh","year":"2023","venue":"Proceedings of International Society for Music Information Retrieval Conference (ISMIR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Foley and Ferri, 2012] Alan Foley and Beth A Ferri. Tech- nology for people, not disabilities: Ensuring access and inclusion. Journal of Research in Special Educational Needs, 12(4):192-200, 2012.","title":"Technology for people, not disabilities: Ensuring access and inclusion","authors":"Ferri Foley, Alan Foley, Beth A Ferri","year":"2012","venue":"Journal of Research in Special Educational Needs","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gabbolini et al., 2022] Giovanni Gabbolini, Romain Hen- nequin, and Elena Epure. Data-efficient playlist captioning with musical and linguistic knowledge. In Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022.","title":"Data-efficient playlist captioning with musical and linguistic knowledge","authors":"Giovanni Gabbolini, Romain Hennequin, Elena Epure","year":"2022","venue":"Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gong et al., 2023a] Yuan Gong, Alexander H Liu, Hongyin Luo, Leonid Karlinsky, and James Glass. Joint audio and speech understanding. In Proceedings of IEEE Au- tomatic Speech Recognition and Understanding Workshop (ASRU), 2023.","title":"Joint audio and speech understanding","authors":"Gong","year":"2023","venue":"Proceedings of IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gong et al., 2023b] Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlinsky, and James Glass. Listen, think, and understand. ArXiv preprint 2305.10790, 2023.","title":"Listen, think, and understand","authors":"Gong","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[He et al., 2022a] Zihao He, Weituo Hao, Weiyi Lu, Changyou Chen, Kristina Lerman, and Xuchen Song. Al- cap: Alignment-augmented music captioner. In Proceed- ings of Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), 2022.","title":"Alcap: Alignment-augmented music captioner","authors":"He","year":"2022","venue":"Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[He et al., 2022b] Zihao He, Weituo Hao, and Xuchen Song. Recap: Retrieval augmented music captioner. ArXiv preprint 2212.10901, 2022.","title":"Hao, and Xuchen Song. Recap: Retrieval augmented music captioner","authors":"He","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Huang et al., 2022] Qingqing Huang, Aren Jansen, Joon- seok Lee, Ravi Ganti, Judith Yue Li, and Daniel P. W. Ellis. Mulan: A joint embedding of music audio and natural lan- guage. In Proceedings of International Society for Music Information Retrieval Conference (ISMIR), 2022.","title":"Mulan: A joint embedding of music audio and natural language","authors":"Qingqing Huang, Aren Jansen, Joonseok Lee, Ravi Ganti, Judith Yue Li, P W Daniel, Ellis","year":"2022","venue":"Proceedings of International Society for Music Information Retrieval Conference (ISMIR)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Huang et al., 2023] Qingqing Huang, Daniel S. Park, Tao Wang, Timo I. Denk, Andy Ly, Nanxin Chen, Zhengdong Zhang, Zhishuai Zhang, Jiahui Yu, Christian Havnø Frank, Jesse Engel, Quoc V. Le, William Chan, and Weixiang Han. Noise2music: Text-conditioned music generation with diffusion models. ArXiv preprint 2302.03917, 2023.","title":"Noise2music: Text-conditioned music generation with diffusion models","authors":"Qingqing Huang, Daniel S Park, Tao Wang, Timo I Denk, Andy Ly, Nanxin Chen, Zhengdong Zhang, Zhishuai Zhang, Jiahui Yu, Christian Havnø Frank, Jesse Engel, Quoc V Le, William Chan, Weixiang Han","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Khandelwal et al., 2019] Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Gen- eralization through memorization: Nearest neighbor lan- guage models. ArXiv preprint 1911.00172, 2019.","title":"Generalization through memorization: Nearest neighbor language models","authors":"Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kim et al., 2023] Haven Kim, Seungheon Doh, Junwon Lee, and Juhan Nam. Music playlist title generation us- ing artist information. ArXiv preprint 2301.08145, 2023.","title":"Music playlist title generation using artist information","authors":"Kim","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kim, 2020] Christine Sun Kim. [closer captions]. https: //www.youtube.com/watch?v=tfe479qL8hg, 2020.","title":null,"authors":"Christine Sun, Kim","year":null,"venue":null,"doi":null,"url":"https://www.youtube.com/watch?v=tfe479qL8hg,2020","internal_url":null},{"position":23,"raw":"[Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Proceedings of International Conference on Learning Representations (ICLR), 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"Proceedings of International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kuang et al., 2022] Zhihuan Kuang, Shi Zong, Jianbing Zhang, Jiajun Chen, and Hongfu Liu. Music-to-text synaesthesia: Generating descriptive text from music recordings. ArXiv preprint 2210.00434, 2022.","title":"Music-to-text synaesthesia: Generating descriptive text from music recordings","authors":"Zhihuan Kuang, Shi Zong, Jianbing Zhang, Jiajun Chen, Hongfu Liu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lee et al., 2022] Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Proceedings of the An- nual Meeting of the Association for Computational Lin- guistics (ACL)), pages 8424-8445, 2022.","title":"Deduplicating training data makes language models better","authors":"Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini","year":"2022","venue":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL))","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lee et al., 2023] Minhee Lee, Seungheon Doh, and Dasaem Jeong. Annotator subjectivity in the musiccaps dataset. In Proceedings if the HCMIR Workshop at International So- ciety for Music Information Retrieval Conference (ISMIR), 2023.","title":"Annotator subjectivity in the musiccaps dataset","authors":"Minhee Lee, Seungheon Doh, Dasaem Jeong","year":"2023","venue":"Proceedings if the HCMIR Workshop at International Society for Music Information Retrieval Conference (ISMIR)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Lewis et al., 2020] Patrick Lewis, Ethan Perez, Aleksan- dara Piktus, Fabio Petroni, Vladimir Karpukhin, Na- man Goyal, Heinrich Kuttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. ArXiv preprint 2005.11401, 2020.","title":"Retrieval-augmented generation for knowledge-intensive nlp tasks","authors":"Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen Tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Lin and Och, 2004] Chin-Yew Lin and Franz Josef Och. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), pages 605-612, 2004.","title":"Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics","authors":"Lin, Och ; ] Chin-Yew Lin, Franz Josef, Och","year":"2004","venue":"Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proceedings of European Conference on Computer Vision (ECCV), pages 740-755, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"Proceedings of European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Lucía et al., 2020] María J Lucía, Pablo Revuelta, Álvaro García, Belén Ruiz, Ricardo Vergaz, Víctor Cerdán, and Tomás Ortiz. Vibrotactile captioning of musical effects in audio-visual media as an alternative for deaf and hard of hearing people: An eeg study. IEEE Access, 8:190873- 190881, 2020.","title":"Vibrotactile captioning of musical effects in audio-visual media as an alternative for deaf and hard of hearing people: An eeg study","authors":"J María, Pablo Lucía, Álvaro Revuelta, Belén García, Ricardo Ruiz, Vergaz, Tomás Víctor Cerdán, Ortiz","year":"2020","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Manco et al., 2021] Ilaria Manco, Emmanouil Benetos, Elio Quinton, and György Fazekas. Muscaps: Generat- ing captions for music audio. Proceedings of International Joint Conference on Neural Networks (IJCNN), pages 1-8, 2021.","title":"Muscaps: Generating captions for music audio","authors":"Ilaria Manco, Emmanouil Benetos, Elio Quinton, György Fazekas","year":"2021","venue":"Proceedings of International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Manco et al., 2022] Ilaria Manco, Emmanouil Benetos, Elio Quinton, and György Fazekas. Contrastive audio- language learning for music. In Proceedings of Interna- tional Society for Music Information Retrieval Conference (ISMIR), 2022.","title":"Contrastive audiolanguage learning for music","authors":"Ilaria Manco, Emmanouil Benetos, Elio Quinton, György Fazekas","year":"2022","venue":"Proceedings of International Society for Music Information Retrieval Conference (ISMIR)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[McKee et al., 2023] Daniel McKee, Justin Salamon, Josef Sivic, and Bryan Russell. Language-guided music recom- mendation for video via prompt analogies. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14784-14793, 2023.","title":"Language-guided music recommendation for video via prompt analogies","authors":"Mckee","year":"2023","venue":"Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Mei et al., 2022] Xinhao Mei, Xubo Liu, MarkD . Plumb- ley, and Wenwu Wang. Automated audio captioning: An overview of recent progress and new challenges. EURASIP Journal on Audio, Speech, and Music Process- ing, pages 1-18, 2022.","title":"Automated audio captioning: An overview of recent progress and new challenges","authors":"Mei","year":"2022","venue":"EURASIP Journal on Audio, Speech, and Music Processing","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Mokady et al., 2021] Ron Mokady, Amir Hertz, and Amit H Bermano. Clipcap: Clip prefix for image cap- tioning. ArXiv preprint 2111.09734, 2021.","title":"Clipcap: Clip prefix for image captioning","authors":"Ron Mokady, Amir Hertz, Amit H Bermano","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: A method for au- tomatic evaluation of machine translation. In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), pages 311-318, 2002.","title":"Bleu: A method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL)","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In Proceedings of International Conference on Machine Learning (ICML), pages 8748- 8763, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark","year":"2021","venue":"Proceedings of International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Revuelta et al., 2020] Pablo Revuelta, Tomás Ortiz, María J Lucía, Belén Ruiz, and José Manuel Sánchez-Pena. Lim- itations of standard accessible captioning of sounds and music for deaf and hard of hearing people: An eeg study. Frontiers in integrative neuroscience, 14:1, 2020.","title":"Limitations of standard accessible captioning of sounds and music for deaf and hard of hearing people: An eeg study","authors":"Revuelta","year":"2020","venue":"Frontiers in integrative neuroscience","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Srivatsan et al., 2024] Nikita Srivatsan, Sofia Samaniego, Omar U. Florez, and Taylor Berg-Kirkpatrick. Alt-text with context: Improving accessibility for images on twit- ter. In Proceedings of International Conference on Learn- ing Representations (ICLR), 2024.","title":"Alt-text with context: Improving accessibility for images on twitter","authors":"Nikita Srivatsan, Sofia Samaniego, Omar U Florez, Taylor Berg-Kirkpatrick","year":"2024","venue":"Proceedings of International Conference on Learning Representations (ICLR)","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Vedantam et al., 2015] Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus- based image description evaluation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4566-4575, 2015.","title":"Cider: Consensusbased image description evaluation","authors":"Ramakrishna Vedantam, Lawrence Zitnick, Devi Parikh","year":"2015","venue":"Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Wu et al., 2023] Yusong* Wu, Ke* Chen, Tianyu* Zhang, Yuchen* Hui, Taylor Berg-Kirkpatrick, and Shlomo Dub- nov. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmenta- tion. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 1-5, 2023.","title":"Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation","authors":"Wu","year":"2023","venue":"Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang et al., 2023] Yixiao Zhang, Akira Maezawa, Gus G. Xia, Kazuhiko Yamamoto, and Simon Dixon. Loop copi- lot: Conducting ai ensembles for music generation and it- erative editing. ArXiv preprint 2310.12404, 2023.","title":"Loop copilot: Conducting ai ensembles for music generation and iterative editing","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}