{"type":"ScholarlyArticle","title":"Semantic Similarity Is Not Legal Correctness: Evaluating RAG Systems in Brazilian Civil Procedure","authors":[{"name":"Eryclis Silva","slug":"eryclis-silva","given":"Eryclis","family":"Silva","affiliation":"University of Illinois at Urbana-Champaign"},{"name":"Madelyn Sanfilippo","slug":"madelyn-sanfilippo","given":"Madelyn","family":"Sanfilippo","affiliation":"University of Illinois at Urbana-Champaign"}],"abstract":"Evaluating retrieval-augmented generation (RAG) systems in legal domains is challenging due to the nuanced nature of legal reasoning and scarcity of\r\ndomain-specific benchmarks. We investigate semantic similarity and legal correctness in Brazilian legal question answering, developing a synthetic dataset of 3,012 evaluation instances from the Brazilian Civil Procedure Code spanning seven query types, validated through human expert assessment. Our evaluation framework combines BERTScore with domain-adapted LLM-as-Judge (GPT-4o-mini), validated against expert legal assessment. Analysis reveals 50.8% disagreement between semantic similarity and legal correctness, with correlation varying substantially by query type (ρ = 0.464 to ρ = 0.732). Explicit article citation emerges as the strongest quality predictor (Cohen’s d = 1.099), with cited responses achieving 145% higher legal correctness scores. Concrete examples demonstrate bidirectional divergence: responses may achieve high semantic similarity yet cite incorrect articles, or provide legally perfect answers with minimal lexical overlap. Findings demonstrate that semantic metrics alone are insufficient for legal RAG evaluation, with disagreement patterns structured by query type.","doi":"10.24963/ijcai.2026/659","url":"https://ijcai.org/proceedings/2026/659","pdf":"https://ijcai.org/proceedings/2026/0659.pdf","markdown":"https://ijcai.org/proceedings/2026/659.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/659","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Natural Language Processing","track":"Main Track","number":659,"pages":{"first":5919,"last":5927},"keywords":["Natural Language Processing: Question answering","AI: Natural Language Processing","Natural Language Processing: Information retrieval and text mining","Natural Language Processing: Applications"],"videos":[],"references":[{"position":1,"raw":"[Asai et al., 2023] Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. ArXiv, abs/2310.11511, 2023.","title":"Self-rag: Learning to retrieve, generate, and critique through self-reflection","authors":"Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, Hannaneh Hajishirzi","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chalkidis et al., 2020] Ilias Chalkidis, Manos Fergadiotis, et al. LEGAL-BERT: The muppets straight out of law school. In Trevor Cohn, Yulan He, and Yang Liu, editors, Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898-2904, Online, November 2020. Association for Computational Linguistics. [Douze et al., 2025]","title":"LEGAL-BERT: The muppets straight out of law school","authors":"Ilias Chalkidis, Manos Fergadiotis","year":"2020","venue":"Findings of the Association for Computational Linguistics: EMNLP 2020","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre- Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. The faiss library, 2025.","title":"The faiss library","authors":"Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, Hervé Jégou","year":"2025","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Es et al., 2024] Shahul Es, Jithin James, Luis Es- pinosa Anke, and Steven Schockaert. RAGAs: Automated evaluation of retrieval augmented generation. In Nikolaos Aletras and Orphee De Clercq, editors, Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 150-158, St. Julians, Malta, March 2024. Association for Computational Linguistics.","title":"RAGAs: Automated evaluation of retrieval augmented generation","authors":"Shahul Es, Jithin James, Luis Espinosa Anke, Steven Schockaert","year":"2024","venue":"Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gao et al., 2024] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. Retrieval-augmented gen- eration for large language models: A survey, 2024.","title":"Retrieval-augmented generation for large language models: A survey","authors":"Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, Haofen Wang","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gao et al., 2025] Mingqi Gao, Xinyu Hu, Xunjian Yin, Jie Ruan, Xiao Pu, and Xiaojun Wan. LLM-based NLG eval- uation: Current status and challenges. Computational Lin- guistics, 51:661-687, June 2025.","title":"LLM-based NLG evaluation: Current status and challenges","authors":"Gao","year":"2025","venue":"Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Grattafiori et al., 2024] Aaron Grattafiori, Abhimanyu Dubey, et al. The llama 3 herd of models, 2024.","title":"The llama 3 herd of models","authors":"Aaron Grattafiori, Abhimanyu Dubey","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guha et al., 2023a] Neel Guha, Julian Nyarko, et al. Legal- bench: A collaboratively built benchmark for measuring legal reasoning in large language models, 2023.","title":"Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models","authors":"Guha","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Guha et al., 2023b] Neel Guha, Julian Nyarko, et al. Legal- bench: a collaboratively built benchmark for measuring legal reasoning in large language models. In Proceedings of the 37th International Conference on Neural Informa- tion Processing Systems, NIPS '23, Red Hook, NY, USA, 2023.","title":"Legalbench: a collaboratively built benchmark for measuring legal reasoning in large language models","authors":"Guha","year":"2023","venue":"Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Curran Associates Inc. Hu and Zhou, 2024] Taojun Hu and Xiao-Hua Zhou. Un- veiling llm evaluation focused on metrics: Challenges and solutions, 2024.","title":"Unveiling llm evaluation focused on metrics: Challenges and solutions","authors":"Curran Associates, Inc Hu, Zhou, Taojun Hu, Xiao-Hua Zhou","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang and Huang, 2024] Yizheng Huang and Jimmy Huang. A survey on retrieval-augmented text generation for large language models, 2024.","title":"A survey on retrieval-augmented text generation for large language models","authors":"Huang Huang, Yizheng Huang, Jimmy Huang","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2024] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qiang- long Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open ques- tions. ACM Transactions on Information Systems, Novem- ber 2024.","title":"A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions","authors":"Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, Ting Liu","year":"2024","venue":"ACM Transactions on Information Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2025] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qian- glong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large lan- guage models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):1-55, January 2025.","title":"A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions","authors":"Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, Ting Liu","year":"2025","venue":"ACM Transactions on Information Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ioannou et al., 2025] Antreas Ioannou, Andreas Shiamishis, Nora Hollenstein, and Nezihe Merve Gürel. Evaluating the limits of large language models in multilingual legal reasoning, 2025.","title":"Evaluating the limits of large language models in multilingual legal reasoning","authors":"Ioannou","year":"2025","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ji et al., 2023] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of halluci- nation in natural language generation. ACM Computing Surveys, 55(12):1-38, March 2023.","title":"Survey of hallucination in natural language generation","authors":"Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye, Jin Bang, Andrea Madotto, Pascale Fung","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jiang et al., 2023] Albert Q. Jiang, Alexandre Sablayrolles, et al. Mistral 7b, 2023.","title":"Mistral","authors":"Albert Q Jiang, Alexandre Sablayrolles","year":"2023","venue":"Mistral","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Junior et al., 2025] Roseval Malaquias Junior, Ramon Pires, et al. Juru: Legal brazilian large language model from reputable sources, 2025.","title":"Juru: Legal brazilian large language model from reputable sources","authors":"Junior","year":"2025","venue":"Roseval Malaquias Junior","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kalai et al., 2025] Adam Tauman Kalai, Ofir Nachum, San- tosh S. Vempala, and Edwin Zhang. Why language models hallucinate, 2025.","title":"Why language models hallucinate","authors":"Ofir Adam Tauman Kalai, Santosh S Nachum, Edwin Vempala, Zhang","year":"2025","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kalra et al., 2024] Rishi Kalra, Zekun Wu, et al. HyPA- RAG: A hybrid parameter adaptive retrieval-augmented generation system for AI legal and policy applications. In Sachin Kumar, Vidhisha Balachandran, Chan Young Park, Weijia Shi, Shirley Anugrah Hayati, Yulia Tsvetkov, Noah Smith, Hannaneh Hajishirzi, Dongyeop Kang, and David Jurgens, editors, Proceedings of the 1st Workshop on Cus- tomizable NLP: Progress and Challenges in Customiz- ing NLP for a Domain, Application, Group, or Individual (CustomNLP4U), pages 237-256, Miami, Florida, USA, November 2024. Association for Computational Linguis- tics.","title":"HyPA-RAG: A hybrid parameter adaptive retrieval-augmented generation system for AI legal and policy applications","authors":"Rishi Kalra, Zekun Wu","year":"2024","venue":"Proceedings of the 1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kant et al., 2025] Manuj Kant, Sareh Nabi, Manav Kant, Roland Scharrer, Megan Ma, and Marzieh Nabi. Towards robust legal reasoning: Harnessing logical llms in law, 2025.","title":"Towards robust legal reasoning: Harnessing logical llms in law","authors":"Manuj Kant, Sareh Nabi, Manav Kant, Roland Scharrer, Megan Ma, Marzieh Nabi","year":"2025","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Katz et al., 2023] Daniel Martin Katz, Dirk Hartung, Lau- ritz Gerlach, Abhik Jana, and Michael J. Bommarito II. Natural language processing in the legal domain, 2023.","title":"Natural language processing in the legal domain","authors":"Dirk Daniel Martin Katz, Lauritz Hartung, Abhik Gerlach, Michael J Jana, I I Bommarito","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kim et al., 2022] Mi-Young Kim, Juliano Rabelo, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, and Ken Satoh. Coliee 2022 summary: Methods fornbsp;legal document retrieval andnbsp;entailment. In New Frontiers in Artificial Intelligence: JSAI-IsAI 2022 Workshop, JU- RISIN 2022, and JSAI 2022 International Session, Kyoto, Japan, June 12-17, 2022, Revised Selected Papers, page 51-67, Berlin, Heidelberg, 2022. Springer-Verlag.","title":"Coliee 2022 summary: Methods fornbsp;legal document retrieval andnbsp;entailment","authors":"Mi-Young Kim, Juliano Rabelo, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, Ken Satoh","year":"2022","venue":"New Frontiers in Artificial Intelligence: JSAI-IsAI 2022 Workshop, JU-RISIN 2022","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2023] Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. Synthetic data generation with large lan- guage models for text classification: Potential and limi- tations. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10443- 10461, Singapore, December 2023. Association for Com- putational Linguistics.","title":"Synthetic data generation with large language models for text classification: Potential and limitations","authors":"Li","year":"2023","venue":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2024] Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. The dawn after the dark: An empirical study on factuality hal- lucination in large language models. In Lun-Wei Ku, An- dre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pages 10879- 10899, Bangkok, Thailand, August 2024. Association for Computational Linguistics.","title":"The dawn after the dark: An empirical study on factuality hallucination in large language models","authors":"Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen","year":"2024","venue":"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lin, 2004] Chin-Yew Lin. ROUGE: A package for auto- matic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics.","title":"ROUGE: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text Summarization Branches Out","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Liu et al., 2023] Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using gpt-4 with better human alignment. In","title":"G-eval: NLG evaluation using gpt-4 with better human alignment","authors":"Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"Houda Bouamor, Juan Pino, and Kalika Bali, editors, Pro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511-2522, Singa- pore, December 2023. Association for Computational Lin- guistics.","title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","authors":"Houda Bouamor, Juan Pino, Kalika Bali","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Liu et al., 2024] Ruibo Liu, Jerry Wei, Fangyu Liu, Chen- glei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, and Andrew M. Dai. Best practices and lessons learned on synthetic data, 2024.","title":"Best practices and lessons learned on synthetic data","authors":"Liu","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Long et al., 2024] Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. On LLMs-driven synthetic data generation, curation, and evaluation: A survey. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024, pages 11065- 11082, Bangkok, Thailand, August 2024. Association for Computational Linguistics.","title":"On LLMs-driven synthetic data generation, curation, and evaluation: A survey","authors":"Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, Haobo Wang","year":"2024","venue":"Findings of the Association for Computational Linguistics: ACL 2024","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Nguyen et al., 2025] Ha Thanh Nguyen, Wachara Fung- wacharakorn, et al. Llms for legal reasoning: A unified framework and future perspectives. Computer Law Secu- rity Review, 58:106165, 2025.","title":"Llms for legal reasoning: A unified framework and future perspectives","authors":"Nguyen","year":"2025","venue":"Computer Law Security Review","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Nigam et al., 2025] Shubham Kumar Nigam, Balaramama- hanthi Deepak Patnaik, et al. Nyayarag: Realistic legal judgment prediction with rag under the indian common law system, 2025.","title":"Nyayarag: Realistic legal judgment prediction with rag under the indian common law system","authors":"Shubham Kumar Nigam, Balaramamahanthi Deepak Patnaik","year":"2025","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Oliva et al., 2025] Maria Paz Oliva, Adriana Correia, Ivan Vankov, and Viktor Botev. The illusion of a perfect metric: Why evaluating ai's words is harder than it looks, 2025.","title":"The illusion of a perfect metric: Why evaluating ai's words is harder than it looks","authors":"Maria Paz, Oliva, Adriana Correia, Ivan Vankov, Viktor Botev","year":"2025","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computa- tional Linguistics, ACL '02, page 311-318, USA, 2002. Association for Computational Linguistics.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Pipitone and Alami, 2024] Nicholas Pipitone and Ghita Houir Alami. Legalbench-rag: A benchmark for retrieval-augmented generation in the legal domain, 2024.","title":"Legalbench-rag: A benchmark for retrieval-augmented generation in the legal domain","authors":"Alami Pipitone, Nicholas Pipitone, Ghita Houir, Alami","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Reimers and Gurevych, 2019] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks, 2019. [Robertson and Zaragoza, 2009] Stephen Robertson and Hugo Zaragoza. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333-389, April 2009.","title":"Sentence-bert: Sentence embeddings using siamese bert-networks","authors":"Gurevych Reimers, Nils Reimers, Iryna Gurevych, Stephen Robertson, Hugo Zaragoza","year":"2009","venue":"Trends Inf. Retr","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Saad-Falcon et al., 2024] Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. ARES: An auto- mated evaluation framework for retrieval-augmented gen- eration systems. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 338-354, Mexico City, Mexico, June 2024. Association for Computational Lin- guistics.","title":"ARES: An automated evaluation framework for retrieval-augmented generation systems","authors":"Jon Saad-Falcon, Omar Khattab, Christopher Potts, Matei Zaharia","year":"2024","venue":"Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2025] Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Hallucination is inevitable: An innate lim- itation of large language models, 2025.","title":"Hallucination is inevitable: An innate limitation of large language models","authors":"Ziwei Xu, Sanjay Jain, Mohan Kankanhalli","year":"2025","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yang et al., 2023] Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, and Xia Hu. Harnessing the power of llms in practice: A survey on chatgpt and beyond. 2023.","title":"Harnessing the power of llms in practice: A survey on chatgpt and beyond","authors":"Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, Xia Hu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yu et al., 2024] Hao Yu, Aoran Gan, Kai Zhang, Shiwei Tong, Qi Liu, and Zhaofeng Liu. Evaluation of retrieval- augmented generation: A survey, 2024.","title":"Evaluation of retrievalaugmented generation: A survey","authors":"Hao Yu, Aoran Gan, Kai Zhang, Shiwei Tong, Qi Liu, Zhaofeng Liu","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2020] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert, 2020.","title":"Bertscore: Evaluating text generation with bert","authors":"Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, Yoav Artzi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang et al., 2023] Xiang Zhang, Senyu Li, et al. Don't trust ChatGPT when your question is not in English: A study of multilingual abilities and types of LLMs. In","title":"Don't trust ChatGPT when your question is not in English: A study of multilingual abilities and types of LLMs","authors":"Xiang Zhang, Senyu Li","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"Houda Bouamor, Juan Pino, and Kalika Bali, editors, Pro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7915-7927, Singa- pore, December 2023. Association for Computational Lin- guistics. Zheng et al., 2023] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a- judge with mt-bench and chatbot arena, 2023.","title":"Judging llm-as-ajudge with mt-bench and chatbot arena","authors":"Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P Xing, Hao Zhang, Joseph E Gonzalez, Ion Stoica","year":"2023","venue":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}