{"type":"ScholarlyArticle","title":"Upgrading Search Applications in the Era of LLMs: A Demonstration with Practical Lessons","authors":[{"name":"Shuang Yu","slug":"shuang-yu","given":"Shuang","family":"Yu","affiliation":"IBM Australia"},{"name":"Nirandika Wanigasekara","slug":"nirandika-wanigasekara","given":"Nirandika","family":"Wanigasekara","affiliation":"IBM Australia"},{"name":"Jeff Tan","slug":"jeff-tan","given":"Jeff","family":"Tan","affiliation":"IBM Australia"},{"name":"Kent Fitch","slug":"kent-fitch","given":"Kent","family":"Fitch","affiliation":"National Library of Australia"},{"name":"Jaydeep Sen","slug":"jaydeep-sen","given":"Jaydeep","family":"Sen","affiliation":"IBM Research"}],"abstract":"While traditional search systems have mostly been satisfactorily relying on lexical based sparse retrievers such as BM25, recent research advances in neural models, the current day large language models (LLMs) hold good promise for practical search applications as well. In this work, we discuss a collaboration between IBM and National Library of Australia to upgrade an existing search application (referred to as NLA) over terabytes of Australian Web Archive data and serving thousands of daily users. We posit and demonstrate both empirically and through qualitative user studies that LLMs and neural models can indeed provide good gains, when combined effectively with traditional search. We believe this demonstration will show the unique challenges associated with real world practical deployments and also offer valuable insights into how to effectively upgrade legacy search applications in the era of LLMs.","doi":"10.24963/ijcai.2024/1045","url":"https://ijcai.org/proceedings/2024/1045","pdf":"https://ijcai.org/proceedings/2024/1045.pdf","markdown":"https://ijcai.org/proceedings/2024/1045.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/1045","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Demo Track","track":"Demo Track","number":1045,"pages":{"first":8843,"last":8846},"keywords":["Natural Language Processing: NLP: Applications","Natural Language Processing: NLP: Question answering","Natural Language Processing: NLP: Natural language semantics","Natural Language Processing: NLP: Information retrieval and text mining"],"videos":[],"references":[{"position":1,"raw":"[CDX, 2024] CDX. Cdxformat. https://archive.org/web/ researcher/cdx file format.php, 2024.","title":"CDX. Cdxformat","authors":"Cdx","year":"2024","venue":null,"doi":null,"url":"https://archive.org/web/researcher/cdxfileformat.php","internal_url":null},{"position":2,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understand- ing. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapo- lis, Minnesota, June 2019. Association for Computational Linguistics.","title":"BERT: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Karpukhin et al., 2020] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. Dense passage retrieval for open-domain question answering, 2020.","title":"Dense passage retrieval for open-domain question answering","authors":"Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen Tau, Yih","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Khattab and Zaharia, 2020] Omar Khattab and Matei Za- haria. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR Conference on Re- search and Development in Information Retrieval, SIGIR '20, page 39-48, New York, NY, USA, 2020. Association for Computing Machinery. [likert, 2020] likert. Likert scale. https://en.wikipedia.org/ wiki/Likert scale, 2020.","title":"Colbert: Efficient and effective passage search via contextualized late interaction over bert","authors":"Omar Khattab, Matei Zaharia","year":"2020","venue":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '20","doi":null,"url":"https://en.wikipedia.org/wiki/Likertscale,2020","internal_url":null},{"position":5,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Milvus, 2023] Milvus. Milvus vector db. https://milvus.io/, 2023.","title":"Milvus. Milvus vector","authors":"Milvus","year":"2023","venue":null,"doi":null,"url":"https://milvus.io/,2023","internal_url":null},{"position":7,"raw":"[Rank, 2008] Wiki:Mean Reciprocal Rank. Mean reciprocal rank. https://en.wikipedia.org/wiki/Mean reciprocal rank, 2008.","title":"Wiki:Mean Reciprocal Rank. Mean reciprocal rank","authors":"Rank","year":"2008","venue":null,"doi":null,"url":"https://en.wikipedia.org/wiki/Meanreciprocalrank","internal_url":null},{"position":8,"raw":"[Robertson and Zaragoza, 2009] Stephen E. Robertson and Hugo Zaragoza. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333-389, 2009.","title":"The probabilistic relevance framework: BM25 and beyond","authors":"Stephen E Robertson, Hugo Zaragoza","year":"2009","venue":"Found. Trends Inf. Retr","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Santhanam et al., 2022] Keshav Santhanam, Omar Khat- tab, Jon Saad-Falcon, Christopher Potts, and Matei Za- haria. ColBERTv2: Effective and efficient retrieval via lightweight late interaction. In Marine Carpuat, Marie- Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, ed- itors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3715- 3734, Seattle, United States, July 2022. Association for Computational Linguistics.","title":"ColBERTv2: Effective and efficient retrieval via lightweight late interaction","authors":"Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, Matei Zaharia","year":"2022","venue":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[sentence transformers, 2022] sentence transform- ers. all-mini-lm-l6-v2. https://huggingface.co/ sentence-transformers/all-MiniLM-L6-v2/tree/main, 2022. [Solr, 2010] Solr. Solr search engine. https://solr.apache. org/, 2010. [Stanford:BM25, 2009] Stanford:BM25. Bm25 retrieval. https://nlp.stanford.edu/IR-book/html/htmledition/ okapi-bm25-a-non-binary-model-1.html, 2009.","title":"Solr. Solr search engine","authors":null,"year":"2009","venue":null,"doi":null,"url":"https://nlp.stanford.edu/IR-book/html/htmledition/okapi-bm25-a-non-binary-model-1.html","internal_url":null},{"position":11,"raw":"[Touvron et al., 2023] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Can- ton Ferrer, Moya Chen, Guillem Cucurull, David Es- iobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, An- gela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv e-prints, page arXiv:2307.09288, July 2023.","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","authors":"Touvron","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[VectorDBs, 2010] Summary: VectorDBs. Vector databases. https://www.elastic.co/what-is/vector-database, 2010.","title":"Summary: VectorDBs. Vector databases","authors":"Vectordbs","year":"2010","venue":null,"doi":null,"url":"https://www.elastic.co/what-is/vector-database,2010","internal_url":null},{"position":13,"raw":"[Wang et al., 2013] Yining Wang, Liwei Wang, Yuanzhi Li, Di He, Tie-Yan Liu, and Wei Chen. A theoretical anal- ysis of ndcg type ranking measures. Journal of Machine Learning Research, 30, 04 2013.","title":"A theoretical analysis of ndcg type ranking measures","authors":"Yining Wang, Liwei Wang, Yuanzhi Li, Di He, Tie-Yan Liu, Wei Chen","year":"2013","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[WARC, 2024] WARC. Warc format. https://www.loc.gov/ preservation/digital/formats/fdd/fdd000236.shtml, 2024.","title":"WARC. Warc format","authors":"Warc","year":"2024","venue":null,"doi":null,"url":"https://www.loc.gov/preservation/digital/formats/fdd/fdd000236.shtml,2024","internal_url":null},{"position":15,"raw":"[Zhao et al., 2023] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Z. Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jianyun Nie, and Ji rong Wen. A survey of large language models. ArXiv, abs/2303.18223, 2023.","title":"A survey of large language models","authors":"Kun Wayne Xin Zhao, Junyi Zhou, Tianyi Li, Xiaolei Tang, Yupeng Wang, Yingqian Hou, Beichen Min, Junjie Zhang, Zican Zhang, Yifan Dong, Chen Du, Yushuo Yang, Z Chen, Jinhao Chen, Ruiyang Jiang, Yifan Ren, Xinyu Li, Zikang Tang, Peiyu Liu, Jianyun Liu, Ji Nie, Wen Rong","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}