Proceedings

Keyword Spices: A New Method for Building Domain-Specific Web Search Engines

Download BibTeX Cite

Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence. Pages 1457-1465.

© 2001 International Joint Conferences on Artificial Intelligence

Abstract

This paper presents a new method for building domain-specific web search engines. Previous methods eliminate irrelevant documents from the pages accessed using heuristics based on human knowledge about the domain in question. Accordingly, they are hard to build and can not be applied to other domains. The keyword spice method, in contrast, improves search performance by adding domain-specific keywords, called keyword spices, to the user’s input query; the modified query is then forwarded to a general-purpose search engine. Keyword spices can be effectively discovered automatically from web documents allowing us to build high quality domain-specific search engines in various domains without requiring the collection of heuristic knowledge. We describe a machine learning algorithm, which is a type of decision-tree learning algorithm, that can extract keyword spices. To demonstrate the value of the proposed approach, we conduct experiments in the domain of cooking. The results confirm the excellent performance of our method in terms of both precision and recall.

  1. [Baeza-Yates and Ribeiro-Neto, 1999] Ricardo Baeza-Yates and Berthier Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley, 1999.
  2. [Cohen, 1998] William W. Cohen. A web-based information system that reasons with structured collections of text. In Agents'98, pages 116-123, 1998.
  3. [Craven et al., 1998] Mark Craven, Dan DiPasquo, Dayne Freitag, Andrew McCallum, Tom Mitchell, Kamal Nigam, and Seán Slattery. Learning to extract symbolic knowledge from the world wide web. In AAAI-98, pages 509-516, 1998.
  4. [Etzioni, 1996] Oren Etzioni. Moving up the information food chain: Deploying softbots on the world wide web. In AAAI-96, pages 1322-1326, 1996.
  5. [McCallum et al., 1999] Andrew McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. A machine learning approach to building domain-specific search engines. In IJCAI-99, pages 662-667, 1999. IJCAI
  6. [Mitchell, 1997] Tom M. Mitchell. Machine Learning. McGraw-Hill, 1997.
  7. [Quinlan, 1986] J. Ross Quinlan. Induction of decision trees. Machine Learning, 1:81-106, 1986.
  8. [Quinlan, 1993] J. Ross Quinlan. C4.5:Programs for Ma- chine Learning. Morgan Kaufmann, 1993.
  9. [Shakes et al., 1997] Jonathan Shakes, Marc Langheinrich, and Oren Etzioni. Dynamic reference sifting: a case study in the homepage domain. In Proceedings of the 6th Inter- national World Wide Web Conference(WWW6), pages 189- 200, 1997.
  10. [Shaw Jr. et al., 1997] W. M. Shaw Jr., Robert Burgin, and Patrick Howell. Performance standards and evaluations in ir test collections: Cluster-based retrieval models. Infor- mation Processing & Management, 33(1):1-14, 1997.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards