SKIMMING NEWSPAPER STORIES BY COMPUTER

SKIMMING NEWSPAPER STORIES BY COMPUTER

Gerald DeJong Computer Science Department Yale University

New Haven, Connecticut 06520

INTRODUCTION

FRUMP (Fast Reading Understanding and Memory Program) is a system being developed at Yale to skim newspaper stories. The United Press Inter­ national news service has recently been connected to the Yale computer to provide FRUMP with real-world data. FRUMP's understanding is based on the concept of a script. A script is a data structure used to represent knowledge about very stylized events.

FRUMP uses what we call "sketchy scripts". The crucial difference between sketchy scripts and the detailed scripts used in other Yale pro­ jects [Cullingford, 1977] is that only the con­ cepts corresponding to the most important events in a detailed script are included. Each of these important events is represented by a "request" in conceptual dependency format. The requests con­ t~in blank slots that are filled when the story is processed. There is a different sketchy script for each type of newspaper story that FRUMP can process. The blank slots are the script variables for that script. At this writ­ ing FRUMP can understand six types of news events: earthquakes, vehicle accidents, heads of states dying, floods, military fighting and meetings of heads of state. We expect FRUMP to have many more scripts by the end of the summer. For FRUMP, understanding is marking the proper requests as satisfied and binding its script variables to things found in the text.

FRUMP is made up of two parts: the concep­ tual fragment parser and the script applier. The parser was inspired by Becker's phrasal lexicon [Becker 1975]. It tries to satisfy the requests of the sketchy script with sentences or fragments of sentences from the input. The script applier guides the parser in selecting meanings for am­ biguous phrases, binds script variables, and loads new requests when appropriate. The top down nature of FRUMP allows it to be very effi­ cient. A news story is typically processed in ten seconds of CPU time on a DEC KA10 processor.

Cl.ASSIFICATION OF ~WS STORIES

A file is maintained of understood news events. Each file entry consists of the requests and bound script variables of the sketchy script used for understanding that event. Before a

The research described in this paper was sup­ ported by the Advanced Research Projects Agency of the Department of Defense and monitored by the Office of Naval Research under contract

N00014-75-G-1111.

story can be processed, FRUMP must classify the story (to select the proper sketchy script) and decide whether it is an update of a previous story or a report of a new event. Thi$ is done by making an initial scan through the first paragraph of the story with a special set of re­ quests active. The set of requests is made up of one key request from each of FRUMP's scripts. A key request is a request which, if satisfied, strongly indicates that the associated script is appropriate for understanding the story. When one of these requests is satisfied, FRUMP knows not only what script to use but also (from the script variables of the request) some information about this particular story. That information is matched against the corresponding information of previously understood stories to decide if the new story is an update. For a more complete de­ scription of FRUMP, see [DeJong, 1977].

AN EXAMPLE

The following consists of the lead para­ graphs of an actual article. The summaries are produced by canned templates. The templates contain blanks filled by properties of the script variables. Since the internal representation is conceptual dependency, it is as easy to generate summaries in Spanish and Russian as English.

INPUT:

3 - 4 PISA, ITALY OFFICIALS TODAY SEARCHED FOR THE Bl.ACK BOX FLIGHT RECORDER ABOARD AN ITALIAN AIR FORCE TRANSPORT PLANE TO DETERMINE WHY THE CRAFT CRASHED INTO A MOUNTAINSIDE KILLING 44 PERSONS.

THEY SAID THE WEATHER WAS CALM AND CLEAR, EXCEPT FOR SOME GROUND LEVEL FOG, WHEN THE U S MADE HERCULES C130 TRANSPORT Pl.ANE HIT MT. SERRA

MOMENTS AFTER TAKEOFF THURSDAY.

THE PILOT, DESCRIBED AS ONE OF THE COUNTRY'S MOST EXPERIENCED, DID NOT REPORT ANY TROUBLE IN A

BRIEF RADIO CONNVERSATION BEFORE THE CRASH.

ENGLISH SUMMARY: 44 PEOPLE WERE KILLED WHEN AN

AIRPl.ANE CRASHED INTO A MOUNTAIN IN ITALY.

SPANISH SUMMARY: HUBO UN ACCIDENTE DE AVION EN ITALIA QUE RESULTO EN 44 MUERTOS.

RUSSIAN SUMMARY: V AVIATSIONNOI KATASTROFE . V

ITALII 44 CHELOVEK BYLO UBITO. REFERENCES

Becker, J.,

"The Phrasal Lexicon", Proc. Theoretical Issues In Natural Language Processing Work­ shop, Cambridge, Mass. June, 1975.

Cullingford, R.,

"Organizing derstanding Engineering World Knowledge for Story Un­ by Computer", Ph.D. Thesis, and Applied Science Dept., Yale

University, 1977.

DeJong, G.,

"Skimming Newspaper Stories by Computer", Research Report 104, Computer Science De­ partment, Yale University, May 1977.