Getting Serious about the Development of Computational Humor

1 Introduction

Future human-machine interaction, according to a widespread view, will emphasize naturalness and ef!ec­ tiveness and hence the incorporation of models of poss1bly all human cognitive capabilities, including the handling of humor. There are many practical settings where computa­ tional humor will add value. Among them: business world applications (e-commerce to name one), general_compu~er­ mediated communication and human-computer mteractton [Morkes et al., 1999], educational and edutainment systems. There are important prospects for humor also in automatic information presentation. In the Web age presentations become more and more flexible and personalized and they will require humor contributions for electron~c comme~ce developments (e.g. product promotion, gettmg selective attention, help in memorizing names, etc ... ) more or less as it happened in the world of broadcasted advertisement.

Yet deep modeling of humor in all of its facets is not some­ thing for the near future; the phenomena are too compl~x, humor is one of the most sophisticated forms of human m­ telligence. It is AI-complete: the problem of modeling it is as difficult to solve as the most difficult Artificial Intelligence problems. But some steps can be followed to achieve results. In the general case, in order to be successfully humorous, a computational system should be able to: recognize situa­ tions appropriate for humor; choose a suitable kind of humor for the situation; generate an appropriately humorous output; and, if there is some form of interaction or control, evaluate the feedback.

We are concerned with systems that automatically produce humorous output (rather than systems that appreciate humor). Some of the fundamental competencies are within the range of the state of the art of natural language processing. In one form or in another humor is most often based on some form of incongruity. For verbal humor this means that different in­ terpretations of utterances must be possible (and must not be detected before the culmination of the humorous process) or must cause perception of specific forms of opposition. Nat­ urallanguage processing research has often dealt with ambi­ guity in language. A common view is that ambiguity is an obstacle for deep comprehension. Most current text process­ ing systems attempt to reduce the number of possible inter­ pretations of the sentences, and a failure to do so is seen as a weakness of the system. From a different point of view, the potential for ambiguity can be seen as a positive feature of natural language. Metaphors, idioms, poetic language and humor use all the multiple senses of texts to suggest connec­ tions between concepts that cannot, or should not, be stated explicitly.

The work presented here is based on various resources for natural language processing, adapted for humor. It is a small step, but aiming at an appreciable concrete re~ult. It has been developed within HAHACRONYM 1 , a proJect devoted to computational humor. A visible and evaluable result was at the basis of the deal. We proposed a situation that is of practical interest, where there is no domain restriction and many components are present, but simpler than in more ex­ tended scenarios. The goal is a system that makes fun of ex­ isting acronyms, or, starting from concepts provided by the user, produces a new acronym, constrained to be a word of the given language. And, of course, it has to be funny. .

The project was meant to convince about the potential of computational humor, through the demonstration of a work­ ing prototype and an assessment of the state of the art and of scenarios where humor can add something to existing in­ formation technologies. The results of the project put us in a better position to move forwards in introducing computa­ tional humor in more complex scenarios.

'European Project IST-2000-30039. HAHACRONYM has been the first EU project devoted to computational humor. The consor­ tium included ITC-irst, as coordinator, and the University ofTwente. See http: I /haha. itc .it.

2 AI and Computational Humor

So far only very limited effort has been put on building com­ putational humor prototypes. Indeed very few working pro­ totypes that process humorous text and/or simulate humor mechanisms exist, and mostly they worked in very restricted domains.

There has been a considerable amount of research on lin­ guistics of humor and on theories of semantics and prag­ matics of humor [Attardo, 1994; Attardo and Raskin, 1991; Giora and Fein, 1999; Attardo, 2002]; however, most of the work has not been formal enough to be used directly for com­ putational humor modeling. An effort toward formalization of forced reinterpretation jokes has been presented by Ritchie [2002].

Within the artificial intelligence community, most writ­ ing on humor has been speculative [Minsky, 1980; Hofs­ tadter et al., 1989]. Minsky made some preliminary remarks about how humor could be viewed from the artificial intelli­ gence/cognitive science perspective, refining Freud's notion that humor is a way of bypassing our mental "censors" which control inappropriate thoughts and feelings. Utsumi [ 1996] outlines a logical analysis of irony, but this work has not been implemented. Among other works: De Palma and Weiner [1992] have worked on knowledge representation of riddles, Ephratt [ 1990] has constructed a program that parses a lim­ ited range of ambiguous sentences and detects alternative hu­ morous readings. A formalization, based on a cognitive ap­ proach (the belief-desire-intention model), distinguishing be­ tween real and fictional humor has been provided by Mele [2002].

Probably the most important attempt to create a computa­ tional humor prototype is the work of Binsted and Ritchie [ 1997]. They have devised a formal model of the seman­ tic and syntactic regularities underlying some of the simplest types of punning riddles. A punning riddle is a question­ answer riddle that uses phonological ambiguity. The three main strategies used to create phonological ambiguity are syl­ lable substitution, word substitution and metathesis.

Almost all the computational approaches try to deal with the incongruity theory at various level of refinement [Raskin, 1985; Attardo, 1994]. The incongruity theory focuses on the element of surprise. It states that humor is created out of a conflict between what is expected and what actually occurs in the joke. This accounts for the most obvious features of a large part of humor phenomena: ambiguity or double mean­ ing.

Specific workshops concerned with Computational Humor have taken place in recent years and have drawn together most of the community active in the field. The proceedings of the most comprehensive events are [Hulstijn and Nijholt, 1996] and [Stock et at., 2002]. Ritchie [200 1] has published a sur­ vey of the state of the art in the field.

3 HAHACRONYM

3.1 Resources

In order to realize the HAHACRONYM prototype, we have re­ fined existing resources and we have developed general tools useful for humorous systems. As we will see, a fundamental tool is an incongruity detector/generator, that makes the sys­ tem able to detect semantic mismatches between word mean­ ing and sentence meaning (i.e. in our case the acronym and its context). For all tools, particular attention was put on reusability.

The starting point for us consisted in making use of some "off-the-shelf' resources, such as WORDNET DOMAINS [Magnini et al., 2002] (an extension of the well-known En­ glish WORDNET) and standard parsing techniques. The tools resulting from the adaptation will be reusable for other appli­ cations, and are portable straightforwardly to other languages (e.g. WORDNET DOMAINS is multilingual).

Wordnet and Wordnet Domains

WORDNET is a thesaurus for the English language inspired by psycholinguistics principles and developed at the Prince­ ton University by George Miller [Fellbaum, 1998]. It has been conceived as a computational resource, therefore im­ proving some of the drawbacks of traditional dictionaries, such as circularity of definitions and ambiguity of sense ref­ erences. Lemmata (about 130,000 for version 1.6) are orga­ nized in synonym classes (about 100,000 synsets). WORD­ NET can be described as a "lexical matrix" with two di­ mensions: a dimension for lexical relations, that is rela­ tions holding among words, and therefore language spe­ cific, and a dimension for conceptual relations, which hold among senses (the synsets) and that, at least in part, we con­ sider independent from a particular language. A synset con­ tains all the words by means of which it is possible to ex­ press the synset meaning: for example the synset {horse, Equus_caballus} describes the sense of "horse" as an animal, while the synset {knight, horse} describes the sense of "horse" as a chessman. The main relations present in WORDNET are synonymy, antonymy, hyperonymy­ hyponymy, meronymy-holonymy, entailment, troponymy.

Augmenting WordNet with Domain information

Domains have been used both in linguistics (i.e. Semantic Fields) and in lexicography (i.e. Subject Field Codes) to mark technical usages of words. Although this is useful informa­ tion for sense discrimination, in dictionaries it is typically used for a small portion of the lexicon. WORDNET Do­ MAINS is an attempt to extend the coverage of domain labels within an already existing lexical database, WORDNET (ver­ sion 1.6). The synsets have been annotated with at least one domain label, selected from a set of about two hundred la­ bels hierarchically organized. (Figure 1 shows a portion of the domain hierarchy.)

We have organized about 250 domain labels in a hierar­ chy (exploiting Dewey Decimal Classification), where each level is made up of codes of the same degree of specificity: for example, the second level includes domain labels such as BOTANY, LiNGUISTICS, HISTORY, SPORT and RELIGION, while at the third level we can find specialization such as AMERICAN_HISTORY, GRAMMAR, PHONETICS and TEN­ NIS.

Information brought by domains is complementary to what is already present in WORDNET. First of all a domain may include synsets of different syntactic categories: for instance MEDICINE groups together senses from Nouns, such as doctor#! and hospi tal#l, and from Verbs such as operate#?. Second, a domain may include senses from different WORDNET sub-hierarchies For ex­ ample, SPORT contains senses such as athlete#!, de­ riving from life_forrn#l, garne...equiprnent#l, from physicaLobject#l, sport#l from act#2, and playing_field#l, from location#!.

Opposition of semantic fields

On the basis of well recognized properties of humor ac­ counted for in many theories (e.g. incongruity, semantic field opposition, apparent contradiction, absurdity) we have mod­ elled an independent structure of domain opposition, such as RELIGION vs. TECHNOLOGY, SEX vs. RELIGION, etc ... We exploit these kind of opposition as a basic resource for the incongruity generator.

Adjectives and Antonymy Relations

Adjectives play an important role in modifying and gen­ erating funny acronyms. WORDNET divides adjectives into two categories. Descriptive adjectives (e.g. big, beautiful, interesting, possible, married) constitute by far the largest category. The second category is called simply relational adjectives because they are related by derivation to nouns (i .e. electrical in electrical engineering is related to noun electricity). Tore­ lational adjectives, strictly dependent on noun meanings, it is often possible to apply similar strategies as those exploited for nouns. Their semantic organization, though, is entirely different from the one of the other major categories. In fact it is not clear what it would mean to say that one adjective "is a kind of' (ISA) some other adjective. The basic semantic relation among descriptive adjectives is antonymy. WORD­ NET proposes also that this kind of adjectives is organized in clusters of synsets associated by semantic similarity to a focal adjective. Figure 2 shows clusters of adjectives around the direct antonyms fast/slow.

Exploiting the hierarchy

It is possible to exploit the network of lexical and seman­ tic relations built in WORDNET to make simple ontological reasoning. For example, if a noun or an adjective has a ge­ ographic location meaning, the pertaining country and conti­ nent can be inferred.

Rhymes

The HAHACRONYM prototype takes into account word rhymes and the rhythm of the acronym expansion. To cope with this aspect we got and reorganized the CMU pronouncing dictionary (http://www.speech.cs.cmu.edu/cgi-bin/cmudict) with a suitable indexing. The CMU Pronouncing Dictionary is a machine-readable pronunciation dictionary for North American English that contains over 125,000 words and their transcriptions.

Its format is particularly useful for speech recognition and synthesis, as it has mappings from words to their pronunci­ ations in the given phoneme set. The current phoneme set contains 39 phonemes; vowels may carry lexical stress.

Parser, grammar and morphological analyzer

Word sequences that are at the basis of acronyms are sub­ ject to a well-defined grammar, simpler than a complete noun phrase grammar, but complex enough to require a nontrivial analyzer. We have decided to use a well established nonde­ terministic parsing technique. As far as the dictionary is con­ cerned, we use the full WORDNET lexicon, integrated with an ad-hoc morphological analyzer.

Also for the generation part we exploit the grammar as the source for syntactic constraints.

All the components are implemented in Common Lisp aug­ mented with nondeterministic constructs.

Other resources

An "a-semantic" dictionary is a collection of hyper­ bolic/epistemic/deontic adjective/adverbs. This is a last re­ source, that some time can be useful in the generation of new acronyms. Some examples are: abnormally, ab­ strusely, adorably, exceptionally, exorbitantly, exponentially, extraordinarily, voraciously, weirdly, wonderfully. This re­ source is hand-made, using various dictionaries as informa­ tion sources.

Other lexical resources are: a euphemism dictionary, a proper noun dictionary, lists of typical foreign words com­ monly used in the language with some strong connotation.

3.2 Architecture and Implementation

To get an ironic or "profaning" re-analysis of a given acronym, the system follows various steps and strategies. The main elements of the algorithm can be schematized as fol­ lows:

• acronym parsing and construction of a logical form
• choice of what to keep unchanged (typically the head of the highest ranking NP) and what to modify (e.g. the adjectives)
• look up for possible substitutions
• exploitation of semantic field oppositions
• granting phonological analogy: while keeping the con­ straint on the initial letters of the words, the overall rhyme and rhythm should be preserved (the modified acronym should sound similar to the original as much as possible)
• exploitation of WORDNET antonymy clustering for ad­ jectives
• use of the a-semantic dictionary as a last resource

Figures 3 and 4 show a sketch of the system architecture.

In our system, making fun of existing acronyms amounts to an ironical rewriting, desecrating them with some unexpect­ edly contrasting, but otherwise consistently sounding expan­ siOn.

As far as acronym generation is concerned, the problem is more complex. To make the task more attractive - and dif­ ficult - we constrain resulting acronyms to be words of the dictionary (APPLE would be good, IBM would not). The system takes in input concepts (actually synsets, possibly the result of some other process, for instance sentence interpre­ tation) and some minimal structural indication, such as the semantic head. The primary strategy of the system is to con­ sider as potential acronyms words that are in ironic relation with the input concepts. By definition acronyms have to sat­ isfy constraints - to include the initial letters of some lexical realization of the inputs words synsets, granting that the se­ quence of initials satisfy the overall acronym syntax. In this primary strategy, the ironic reasoning comes mainly at the level of acronym choice in the lexicon and in the selection of the fillers of the open slots in the acronym.

For example, giving as input "fast" and "CPU", we get static, torpid, dormant. (Note that the com­ plete synset for "CPU" is {processor#3, CPU#l, centraLprocessing_unit#l , rnainfrarne#2}. So we can use in the acronym expansion a synonym of "CPU". The same happens for "fast"). Once we have an acronym proposal, a syntactic skeleton has to be filled to get a correct noun phrase. For example given in input "fast" and "CPU", the system selects TORPID with the possible syntactic skele­ ton:

< adv >T < adj >o Rapid Processor <prep>< adj > 1 < noun > o or

< adj >T < adj >o Rapid Processor <prep>< noun >1 <noun >o

where "rapid" and "processor" are synonyms re­ spectively of "fast" and "CPU" and the notation < Part_oLSpeech> Letter means a word of that par­ ticular parLof...speech with Letter as initial. Then the system fills this syntactic skeleton with strategies similar to those described for re-analysis.

The system is fully implemented in Common Lisp, exploit­ ing CLOS and the Meta-Object protocol to import WORD­ NET DOMAINS and to implement the navigation/reasoning algorithms.

3.3 Examples

Here below some examples of acronym re-analysis are re­ ported. As far as semantic field opposition is concerned we have slightly tuned the system towards the domains FOOD, RELIGION and SEX. We report the original acronym, there­ analysis and some comments about the strategies followed by the system.

ACM - Association for Computing Machinery ~Association for Confusing Machinery FBI - Federal Bureau of Investigation ~ Fantastic Bureau of Intimidation

The system keeps all the main heads and works on the adjec­ tives and the PP head, preserving the rhyme and/or using the a-semantic dictionary.

CRT - Cathodic Ray Tube

• ~Catholic Ray Tube

ESA- European Space Agency

• ~Epicurean Space Agency

PDA- Personal Digital Assistant

• ~Penitential Dernoniacal Assistant
• ~Prenuptial Devotional Assistant

MIT - Massachusetts Institute of Technology

• ~Mythical Institute of Theology

Some re-analyses are RELIGION oriented. Note the rhymes.

As far as generation from scratch is concerned, a main con­ cept and some attributes (in terms of synsets) are given as in­ put to the system. Here below we report some examples of acronym generation.

Main concept: processor (in the sense of CPU); Attribute: fast

OPEN -On-line Processor for Effervescent Net

• PIQUE - Processor for Immobile Quick Uncertain Experi­ mentation
• TORPID - Traitorously Outstandingly Rusty Processor for Inadvertent Data _processing
• UTMOST -Unsettled Transcendental Mainframe for Off-line Secured Tcp/ip

We note that the system tries to keep all the expansions of the acronym coherent in the same semantic field of the main concept (COMPUTER_SCIENCE). At the same time, whenever possible, it exploits some incongruity in the lexi­ cal choices.

3.4 Evaluation

Testing the humorous quality of texts is not an easy task. There have been relevant studies though, such as [Ruch, 1996]. . For HAHACRONYM, a simpler case, an evaluation was conducted under the supervision of Salvatore Attardo at Youngstown University, Ohio. Both reanalysis and genera­ tion have been tested according to criteria of success stated in advance and in agreement with the European Commission, at the beginning of the project.

The participants in the evaluation were forty students. They were all native speakers of English. The students were not told that the acronyms had been computer generated. No record was kept of which student had given which set of answers (the answers were strictly anonymous). No demo­ graphic data were collected. However, generally speaking, the group was homogeneous for age (traditional students, be­ tween the ages of 19 and 24) and mixed for gender and race.

The students were divided in two groups. The first group of twenty was presented the reanalysis and generation data. We tested about 80 reanalyzed and 80 generated acronyms (over twice as many as required by the agreement with the Euro­ pean Commission). Both the reanalysis module and the gen­ eration module were found to be successful according to the criteria spelled out in the assessment protocol. The acronyms reanalysis module showed roughly 70% of acronyms hav­ ing a score of 55 or higher (out of a possible 100 points), while the acronym generation module showed roughly 53% of acronyms having a score of 55 or higher. The thresholds for success established in the protocol were 60% and 45%, respective! y.

A set of randomly generated acronyms were presented to a different group of twenty students. A special run of the sys­ tem was performed in which the semantic filters and heuris­ tics had been disabled, while only the syntactical constraints were operational. (If the syntactical rules had been disabled as well the output would have been gibberish and it would be difficult to compare with the regular system production). In this test, less than 8% of the acronyms passed the 55 points score test, we conclude that the output of HAHACRONYM is significantly better than random production of reanalysis.

A curiosity that may be worth mentioning: HAHACRONYM participated to a contest about (human) produc­ tion of best acronyms, organized in December 2002 by RAJ, the Italian National Broadcasting Service. The system won a jury's special prize.

4 Prospects for advertisement

Humor is the healthy way of creating 'distance' between one's self and the problem, a way of standing back and look­ ing at the problem with perspective. Humor reveals new as­ pects, disarms and relaxes. It is also infectious and it is an important way to communicate ideas. On the cognitive side humor has two very important properties:

• it helps getting and keeping people's attention. Type and rhythm of humor may vary and the time involved in building the effect may be different in different cases: some times there is a context - like joke telling - that from the beginning let you expect for the humorous cli­ max, which may occur after a long while; other times the effect is obtained in almost no time, with one perceptive act - for instance in static visual humor, funny posters or in cases when some well established convention is re­ versed with an utterance;
• it helps remembering. For instance it is a common ex­ perience to connect in our memory some knowledge we have acquired to a humorous remark or event. In a for­ eign language acquisition it may happen that an involun­ tary funny situation is created because of so called "false friends" - words that sound similar in two languages and may have the same origin but have a very different mean­ ing. The humorous experience is conducive to remem­ bering the correct use of the word.

No wonder that humor has become one of the favorite strategies used by creative people involved in the advertising business. In fact among the various fields in which creative language is used, advertising is probably the area of activity in which creativity (and hence humor) is practiced with most precise objectives.

From an applied AI point of view, we believe that an envi­ ronment for proposing solutions to advertising professionals can be a realistic practical development of computational hu­ mor and one of the first attempts in dealing creative language.

Some examples of the huge array of opportunities that lan­ guage offers and that existent NLP techniques can cope with: rhymes, wordplays, popular sayings or proverbs, quotations, alliterations, triplets, chiasmus, neologism, non sequitur, adaptation of existing expressions, adaptation of proverbs, personification, synaesthesia (two or more senses combined).

An intelligent system devoted to advertisement, based on an apparatus that extends the one we have described, can play with these expressions, alter them, change the context, use them as a platform for completely new ideas (e.g. create the slogan: Thirst come, thirst served for a soft drink).

In the longer run the aim is to go for individual-oriented advertisement. Let us leave aside issues of privacy for the moment, though we know this is a critical theme. The avail­ ability of individual profiles, automatic inferencing of interest and behavior models, information about the location of the individual, combined with reasoning on the offer side (what is advisable to promote in a given context of business) will all provide the basic input for advertising things worth get­ ting the interest of a specific individual in a given context. Commercial advertisement will be just one case along this line- one can think of influencing the individual behaviour on the basis of the availability of any possible resources such as, for instance, cultural goods. A prediction is that when such situation-oriented advertisements will become widespread, the same features that prevail now with broadcasted material will be sought after for the new case. So, forms of humorous advertisement for the individual (even playing on personal as­ pects) will become a plus.

An important aspect to be taken into account is how humor is appreciated by different individuals. Personality studies re­ garding this specific theme give important indications [Ruch, 2002]. One option will also be to develop humor for conver­ sational systems, based on embodied agents. The work of Ni­ jholt [2002] Andre and Rist [2000] and Cassell [2001] could provide the starting point for introducing dynamic humor.

For obtaining good results (there are not many things that are worse than bad humor!) for these longer term objectives, much work will be needed, especially further integration with ontologies, common sense reasoning and pragmatics.