Using Focus Rules in Requirements Elicitation Dialogues

1 Introduction

Requirements elicitation is a difficult part of software engin­ eering in which the specifications for a new system are dis­ cussed with potential users. Because verifying that require­ ments are correct is a complex task, computer support is be­ neficial. This support requires formal specifications. How­ ever people are usually not trained to use formal specification languages. Task or domain specific languages smooth the learning curve to write formal specifications but the elicita­ tion process often remains error prone. Users need more sup­ port while writing specifications. In particular, a tool which interacts with them and helps them express their requirements in a domain specific way could lower the number of require­ ments elicitation errors. However, the interaction between the requirements engineer or future users and requirements elicitation tools has often been neglected. This is a problem since the easiest way to acquire the information for the re­ quirements elicitation tools is often not the most acceptable in terms .of human dialogue. Adapting the dialogue so that users can make sense of the information provided by the tools is therefore important. On the other hand we do not want to force requirements elicitation tools to be aware of dialogue management strategies. A solution to this problem is to separ­ ate requirements elicitation tools and dialogue managers but make them interact and constrain each other.

*PSI-LIRINSA, I.N.S.A., Place Emile Blonde!, 76130 Mont­ Saint-Aignan, France

t oepartment of Artificial Intelligence, 80 South Bridge, Edin­ burgh EHI lHN, Scotland

We have developed a system which deals with this issue. This system communicates with its users to acquire require­ ments interactively. Its main feature is to organise the dia­ logue it has with users in order to ensure a coherent inter­ action. It is composed of two parts: a requirements elicita­ tion tool and a dialogue manager as shown in figure 1 [Ward, 1986, details the graphical notation used].

The Requirements Elicitation Tool checks that the require­ ments entered by users are correct. It also provides guid­ ance by making deductions on the requirements. It uses a Domain Specific Specification Model which contains the do­ main knowledge of the system about what can be elicited. It also accesses a Specification which represents the require­ ments that have been elicited so far. The specification is the instanciation of the domain specific specification model for the specific situation the users have described.

In our system, the requirements elicitation tool and the dia­ logue manager interact by means of "communications". The Communication Pool contains the communications to be out­ put by the the dialogue manager and the answers provided by the users. The requirements elicitation tool puts communica­ tions in the communication pool depending on the deductions it makes and the dialogue manager puts users' answers to the communications as they become available.

The Dialogue Manager outputs the communications cre­ ated by the requirements elicitation tool in natural language. It also enables users to write English sentences that are then transformed into a notation appropriate for the tool. It organ­ ises how the communications are output by means of Focus Theories. Focus can be defined informally as "the set of all the things to which participants in a dialogue are attending at a certain point in the dialogue and the point of view they have on these things" [Lecreuche et al., 1998]. Focus theories ensure that the guidance provided by the tool is appropriate to the users' current task. The dialogue manager has access to different focus theories that can be "plugged-in" the sys­ tem. Therefore we can adapt the level of text structure by us­ ing different focus theories. Focus theories seem well-suited for organising the kind of dialogues we consider here. Other techniques, such as those based on the intentions underlying the dialogue, would require the dialogue manager to know what the elicitation system is trying to achieve and what its plan is. For some elicitation systems, this knowledge may not be available. Focus theories require less information and allow a stronger separation between the elicitation system and the dialogue manager.

We have tested our system with two theories: a global focus theory organising the dialogue at a high level and a local focus theory (not presented here but based on center­ ing [Grosz and Sidner, 1986]) organising the sentence-by­ sentence flow of the dialogue. The dialogue organisation de­ pends also on the communications available in the commu­ nication pool and therefore on the requirements elicitation tool. Conversely, the reasoning of this module is depend­ ent on which communications the dialogue manager outputs since this directly influences users' answers. The focus the­ ories have access, through the dialogue manager, to the spe­ cification and the specification model in order to re-use the knowledge represented in these data stores. The access to these datastores is mediated by Translation Rules.

In this paper, we present in section 2 one of the focus the­ ories we use in our system. We advocate the use of a formal focus theory independent of any domain. Because the the­ ory is formal, we can prove properties about the dialogues it allows. In section 3, we describe some translation rules we use to transform the domain knowledge used by our require­ ments elicitation tool into a form suitable to our formal focus theory. We show in section 4 an example of a requirements elicitation dialogue when using our dialogue manager. In that example, our system enables users to specify how to present a research group WWW site. We then summarise an evaluation of the effectiveness of this approach for requirements elicita­ tion dialogues in section 5. The fact that focus rules improve the quality of texts is usually taken for granted and very few approaches provide a precise evaluation of their role. Finally we conclude the paper in section 6.

2 Focus Rules

In this section we present the formal focus theory that our sys­ tem uses. The theory ensures that dialogues between the tool and its users are globally coherent. It is based on a simplific­ ation of Reichman's global focus theory [Reichman, 1985]. The basic idea is that the dialogue is supported by a sequence of changes to a focus spaces set. The focus space set contains focus spaces which represent the information in focus during part of the dialogue. If S = { Ji} is the initial focus space set, only containing the initial focus space Ji at the beginning of the dialogue, then the set when some communications have been output is S = { Ji, .. . , :!,1} where each :Ji contains some of the things spoken about in the dialogue. We may need to create several focus spaces since the dialogue may deal with more than one topic. Focus spaces have different activation levels. We use three activation levels in our formalisation:

Active This is the space to which current communications are added. This space is unique at any given point in the dialogue.

Controlling These are the spaces expected to become active again when the space they control is closed. The con­ trolling spaces form a tree: each space is at most con­ trolled by one other space and a space cannot control a space already controlling it or. controlling any of its con­ trolling spaces.

Closed These are the spaces which have been dealt with and are not expected to be returned to.

Communications cause things to be included in the active focus space. They can also cause focus space activations to be modified and focus spaces to be created. In our theory, communications have one main subject which is a thing to be discussed, and possibly some other subjects which provide some context for the communication.

Our theory is composed of seven rules. The focus rules define the possible ways the dialogue may develop. Associ­ ated with an ordering, they represent what we expect to say next in the dialogue. The rules are based on four relations between the things that can be mentioned in the dialogue:

Direct relation There is a direct relation from one thing to another if the other is closely related to it and can be mentioned in the same focus space.

Specialisation relation There is a specialisation relation from one thing to another if the other is more specific than it. In that case the more specific thing can be dis­ cussed in the perspective of the more generic one.

Generalisation relation There is a generalisation relation from one thing to another if there is a specialisation re­ lation in the other direction between the two things and no direct or specialisation relation already links the first thing to the other.

Simple relation There is a simple relation from one thing to another if it is related to the other and no direct, special­ isation or generalisation relation already links it to the other.

The exact nature·of these relations depends on the application domain. The translation from the domain relations to these relations is discussed in section 3.

Because focus rules only indicate what changes in the fo­ cus spaces (most things stay the same), we are faced with a simple version of the frame problem, i.e., how to indicate what changes and what does not. We tackle this problem by indicating the time during which a relation holds rather than asserting and deleting the relation itself [Kowalski and Ser­ got, 1986]. In particular, we represent the time in the dialogue as the sequence of communications and their associated focus moves performed since the beginning of the dialogue. For example, at the start of the dialogue, the time is the empty se­ quence (), and after communications c,, Cz and C3 have been output with their associated moves m,, mz and m3 respect­ ively, the time is the sequence (m, ( C,),mz( Cz),m3( C3)).

We define the ~ relation on time in the following way: t1 ~ tz <=> :lt3.(t1 = tz · t3) where· is the sequence concatena­ tion operator. The ~ relation is then defined as t1 ~ tz <=> t1 ~ tz 1\ ·(t, = tz). Relations that begin to hold at a certain time are noted start(P,t) where Pis the relation and t the time. Re­ lations that stop holding at a certain time are noted end(P,t). In order to know if a relation holds at a certain time, we define the predicate hold(P,t). The.definition of this predicate is

·

Because the notation used above is quite cumbersome, we will write hold(P(A,, ... ,An),t) as P1(A,, ... ,An). The same applies for infix operators. For example, inclusion in a set at time t is noted E1· We will ignore the time if it does not influence the truth value of the predicate.

The notation used to represent the rules is expressed in our simplified notation in table 1.

In the following rules, the time variable, t, and the com­ munication variable, C, are universally quantified.· All other variables are assumed to be existentially quantified unless ex­ plicitly universally quantified. When a new space is created, it is assigned an unused number. This number is computed by the function new(t) where t is the dialogue time from which the number of existing focus spaces can be deduced.

Rule 2.1 (No change) The focus space does not change and new information is added to it. This rule is used to speak

)

about things closely related to the things in the current active focus space.

where t' stands fort· (no change( C)).

Rule 2.2 (Resetting) A new focus space is created. This rule is used to speak about more abstract things than the ones in the current active focus space. It may therefore serve to give background information on the things in the current active focus space.

where t' stands fort · (resetting( C)).

Rule 2.3 (Additive) A new focus space is created. It is con­ trolled by the current active space. Entities in the current active space are copied to the new space. This rule is used to speak about things that are more precise than the ones in the current active focus space. The new things are discussed from the perspective of the current active focus space.

where t' stands fort · (additive( C)).

Rule 2.4 (Generating) A new focus space is created. Entit­ ies in the current active space are copied to the new space. Any controlling relation is passed from the current active space to the newly created space since we may not expect to come back to the current active space but we still expect to come back to its potential controlling space. This rule is used to speak about things related to the things in the current active focus space but not closely associated with them.

where t' stands fort· (generating( C)).

Rule 2.5 (Pop) A controlling space becomes active again. This rule is used to come back to a space that was expected to be reactivated. This rule does not output any communica­ tions. It is always used in conjunction with one of the other rules which outputs a communication.

where t' stands fort· (pop( C)).

Rule 2.6 (Digressing) A new focus space is created. It is controlled by the current active space. This rule is used to change the focus of the dialogue for a period of time after which the dialogue will resume where it was interrupted.

where t' stands fort· (digressing( C)).

Rule 2. 7 (Reopening) An old space becomes active again. This rule is used to come back to a topic that was considered dealt with. There are two ways a topic may be re-introduced in a dialogue: (I) we realise we forgot to say something about it and come back to it (2) the topic is discussed from another perspective. The reopening move only addresses the first type of re-introduction. For the second type a new focus space would be created since the dialogue context (i.e., the con­ trolling space) has changed.

where t' stands fort· (reopening( C)).

The dialogue starting state for this theory is active() ( 1i) with 1i =o {}.

Now that the rules have been formalised, it is possible to prove properties of them. For example, we can verify that in any dialogue, there is exactly one active space. This can be shown by induction on the dialogue time. At timet= (), the property is verified since there is only one focus space and this space is active. If we then suppose that for any time t -< T, the property holds, it also holds at time t = T because (1) every move starting a new active relation ends the cur­ rent active relation, (2) no move ends the current active re­ lation without starting a new one. Checking these properties is difficult in most other focus theories because they are not formalised.

2.1 Rule ordering

Several rules may have their preconditions satisfied at the same time. We prefer then to apply the rule that maintains the focus if possible, or minimises its movement. We min­ imise the focus movements by presenting general concepts before specialised ones and by avoiding references to unre­ lated concepts. Rules are therefore applied in the following preference order:

no change > resetting > additive > generating > pop > digressing > reopening.

This means that we first try to find a communication to be output which would allow a no change move. If such a com­ munication does not exist, we try to find a resetting commu­ nication and so on until a communication is found.

There is an exception to this ordering. If the main subject of a communication to output is already a member of a space controlling the current active space then a pop is the preferred move. This avoids reintroducing concepts that we are expect­ ing to return to later in the dialogue.

3 From Domain Knowledge to Focus Rule Representation

Because the focus rules we presented in section 2 are based on generic concepts such as direct, specialisation, generalisa­ tion or simple relations which are not necessarily used in the application domain of the system, we need to bridge the gap between the representation used in this domain and the rep­ resentation used by the focus rules. This is done by "transla­ tion rules". These rules allow the use of this focus theory in different domains and for different applications.

We have created a set of translation rules to map the spe­ cification model used by our requirements elicitation tool to the generic concepts used by our formalised focus theory. In our system, the specification model is represented by an Entity-Relationship (ER) model [Chen, 1976]. A part of the model used to check the specification of a WWW site de­ scribing a research group is presented in figure 2 [Wieringa, 1996, details the graphical notation used] . A research group involves a set of researchers and a set of publications. These three entities can be presented by a single WWW page or by a WWW site, i.e., a group of pages. Sites can be described by a home page. Pages and home pages can be linked together.

The main idea behind the translation rules is to allow things that are related to the current focus and cannot be accessed easily otherwise to be mentioned immediately. We only present here two examples of such rules (These rules can be formalised [Lecreuche, 1999]. We do not show the formal­ isation here as it would require us to also explain the formal framework used to represent the specification model):

• An entity is in direct relation with another if it may not exist at all and if it cannot exist without the other one. In our example, a home page is an optional entity whose existence requires the existence of a site. There is there­ fore a direct relation from the site to the home page.
• If an entity can be linked to several others playing the same role, then there is a relation of specialisation from this entity to the others. In our example, the research group involves the researcher set and the publication set which fill the same role involving. These two ele­ ments are therefore in a specialisation relation with the research group.

Using the translation rules, the ER relations presented in fig­ ure 2 are associated with the following focus relations:

• Direct relations: present from publication set, researcher set and research group to site and page, describe from site to home page.
• Specialisation relations: involve from research group to publication set and researcher set, 1 ink from home page to page and conversely.
• Generalisation relations: involve from publication set and researcher set to research group, present from site and page to publication set, researcher set and research group.

Simple relations: All other relations.

Basing the global focus relations on the specification model is a great advantage since it enables us to re-use the domain knowledge of the tool checking the specification. Moreover the translation rules we presented here are depend­ ent on the ER model structure but not on the application domain itself. This makes them more generic. As a con­ sequence we have been able to use them in other domains than the WWW site design one. In particular we have applied our system for a simple library loan process description tool and for a more generic ER model acquisition tool. The trans­ lation rules could however be made more domain dependent if need be.

4 Example

In this section we present an example of a requirements eli­ citation dialogue. Our system enables users to specify how

• S: Research group #rg1 is the research group. Do you want a site presenting it?
• U: Yes
• S: A site presents the research group. Which site presents the research group?
• U: A new site.
• S: Site #s1 presents the research group. Do you want a home page describing site #s1?
• U: Site #s1 is described by a new home page.
• S: Home page #hp1 describes site #sl. What is the title of home page #hp1?
• U: "Research group".
• S: Do you want a page linked from it?
• U: I want to speak about the researcher set
• S: OK. Researcher set #rs1 is the researcher set. It is involved by the research group. Do you want a page presenting it?

Table 2: Dialogue example

to present a research group by a WWW site. Once this has been done, the specification can be transformed into HTML and the site can be displayed by a WWW browser. Our sys­ tem allows users' intervention in the elicitation process but we show here an example where it has the initiative and is in charge of acquiring the requirements.

The system uses templates to generate sentences and in­ terpret users' answers. Depending on the focus information available, the templates can be adapted to produce pronouns, or definite noun phrases. Entities introduced in the dialogue can be referred to using a unique identifier provided in the dialogue by the system.

Using the results of the translation rules, the system tries to keep elements in direct relation together as explained in sec­ tion 2.1 . This is shown in the example presented in table 2. The system first deals with everything directly related to the research group, i.e., site and home page, in the same focus space, :fi. Then it selects a specialisation relation from the home page to a page. This move opens a new focus space, 12. under the control of the previous one. However, the user redirects the dialogue by choosing another specialisation rela­ tion from research group to researcher set. This move closes the controlled focus space .and opens another one, :!3, under the control of the first focus space. The final state of the dia­ logue is presented graphically in figure 3. This dialogue is more coherent than a dialogue where the system is free to choose the topic and could possibly shift randomly between

Local and global theories order Global theory order Elicitation module order Local theory order

39

34

17

10

----roo-

Table 3: Evaluation

research group, home page, site and researcher set.

5 Evaluation

A detailed evaluation of our focus theories has been carried out [Lecreuche, 1999]. In this section we summarise the res­ ults of initial tests with users.

We tested our system by asking people to read tran­ scripts of requirements elicitation dialogues generated with and without focus theories and asking them to compare the dialogues pair-wise for coherence [Saaty, 1990]. Transform­ ations can then be made on these comparisons to compute the overall ranking of the theories. For example, a theory with a value of 60 is considered three times more coherent than a theory with a value of 20. The values of all the theories add up to 100.

The experiment involved six persons for 90 minutes. Six dialogues were evaluated. Each participant compared four dialogues, resulting in 36 pair-wise comparisons. The res­ ults are presented in table 3. (The consistency ratio is equal to 0.0993 which indicates reliable results. The participants may however not be representative of the whole user com­ munity.) They show that our global focus theory was judged to improve the global coherence of the dialogue compared to following the requirements elicitation tool order, i.e., without using focus rules. The results also show that the local the­ ory we use in our system performs badly on its own. This may be explained by the tendency of local theories to produce "spaghetti" dialogues without global structure. The local and global theories together perform somewhat better than the global theory alone. This can be explained by the local theory improving the transitions between communications in each global focus spaces and the transition between global focus spaces themselves.

This evaluation shows that our approach provides improve­ ments in perceived dialogue quality. However using any indi­ vidual focus theory may not be sufficient to achieve this result as demonstrated by the poor evaluation of the local focus the­ ory on its own.

6 Conclusion

We have presented an interactive system for requirements eli­ citation dialogue. Our system uses focus rules to improve the quality of its interaction with users. Focus rules have rarely been used in interactive systems, especially to direct the sys­ tem's reasoning. Other systems used for requirements elicit­ ation [Rolland and Ben Achour, 1998; Reubenstein, 1990] do not take them into account and leave users in charge of picking up the relevant information from the system output. We have discussed a formal focus theory. Because of its formality, properties of the theory can be proven mathem­ atically. We then presented "translation" rules, bridging the gap between the representation used by the focus theory and the representation used for the domain knowledge. We then presented an example of dialogue showing the influence of the focus theory. Finally, we showed that our approach was judged to improve the perceived quality of requirements eli­ citation dialogues. Although we presented our results in the domain of requirements engineering, we believe that our the­ ory could be used in other domains where the quality of in­ teraction with users is important. We now plan to investigate how the theory presented in this paper could be integrated with other techniques (such as intentional ones in case the elicitation system is able to indicate its plan) and to use it in multi-agent systems where one of the agent would be in charge of the natural language interation with users.