A Theory of Language Acquisition Based on General Learning Principles

A THEORY OF LANGUAGE ACQUISITION BASED ON GENERAL LEARNING PRINCIPLES

John R. Anderson Department of Psychology Carnegie-Mellon University Pittsburgh. PA 15213

thought by the learner to be part of the sentence. So, we have worked out mechanisms for achieving sentence-meaning pauings in s1mple ostensive learninf situations. However, much of what a child must learn about language will lack simple ostens1ve referents. For instance, most of the verb auxiliary system refers to non-ostensive meamng. How a child (or any system> would come up with sentence-meaning pamngs m these situations is not clear and remains an issue for future research.

A major assumption of the LAS model that is maintained in the current system is that the system already knows the meaning of a base set of words. LAS was unable to learn the meaning of any words in context while the current system can; however the basic learning algorithm in both still requires that a substantial number of words in the sentence have their meanings previously learned. In principle <see Anderson, 19741, it would be possible to call to bear statistical learning programs to extract the meaning of the base set of words from a sufficiently large sample of meaning-sentence painngs. However, the evidence <McWhinney, 1980> is that children accomplish their inillal lexicalizallon by having individual words paired directly with their referents.

Identifying Phrase Structure: The Graph Deformation Condition

A major problem in language learning is to identify the phrase structure of the sentence. There are a number of reasons why inducing the syntax of language becomes easier once the phrase structure bas been identified: ( I l Much of syntax is concerned with placing phrase units witlun other phrase units. ( 2 l Much of the creative capacity for generatmg natural-language sentences depends on recursion through phrase structure units. ( 3) Syntactic contingencies that have to be inferred are often localized to phrase units, bounding the size of the induction problem by the size of the phrase unit. ( 4 l Natural language transformations are best cbaractenzed wilh respect to phrase units as the transformational school bas argued. ( 5) Finally, many of the syntactic contingencies are defined by phrase unit arrangements. So. for instance, the verb is inflected to reflect the number of the surface structure subject.

A major mechanism for identif}'lng phrase structure in LAS (and which is conllnued JD ALAS> is use of the graph­ deformation condition. The idea is to use the structure of a sentence's semantic referent to place constraints on surface structure. The application of the graph deformallon condition 1s illustrated in Figure I. In part (a) we have a semantic network representation for a series of propositions and JD part (b) we have a sentence that communicates this informa!Jon. The network structure in <al bas been deformed JD <bl so that it sits above the sentence but all the node-to-node linkages have been preserved. As can be seen, this captures part of the sentence's surface structure. At the top level we have the subject clause <node X in the graph), gave, book, and the

Abstract

A simulation model IS described for the acquisihon of the control of syntax in language generation. Tlus model makes use of general learning principles and general principles of cognihon. Language generahon is modelled as a problem solving process involving principly the decomposihon of a to­ be-commumcated semantic structure into a hierarchy of subunits for generation. The syntax of the language controls !Ius decomposition. It is shown bow a sentence and semantic structure can he compared to infer the decomposition that led to the sentence. The learning processes involve generalizing rules to classes . of words, learning by discnmination the various contextual constraints on a rule application, and a strength process which monitors a rule's history of success and failure. This system is shown to apply to the learning of noun declensions in Latin, relative clause constructions in French. and verb auxiliary structures in English.

INTRODUCTION

This research bas its background in past work on language acquisition (for reviews, see Anderson, 1976; Pinker, 1979--see also Langley, 1981 ), especially in my previous work on LAS (Language Acquisition System--see Anderson, I 977). For various reasons that will be explained, there were problems with LAS and a more general concept of human cognition was developed called ACT (Anderson, 19761. The system to be reported here is an attempt to merge the ideas in the ACT project and the LAS proJect. It is called ALAS for ACT's Language Acquisition System. First in this paper I will review those aspects of the LAS and ACT systems that are relevant to understanding the current project and then I will turn to describing the ALAS system.

THE LAS SYSTEM

LAS accepted as input strings of words, which it treated as sentences, and scene descriptions encoded as associative networks. When learning, the program attempted to construct and modify augmented transition networks which described the mapping between sentence and scene descnptions. This assumption, that the program bas access to sentence-meaning pairings, is the bas1c assumption underlying most of the recent attempts at language acquisition. This assumption might be satisfied in the circumstance where the child is hearing a sentence describing a Situation he is attendmg to. Even here it is likely that the child will represent aspects of the situation not described and fail to represent aspects described. In LAS we worked out mecbamsms for filtenng out the non-described aspects of the meanmg representahon by companson with the sentence. In the current ALAS system there IS a d1scnm1nation mecbamsm for bnnging in aspects of the situation not m1hally recipient (node Yl identified as a unit. The · two noun phrases are segmented mto phrases according to the graph structure. For instance, the graph structure identifies that the words I i ves and house belong together in a phrase and that big, girl, I ives, and house belong together in a higher phrase.

·Thi!> project i,. 'uproncd by contract NOOOI4·~<-()0()I lrom &he OUicc of Nav111l Rc~arch . I would ·like Lfl &han~ SLeven Pinker , Lynn..: Reder an..J Mtrutm Schustacl.. lor

their c:ommcnuo on earlier drah" or tht~ mt~nu.<~Cnrt

The graph deformation in part (b) identifies the location of the terms for which meanings are possessed in the surface structure of the sentence. However, terms like the before big girl remain ambiguous in their placement. It could either be part of the noun phrase or directly part of the main clause. Thus, there remains some ambiguity about surface structure that will have to be resolved on another basis. In LAS the remaining morphemes were inserted by a set of ad hoc heunslics that worked in some cases and not in others. One of the goals m ALAS was to come up with a better set of principles for determining the boundaries of phrases.

The graph deformation condition is violated by certam sentences which have undergone structure-modifying transformations that create discontinuous elements. Examples in English are:

l. The news surprised Fred that Mary was pregnant.

2. John and Bill borrowed and returned, respectively,

the Jawnmower. Transformations which create discontinuous elements are more common in languages that use word order less than English. However. the graph deformation condition remains as a correct characterization of the major tendency in all languages. The general phenomena has been frequently commented upon and has been called Bebaghel's First Law (see Clark & Clark, I 977). A problem with LAS was that it bad no means of dealing with exceptions to the graph deformation conditions or of learning transformations in general. Another goal for the ALAS current enterprise is to be able to detect sentences that violate the graph deformation condition and to use these as opportunities for learning transformations.

A major source of my dissatisfaction with LAS is that its processing discipline and learning mechanisms are specific to language and it was bard to imagine how they 'OUid relate to other types of skill learning. While many people believe the principles underlying language acquisition are unique, I do not. I tlunk the other problems with the LAS enterprise could be repaired but I felt a fresh start was needed if we were to show that general skill acquisition principles could plausibly apply to natural language as a special case. This led to the development of the ACT theory (Anderson, I 976; Anderson. Kline, & Lewis, 1977) and to a set of learning principles for that theory.

As originally formulated, ACT was a productiOn system Without any commitment to the mechamsms of skill orgamzat 1011 or skill acquiSitiOn. However. a set of prmciples have emerged ID our more recent work (Anderson. Khne. & Beasley. 1980; Anderson & Khne. I 979; Anderson, Greeno, Khne. & Neves. I 98 I) and it IS these developments winch are essential for the current . application. These ideas have been developed m non­ hnguJshc domams--schema abstraction, acquisition of proof sk1lls m geometry, and most recently m the acquisition of programming skills.

We see any skill as being hierarchically orgamzed mto a search of a prohlem spare 111 winch ther<' IS a ma1n goa.l . wh1ch is decomposed mto subgoals, and so on until the decompos1hon reaches achievable subgoals. Much of what IS distmcllvc about a particular skill IS the way m wh1ch the problem space IS searched for a solution. In our model of language generation. this is seen as a s1mple top-down generation of subgoals < correspondmg to phrases) where there is no real search needed unless transformations have to be applied. We will illustrate this application to language shortly.

In simulating language acquisition we have focused on Ilk: learning mechanisms concerned with operator selection: generalization, discnmination, and strengthening. Generalizallon takes rules developed from special ·cases and tries to formulate more general vanants. Discnmination is responsible for acquuing vanous contextual constramts to delimit the range of overly general rules. Strength reflects the success of a rule m the past and controls its probability of future application. In combination. these mechanisms function like a statistical learnmg procedure to determine which problem features are predictive of a rule's success. They have been extensively documented in our efforts to model the literature on schema abstraction (Anderson & Kline. 1979; Elio & Anderson. in revision), but they have had a richer application to acquisition of proof skills (Anderson, submitted; Anderson, Greeno. Kline, & Neves. I 98 I) . I will sketch theu apphcat1011 to the language acquisition domam, but the reader should go to these other sources (and parbcularly Anderson. Kline. & Beasley, 1980) for a fuller development.

CURRENT FRAMEWORK FOR LANGUAGE LEARNING

The language learner IS cbaractenzed as having the goal of communicatmg a particular set of propositions. Th1s set of propositions IS organized into a main proposition and subproposillons. So, for instance, the goal belnnd the generation of The girl kicks the boys might be a communication structure which we can represent as <KICK <GIRL xl <BOY y)) where x is tagged as singular and y is tagged as plural. To achieve the goal, the learner tries to decompose this h1gher level goal into subgoals. accordmg to the umts of the overall communication structure. So, he will decompose this into the subgoals of communicating kick. of commu01catmg <GIRL xl. and of communicating <BOY y). He looks to his language for some means of organizing these subgoals. So. he might have learned a rule of the form:

IF the goal is to communicate (LVrelalion LVobjectl LVobject2)

and L Vrelation is in the VERBX class THEN set as the subgoals to

communicate L Vob jectl

say the morpheme for LVrelallon

say "s"

and to communicate LVobJect2

or we might more compactly denote this rule:

(1 2 3) --> 2 + 1' + s + 3 if 1 in VERBX

In the above, the I, 2, and 3 match the three elements in the meaning structure--KICK, (GIRL x), and <BOY y>. The ngbt side of the arrow specifies their order in the sentence and the insertion of mophemes like S. The star above the I indicates its lexical form is to be retrieved. The other elements will have to be further unpacked.

If it is early in the language learning history and the learner does not · have a rule for realizing this construction. then he might try to invent some principle. He may only produce a fragment (e.g., girl bit) or a non-allowed order <e.g., girl boy bit). There is soine evidence in first language acqui.sition that children will use word orders not frequent in adult speech (Clark, 1975; de Villiers &. de Villiers, 1978; McWhinney. I 9 80). For instance, there is a tendency to prefer agents first even ·when one's language does not. Also. it is well known that second language learners fall back on their first language word orders when knowledge of word order fails.

The embedded subgoals are unpacked into actions or further subgoals in the same way that the lop level structure is unpacked. For instance, if the object to be communicated were (girl x (like x (sailor z)) ), the lop level of this structure might be commumcated by the rule:

where (hke x (sailor z)) is item 3 in the above and would be communicated by the rule:

(1 2 3) --> wno + 1· + 3

if 1 is a verb and the construc­

tion is embedded

F1gure 2 illustrates the hierarchy of subgoals 1n the generatiOn of a relatively complex sentence: The young pol iceman sees the lawyer whom the crook paid. It should be clear that if sentences are generated by sett10g subgoals to reflect the structure of the referent, then the graph deformation condition will tend to be satisfied in natural language.

f'iQU't 2

The learning that occurs in ALAS is basically learning by doing. The learner generates an utterance and it is assumed that he bas access to feedback about the correctness of the construction he generated and perhaps Information about what the correct utterance should have been if he bas made an error. There are many ways this can happen. The learner may generate a sentence and be corrected by a teacher. He may generate a sentence and remember a sentence or sentence fragment beard earlier. He may bear a sentence, infer its meamng. and compute bow be would express the meaning. By whatever means the learner sometimes 1dentif1es some fragment of Ius generation to be in error and sometimes has a hypothesis as to the correct utterance. Tlus is the stimulus for learnlDg. In the actual simulations that will be reported, the program is given a model sentence along with each meanlDg and the program compares its generation with the model sentence. No doubt this is an unrealistically ideal assumption and results in a considerable speed up of the learning process in ALAS. However, the same learning mechanisms would apply in more psycbolo~ically realistic situations where the program was given only occasional information and often fragmentary information about what the correct target sentences were.

Formation of Initial Rules

The initial rules that ~ system acquires are. of course, quite specific. So, for instance, consider the rules it might form upon receiving a pairing of the Latin sentence ((Equ i)(agricol as)port ant) and the meaning representation (carry (horse x)(farmer y)) . With a partially complete lexicalization, ALAS knows the meaning of equ is horse, the meaning of agricol is farmer, and the meaning of port is carry>. ALAS then formulates the following rules:

(1

(1

(1

2

2)

2)

3) -->

-->

-->

2 + 3 + 1· + ant

1· + 1

1· + as if 1 = carry if 1 = norse if 1 = farmer

Thus, its acquired rules are exact encodings of the relations at each level in the meaning hierarchy. The evidence is that children als(l start out with rules specific to individual words <MacWhinney. 1980; Maratsos &. Chalkley, 1981) and indeed the nature of natural language makes this a wise policy in that rules are quite specific to vanous lexical items <Bresnan, !981; Maratsos & Chalkley, 1980; Pinker, 1981 ). This also is exactly bow learning proceeds lD other areas to which we have apphed ACT. Initially, the system acquires rules that encode the exact goal structure of specific exampres. Later, generalizations are formed.

While, on one band, these rules are too specific, on the other band. they are too general. The inflections associated with the nouns and verbs are only correct for the specific case and number combinations but these rules do not reflect that constraint. The system will have to acquire discriminating features that will properly constrain the range of application of these rules. Again that corresponds to child language. Children initially use words with a single inflection in all situations and only later acquire the contextual constraints. It also corresponds to our other learning endeavours where contextual constraints on goal decomposition are acquued through discrimination.

Discrimination

·To illustrate the discrimination process consider again the rule for realizing far mer:

Suppose the system encounters a second instance of farmer in the meamng-sentence . painng (call <farmer u) (girl vll «agricol al <puell am> voc ao. It would detect a confhct between its generation of agricol+as and the target agricol+a. In tlus case it would look for differences between the context of its current application and the previous. The relevant differences are:

1. y 1n the prev1ous application is tagged as plural while u in the above structure is singular
2. The object structure was in third position m the embedded clause of the first mean10g structure. but now it is in second position.

However, such as there are any number of other potential differences

1. The previ(lUS verb was port and the current voc
2. The second position of the embedded clause

was plural and the current is singular.

1. The current sentence mvolves a femmine obJect.
2. LAS bas an ordering of d1stance ( t(l-be-explainedl such that 4 and S above would be definitely less preferred but there 1s n(l clear bas1s for choosing I and 2 over 3. A feature to d1scrimmate upon is chosen at random aud a new rule is formed such as:

Note that this is a discrimination for the current context. not the previous. ALAS can also form a rule for the old context (l 2) --> l' + as if 1 = farmer (c) an~ 2 is plural

but only if the old rule (a> exceeds a threshold of strength tCl md1cate that it has applied successfully more often than not and 1s therefore not a pure m1stakc.

The correct rules above need another round of discrimination before they pick up the semantic position feature. Then they will become

• (l 2) -->
• l' + a if l = farmer and 2 is singular· an~ this occurs in secon~ position in the semantic referent (d)
• (l 2) --> l' + as if l = farmer and 2 is plural and this occurs in third position in the semantic referent (e)

The set of possible features for discrimination is defined by a network that includes the semantic referent, the goal structure, and any properties tagged to terms in the semantic referent or the goal structure. The program does a breadth first search out from the current position in this network looking for features that distinguish between current and past applications of the rule. It chooses the features it first finds in that search. This means that the system IS sensitive tCl both syntactic and semantic contmgencies of the context of application.

Generalization

Let us consider what would happen if the currently implemented ACT generalization process were to apply to rule \e) from before and to the following rule that the system might

derive in a similar manner:

(l 2) --> l' +as if 1 =girl and 2 is plural and this occurs in the thir~ position in the semantic referent

ACT would generalize these two rules by s1mply dropping the constraint that I be farmer or girl. This would yield:

(l 2) --> l' + as if 2 is plural and this occurs in thir~ position of the semantic referent

This would lead to an enormous overgeneralization in that the above rule is only valid for first-declension nouns.

Thus, we have had to assume that generalization cannot occur in language by the wholesale replacement of a constant by a variable. Rather what we assume is that generalization occurs by replacing a constant by a word class. So, the proper form

of the above rule becomes

(l 2) --> l' + as if l is in class X and 2 is plural and this occurs in third position in the semantic referent

where class X will contain farmer and girl among others. It 1s unclear at present whether this is a true instance of where language acquisition differs from other cognitive learmng or whether the generalization mechanism should be set up to produce constrained variables in all situations.

A major issue in ALAS concerns when words should be merged into the same class. It is not the case that this occurs whenever there is the potential to merge two rules as above. The existence of overlapping declensions and ovcrlappmg conjugations in many languages would result In disastrous ovcrgcneralizations. Rather we have brought to bear an extension of our schema abstraction ideas (Anderson & Kline. 1979). What ALAS does is look at the pattern of rules that individual words appear m. It will merge two words into a

smgle class when

1. The total strength of the rules for both words exceeds a threshold indicatmg a satlsifactory amount of expenence
2. A fraction (currently 2/3 > of the rules that have been formed for one word las measured by strength> have been formed for the other word.

When such a class is formed, the rules for the md1vidual words can be generalized to that class. Also. any new rules acquued for one word will generalize to the other. Once a class is formed new words can be merged with the class accordmg tCl the same criteria (I> and <2> for mergmg words. Further. twCl classes can be merged together, again according to the same criteria. Thus. it 1s possible to gradually build up large classes like first declension.

The word-specific rules are not lost when the class generalizations appear. rurthermorc. one form of d1scnminal1on is to propose that there IS a rule special to a word. Because of the spec1f1city ordering m produchon selection. these word­ specific rules will be favored when applicable. This means that the system can live with a situation where a particular word (such as dive> can be in a general class but still main tam some exceptional behavior.

Thus, the system begins with a lot of word-specific rules which gradually expand in their scope of application. This 1s basically the development observed m child language.

It should be noted that there ·is another dimension in which the system's behavior starts out very general. The rules f Clr communicating a parhcular construction, such as an ob )ect construction (e.g. noun phrase) or qualifying proposition (e.g:. a relahvc clause), arc assumed to apply in every location. Thus. the system automatically assumes rules are recurs1ve and does not need, as did LAS. to verify such points of recursiOn. Rather. the learning here takes the form of constrammg this assumption where ovcrgencral--as we have discussed. Correspondingly, children seem not reluctant to venture old constructions in new syntactic contexts.

Phrase Structure Segmentation

Up to this point we have assumed that the target sentences were segmented into phrase structure units. The graph deformation condihon can be used to ass1gn the words wbClse meanmg is known to phrase units but this leaves unspecified the other morphemes. To take an example from my work w1th Latm consider the following meaning-sentence pairing:

<praise (friend u (have (man v) u» (field x

(have (farmer y> x)))

1

amic us vir i ager os agncol ae laud at <translated: The man's fnend pra1ses the

2

farmer's fields>.

Clearly, the semantic structure mdicates VIr <man) assocmtes with amic (friend> as a modifier and not with ager (field> since man is contained in the same meanmg unit as friend. However, th · semantic structure provides us no way of dec1dmg whether the non-meaning-bearing morpheme us associates with vir or amic. Similarly. it 1s ambiguous bow to locate the other noun mflections: i. os, and ae. On the other hand. at occurrmg at the end of the sentence defimtely must associate with laud. Thus. by means of the graph deformahon conditiOn and only taking unambiguous cases, we get the foiiClWIDf hierarchical organization for the Latin stung:

(( amic ((vir))) ( ager (( agricol))) laud at>

3

where the mdeterminate morphemes are left out. At one pmnl m 1ts appht;o<ion of the graph deformation condition ALAS calculates JUSI th1s structure. If nothing more can be done. th1s 1s the form of the string provided to the learning system-­ i.e .. with the ambiguous morphemes deleted.

How can this strmg be improved upon to insert the non­ meanmg bearing morphemes? In the literature there are three suggestions. Fust, there may be pauses in the speech signal to mdicate the correct associations. There would be no ambiguity if there were long pauses after us, i, os, and ae in the above message. Normal speech does not always have such pauses m correct places and sometimes has pauses in wrong places. Still, !Ius basis for segmentahon would be correct more often than not and ALAS's error correcting facilities have the potential to recover from the occas10nal m1ssegmenta10n. Also, it IS argued that parent speech to children is much better segmented than adult speech to adults <see de Villiers & de Villlers. 1978). In ALAS pausing is used when g1ven, but the system does not requue pause segmentahon.

A second suggestion is to use past instances of successful segmentation to segment in the current case. Thus. if the system has previously identif1ed agricol +ae as associating together it can assume they associate together now. The past experience could derive from bearing the word m isolation or from other sentences where some other basis could be applied for segmentation. Memory for words spoken m ·solahon is a particularly useful solution to the problem of 1dentifymg wluch morphemes belong together to defme a word. The evidence IS qUite clear that children do bear many words 1n ·solahon (McWhinney. 1980). This is less helpful m identifymg phrase boundanes for structures like noun phrases or relative clauses-­ both because these structures are less likely to be spoken in ·solation and because the same word sequence is rarely repeated. Th1s may explam why missegmentahon of morphemes within words is rare in child speech relative to missegmentahon of words w1th phrases (Siobin, 197 3) . Although we could in pnnciple use this strategy, our simulation that attempted to segment without pause structure was not given words in ·solation.

The tlurd basis for segmentation relies on the use of statistics about morpheme- to-morpheme transitions. For mstance, the segment ae will more frequently follow agricol with which it IS associated than it will precede laud with which it is not. The differences in transitional frequencies would be very sharp m a language like Latin w1th a very free word order but they also eXIst m English. Thus, ALAS can assoc1ate ae with the agricol if it has followed agricol more frequently than it bas preceded I aud. Th1s requires keeping stahstics about word- to­ word transitions. Currently, the system will favor one assoc iahon of a morpheme over another if there IS a difference m frequency of two. This · might seem a rather small thresbhold but I have gotten satisfactory performance out of ALAS, partly because ALAS can recover from occasiOnal m·ssegmentahons. Agam the evidence is that children do occasiOnally m1ssegment ( McWiunney, 1980) and, of course, recover eventually. It strikes some as Implausible to suppose that people could keep the stahstical information requued about word to word trans11lons. However, Hayes and Clark ( 1970) have shown that subjects m listening to nonsense sound streams can use differenllal transition probabilities as a basis for segmentation. Such information has also proven useful m computational models of speech recognihon (Lesser, Hayes-Roth. Birnbaum, & Cronk, 1977 ).

It IS possible and frequently has been the case that none of the ALAS segmentallon mechamsms could apply to ass1gn a morpheme to a level Ill the phrase structure. In such cases the non-ass·gned morpheme was simply om1tted from the phrase structure. Thus. the m1t1al utterances produced by ALAS, like the utterances produced by young children. are telegraphic m character. That is. they are missmg many functors.

Latin: The issue of segmentation

Our first endeavour was to learn a fragment of Latin that mvolved first and second declension nouns, mflected for the nominative, accusative. and genihve cases and for plural and smgulnr. An example of the input to ALAS is Agricol ae puel am legal i laud ant (praise <farmer x) (girl u <have (lieutenant v) u)) where x is plural, u and v are singular That is, the input was a string of Latin morphemes that comprised the target sentence and a hierarchical representation of the meanmg of this sentence. The program was prov1ded w 1th a long sequence of such pairings. Over the sequence all syntactic possibilities were realized. With each painng, ALAS consulted its rules to see if they would map the meanmg structure onto the target string. Its learning principles were evoked to modify the rules if they failed to produce the n ght mappmg. In this simulation <and the others) we provide the strmgs segmented into morphemes. Acquisition of morpheme segmentahon IS thus being ignored. The verbs used were 8 first -conJugatiOn verbs; the nouns were 8 first-declension nouns and 7 second-declension nouns. One of the !lungs our simulation was gomg to get at was the adequacy of our class heunstlcs to separate our first and second declensiOn nouns. We performed two Simulations over tlus target language subset. In the first we provided the system with no information about segmentation and it was forced to use the graph-deformat ion condition and transitional probabilities to segment into surface structure umts. In the second simulation we provided pause mformallon to mdicate with which words the mflections were associated.

To avoid any possible b1asmg in mput order. the sentence­ meanmg paus were generated by a randomJzahon program. The simulatiOn without the pause mformallon requued 525 paumgs before it bas identified all the needed grammatical rules and ran a criterion 25 pairings with no mispredictions of the target strings. With pause information, only I 00 sentences were required to reach the same criterion. Figure 3 illustrates the mean number of errors for the two conditions plotted as a function of the logarithm of number of pairings experiences. An error was defined as a misordering of elements at any phrase level, the insertion of an incorrect morpheme, or the ommlSSIOD of a morpheme.

In the case where the system was not given informallon about pause structure. 11 had to use transihonal frequency to segment. After the fust 25 sentences it was correctly assoc1at1ng about 50% of the noun mflections with the nouns. Most of the remammg 50% were failures to Insert the morphemes but there were occasional m·ssegmentallons. Desp1te the fact that it was correctly segmenting over half of the input to the learmng program after the 25th trial, it was only after 7 5 trials that any learnmg of inflections showed up in its performance (I.e. , it started using these inflections with significantly greater than chance accuracy) . Even after 150 sentences ALAS is failing to segment some nouns in I 0% of the sentences. The difficulty m segmentation IS what IS accounting for the slow learning of the program. The examples that follow present, fust, the Latm morpheme strmg that the program generated to express a meanmg structure <not shown) and, second. the target string that was correct. I have given a non-random selection of these to give the reader a sense of the progress of the system throughout the course of the 525 pairings:

The class formatiOn heunstics worked quite well in these simulations. Both with and without pause information, the two declensions were identified as two word classes and all the verbs were brought together into another word class. Figure 4 illustrates the history of discrimination that led to correct use of mflections for the second declension in the simulation with pause information. Time goes to the right and down in the figure. It turned out that on four occasions the system proposed an unconstrained rure for the us inflection. This IS reflected in the horizontal dimension. Going down we have the history of discrimination for each rule. Arrows lead _from a rule to a discriminated rule. The label on the arrow md1cates the feature added in the discnminallon. Thus, for instance. A35 IS a rule that calls for the us inflection <appropriate for nominative singular). It was used incorrectly in an accusative plural situatiOn and an os rule. A66, was formed With _the dlscnminatmg test that the noun be in accusallve case < l._e .. thud position in the semantic structure) . This rule m1sapplled in an accusative singular situation and so a singular feature was added. Rules in boxes are ones that were so weakened by misapplication that they were removed.

Note that there are four rules with all the necessary features: A246 for accusative singular, Ai95 for genitive plural, A455 for accusative plural, and A456 for nominative plural. On the other hand, A354 for genitive singular only tests that it is in a possessive context and not for number. However, because of the specificity ordering on production selection, the more specific genitive plural rule (A195) will apply whenever applicable leaving A354 only the genitive singular situations in which to apply. Similarly, A438 which bas no discriminating features will only apply when no other rule is applicable--which is to say it will apply only the . nominative singular case for which it gives the appropriate inflection.

We ran another simulation training ALAS on tbc; same subset of French that LAS <Anderson, 1971) bad been trained on. This subset is interesting because it introduces the recursive properties of natural language through relative clause recursion. ALAS successfully learned this subset. The details of this simulation are being omitted because of space constraints. However, it 1s mentioned here to note that ALAS can reproduce the success of the previous program. Information about th1s simulation can be obtained by writing to the author.

Verb Auxiliaries

The third s·mulallon was an attempt to have ALAS learn the verb auxiliary system of English. This IS one of the standard language fragments used to introduce and motivate transformallonal grammar <e.~; . . Cullcover, 1976). Tills IS mteresting because the verb auxiliary system does not mvol ve any violallons of the graph deformation condillon and should be learnable by ALAS without resortin~; to transformations. The modals we used were can, could, should, would, will, and may with corresponding meaning components of present-able. past-able. obligation, intention. future. and possibility. These meaning components were not assigned to the terms but rather bad to be learned from context. The sentences were also marked for tense and. optionally for perfect, progressive, and stative. We used sets of four adjectives, eight nouns. six transitive verbs. and four intransitive verbs. Among the verbs were hit, shoot, and run which all have irregular inflections. Therefore, another problem for the simulation will be to learn the special mflections associated with these terms. We provided these strmgs with the pause structures to permit segmentation. Space limitations prevent a detailed specification of the semantic representation. but the author can be written to for a fuller report.

Figure 5 plots the performance of the system for 700 pairings which is the number of pairings required to reach 25 trials of errorless performance. Examples of sentences it generated are:

Sentence 1: Jump angry debutante

Sentence I 0: A tall lawyer s could jump ed

Sentence 16: Some smart actress have tickle ed the sailor s

Sentence 30: Being smart a angry lawyer

Sentence 51: The sailor s were dance ing

Sentence 75: A smart sailor tickle wg a bad lawyer

Sentence 148: The farmer may have shoot ed some Arab s

Sentence 195: The fat doctor s should dance ed

Sentence 213: A fat lawyer can be tall ed

Sentence 228: Some smart lawyer s should be tickle ing the angry actress s

Sentence 253: A sailor are tickle ed by some good lawyer s

Sentence 354: Some sad ed lawyer s have run Sentence 370: The sad doctor s are kick ed by the angry farmer s

Sentence 426: Some lawyer s were being hit ed.

These sentence5 illustrate one of the unexpected developments in the simulation. ALAS collapsed adjectives, transitive verbs, and untransitive verbs into a single word class over time because all these are involved in numerous similar auxiliary structures. This accounts for the appearance of constructions like "sad ed lawyers" and "can be tall ed" where the "ed" mflection has generalized from the verbs to adjectives. Then ALAS had to go through a number of discriminations in which it used the action-quality property distinction between verbs and adJectives to properly restrict the rules.

An Important feature of the verb auxiliary system is that. 1f we cons1der the verb matrix sequenced tense-modality-perfect­ progressive-verb, tense coJditions an inflection in the term that immediately follows it. perfect an inflection in the term that follows it, and similarly progressive. Tnis is interesting because the modality. perfect, and progressive terms are all optional. Tlus means that the term inflected for tense or perfect will vary. So. for instance, depending on the verb matrix we inflect perfect (has/bad), progressive (is/was), or verb (kicks/kicked> for tense. This is handled 1n standard transformational analysis by a transformation called affix hopping. This IS handled in our simulation by making the prior term part of the rule. So, for instance. ALAS learned the rule:

1 + 2 --> 1 + s + 2

if 1 is progressive and the context is present and the syntactic subject is singular

I I is not a s1mple matter to judge whether the affix hoppmg transformation (together with its many support rules) provides a more parsimonious characterization of verb auxiliary structure or whether our context-sensitive rules do. However, the ALAS rules seem much easier to learn. This is one illustration of many where learning considerations can be used to gu1de linguistic description.