Robot Control Strategy

ADS TRACT

1\ bei1avioral theory of information pro­ cessing based on ethological observations has been tested experimentally by program­ ming a robot control system. The result­ ant software has successfully simulated robot operation. Three basic types of software modules are assumed as elements. These accept input stimuli and generate de tailed strings of motor response "units" to attain design goals. A complex process of interaction between the modules permits the consideration of many alternative actions while focusing attention on a selected set of input signals. · The organ­ ization is shown to have especially flex­ ible characteristics adapted to robot con­ trol and permits a recursive definition of some of the modules. Brief comparisons with other robot systems in being indicate some o f the relationships between the dif­ ferent design approaches so far proposed.

INTRODUCTION

The software for integrated robot design may be functionally divided into pattern recognition programs and control systems. Control system programs analyze the envi­ ronmental state data furnished by the pat­ tern recognizers, initiate decision pro­ cesses, and issue motor commands intended to arrive at goal states. Many different approaches have characterized the techniques used to arrive at decisions which generate sub-goals and produce detailed motor acti­ vity.

The Stanford Research Institute and Stan­ ford University groups have stressed mathematical and engineering schema to solve these problems.l,2 A team from MIT Instrumentation Labs. have simulated the hypotheses of a neurophysiological theory3. A control system model based on instinctive behavior production principles is described here that provides still another viewpoint of the general problem of producing intelli­ gent behavior by machine. The rationale for relating animal behavior to this model is described elsewhere.4 Ethology (the study of animal behavior) has provided a mass of data on instinctive or preprogrammed animal behavior. It has also introduced some theoretical concepts that indicate there may well be common features in the organ-

Leonard Friedman

1'RH Sys t err:s Group One Space Park Redondo neach, California

ization of information processing at the instinctive level through much of the animal kingdom from insects to mammals.S In any event, the robot control model presented here has processing features which are divorced from the particular robot design inputs (the sensing elements) or the outputs (motor capabilities). This permits a common system of control organ­ ization for widely differing sens o r y capacities and motor repertoire.

After describing the main characteristics of the system, its operation will be com­ pared with some of the other robot systems designs. The general strategy employed for finding a path through state space from initial state to goal will be dis­ cussed. For a certain class of problems, this strategy offers advantages.

To test the validity and usefulness of the model, a robot simulation was pro­ grammed in JOVIAL on a Philco 2000 com­ puter. As part of the simulation, a dis­ play generated an animated cartoon in real time of the robot moving about in its environment, solving the problem si tua­ tions it encountered. In fact, a movie has been made of typical robot activity. The simulation program is divided into two parts. One part simulates the con­ stantly changing environment; the other part is the robot control program, organ­ ized according to the hypothetical de­ cision processes assumed in the behavioral theory4. 'l'he architecture of the computer program itself, while faithfully repre­ senting the behavior theory, introduces another assumption having nothing to do with behavior; i.e., most of the infor­ mation processing is carried on in iden­ tical modules or, equivalently, is assumed to be recursive. This formalization has implications for further development dis­ cussed after the principles of organiza­ tion are explained.

ELEMENTS OF THE HIERARCHY

The decision and control function is per­ formed by two groups of information pro­ cessing modules called Releasing Hechan­ isms (RM's) and Selectors of Releasing Mechanisms (SRM's). The internal structure of each RM and SRM is nearly tdentical.* They differ principally in the nurr~er and destinations of their in­ terconnections (input/outputs). The behavioral theory assumes a hierarchy of many RM's and SRM's functioning in parallel. Correspondingly the simulation program uses a single subroutine to repre­ sent each RM and each SRM, entered recurs­ ively for each level of the hierarchy.

As the result of interactions between sensory inputs (stimuli) and these pro­ cessors, a third group of processor ele­ ments is activated. These are called Motor Control Units (MCU's) and are pro­ grams controlling the purely motor func­ tions such as turning wheels, steering or moving arms. See Figure 1. Clearly the internal structure of each MCU is depend­ ent on the specific mechanical design of the robot function it must control.

It is the task of the executive control program to activate in proper order, sometimes even simultaneously, the MCU's, thus forming strings of MCU's in order to accomplish desired changes in the environ­ ment. The control program has to perform very much like a piano player, with the keys analogous to the MCU's and the melody and chords the desired output. For the robot simulation in this study, the list of these motor control units is naturally a simplified one. The following MCU's have been implemented:

MOVE BY ROTATING DRIVE WHEELS CHANGE STEERING WHEEL DIRECTION MOVE ARM GRASP WITH "HAND" RELEASE FR0!1 "HAJ.'lD"

INTERCONNECTIONS AND OPERATION

Each RM possesses a set of sensory inputs \vhich is peculiar to it. These stimuli may be a few simple signals or compound events, a small or a large set. The RM's, subjected to complex and often irrelevant stimuli, must produce goal-oriented responses, the MCU strings. The RM's constitute the nodes of a set of hier­ archies or tree structures (Figure 2) · In order to p roduce an output, control must pass to an activated RM. Activation is accomplish ed by a separate mechanism associated with each level of every hier­ archy. This is the S RH net shown in Figure 3. RM's may or may not be excited by input stimuli. If they are, the output

• An exception is goal selection and sub­ goal generation in the RM. This may be different for each RM.

is sent to the SRM at that level of the hierarchy. The SRM then picks the maxi­ mally excited RM for activation.

RM's are of two kinds, branching and terminating. A branching RM has control of several subordinate RM's; any of which can take control next. A terminating RM either controls an HCU directly or con­ trols the equivalent of a subroutine. These subroutines so far have consisted of complete subassemblies of the three elements with all branches termina ting in HCU' s. Thus, a given hierarchy a h1ays terminates in MCU's.

Subroutines have been introduce d because many functions such as "search" patterns and "go to" performance have such wide­ spread use it would clearly be wasteful to duplicate them for different hierar­ chies. Besides branching and subroutines , looping, the remaining important facility of a program, is easily provided as explained later.

CONTROL PROGRAM .OPERATION

Each hierarchy or tree structure of RM' s is dedicated to the accomplishment of some specific major goal with the set of hierarchies united at the top into a single overall hierarchy, the instinct drive structure. Biological examples of major goals are food seeking, reproduc­ tion,and defense. The implemented robot simulation at present is concerned with only a single major goal, "nest-building."

To make responses corresponding both to the state of the environment and the task, the control program must first select an appropriate hierarchy of RM's. To ac­ complish this initial selection, the top or zero level SRH network (Figure 3) is organized into a PANDEMONiill-1 in the fol­ lowing manner6. Processed sensory sig­ nals from both the internal and external environment are sent to the top level RM's. Different sets of signals are directed to each top level ill1. The RM node scans its specific inputs from the e nvironment and outputs a s igna l to t he SID-1. An output g r e ater than zero occ urs \vhen the pattern of inputs from the environment corresponds to the appropriate sit­ uation to which the given RM is supposed to react. That is, we have in effect compared a model situation preprogrammed into the RM and the actual environmental input. There are many RM's competing in parallel for activation by the SRM. The SRM receives the output signals from all the RM's.at the top level, selects the largest signal from this set, and activates the RM from l'lhich the largest signal came, thus selecting a hierarchy. A concrete example may clarify this pro~ cess. Suppose the exciting signals for nest building in Figure 4 are "temperature below threshold" or "direct command" from a human controller,and the exciting signal for refueling is "fuel level belov· a threshold". In this arrangement, we would design our signal to the SRM from the "Refueling" RH to be larger than the maxi~ mum signal from "Nest Building" before a dangerously low level of fuel was reached. Then if the robot started with a full gas tank and a command to construct a nest, it would interrupt its structure building to engage in refueling when the analogue output signal to the SRM from the "Refuel~ ing" RH exceeded the corresponding signal from the "Nest Building" RM.

The RM's at the top of the hierarchy are never blocked from receiving input signals, but, as Figure 4 shows, each level of RH' s bel01., the top is blocked from receiving inputs until its superordinate ·RJ.'\1 is acti~ vated. Once inputs are unblocked, however, the selection of an ill-1 at the new level occurs in a fashion similar to the top~ level selection process, thus constituting a multi~layered PANDEHONIUH. As the acti~ vation of RM's descends, a level at a time, a new set of RM's is permitted to look for the particular patterns in the environment to which these RH' s are ready to respond. If all goes well and the necessary patterns are found in the proper sequence, eventu~ ally a terminating RM is reached and one or more Motor Control Units are called which, by their operation, will change the environmental state. When this happens, the whole process of searching through the entire hierarchy from the top level dol'ln is repeated, possibly activating a differ~ ent HCU and simulating parallel operation of the hierarchies.

Provision must be made ·for unsuccessful performance of an HCU or a failure to find the necessary environmental signals. When this occurs, it is necessary to back up in the hierarchy to attempt a pass through a different part of the tree structure. This backup is accomplished by a superor~ din ate RM, monitoring its subordinate pro~ cessors for success or failure. If a failure is detected from its subordinates after an RM is activated, the superordinate reduces its output to zero, regardless of its inputs, and a new choice is made at the superordinate level if one is possible. If no choice is available, the processing backs up still another level, clear up to the top if necessary. The top level is arranged so that it always finds a choice. An absence of external signals could con~ stitute a signal to enter a maintenance routine, for example. An example of such a backup may be traced in Figure 4 with the inputs defined in the previous ex~ ample. Suppose "Nest Building" is com~ manded and "Seek Nest Site" has for an exciting signal "Has a proper site been found?" If the search strategy of this HM is limited to a single sweep of the area (which is actually the case in our implementation) there might not be any site found in this time that met the stored criteria being sought. According~ ly, a failure would be reported to "Nest Building" that would cause it to reduce the output signal to its Sill-1 to zero. The robot would then try refueling or main~ te~e depending on their relative exci~ tations.

A similar process of backup may take place if two HH' s at the same level pro~ duce maximal outputs to the SRM of exactly the same magnitude. If the outputs of the two R!>-l's do not conflict, both may be chosen for activation by the SRH. If there is a conflict possible, both may be suppressed and an alternative activity sought in the hierarchies. This phenom~ enon is actually observed in animals, and the circumstances under which it arises are described in the ethological litera~ ture relating to "displacement activity".s

GOAL SELECTION MECHANISM OPERATION

A terminating ru-1, which may be at any level of the tree, calls a ~lotor Control Unit or a terminating sub~routine. To do so however, the RM must be able to specify all the parameters needed to serve as a goal for the output procedure. For ex~ ample, if the subroutine is "GO TO de~ signated object," and three or four ob~ jects possessing some of the required a~~ tributes are seen by the robot, some cr1~ terion must be used by the RM to select just one. The criterion might be "clo~ sest" or "leftmost." vlhatever the cri~ terion, each RM must be able to examine goal determining inputs (not necessarily the same as the exciting stimuli which caused an output to the SRH) , and on the basis of these inputs, it must select the goal parameters needed to operate its subordinate data processors. This mechan~ ism, internal to an ru4, is called the Goal Selection Mechanism (GSH). The pro~ cess of generating subgoals has not been standardized. Each HM may have a differ~ ent routine in its GSH.

l-Ihat is meant here may be illustrated by some of the goal selection processes in the GO TO subroutine. The primary goal in the GO TO subroutine, 0nce a prime tar~ get has been selected, is a simple meas~ ure. The goal is attained if distance to target is less than a stored quantity (determined by the robots arm length).

If there are obstacles between the prime target and the robot, subgoals are gener­ ated using some fairly complex algorithms. These subgoals are directional vectors that are determined by calculating the smallest angular deviation permitting the robot to pass around or through the set of obstacles it is facing. The deviation is calculated from the prime target direct­ ion at the robots position and the algo~ rithm selected depends on whether a "Single Obstacle" RH, "Two Obstacle" RH or "Hore Than Two Obstacle" RH is excited. On the other hand, the goal selection previously cited, involving choice of a prime target from a group of three or four suitable objects, can be implemented using the PANDEHONIUH apparatus with "closest" ex­ citing the largest signal.

If the fu~ has been tentatively selected for activation and cannot find a suitable goal, it reports a failure and the b~ckup process is initiated. In addition to spe­ cifying goal object, the GSH specifies a desired goal state. The GSH monitors the environment for a defined goal state of success or failure and relays the appro­ priate signal to the SRM when such a state is attained.

INTERNAL STRUCTURE OF THE RH AND Sfu~

Figure 5 is a block diagram that represents in simplified fashion the flow of informa­ tion through an RH to an SRH and back. Initially, inputs are summed. These inputs originate from any place external to the given HH and are quantities with a posL­ tive or negative numerical magnitude. The inputs represent highly processed patterns: i.e., the answers to a test such as, "lias a nest site been found?" The negative mag­ nitudes tend to inhibit an RH output. After the inputs are summed, a threshold quantity is subtracted and the resultant, representing degree of correlation with the expected situation for that RH, is transmitted to the SRH. The Haximum Select procedure of the SRH finds the RH that has sent the largest positive signal. If this is a new choice, the Sfu~ signals to the selected RH to try to find goal parameters. If the RM can find goal para­ meters, it is activated and the processing descends another level in the hierarchy. If all the parameters can not be specified, the output of the tentatively selected RH is reset ·to zero and the SRH repeats its processing. If there is no output greater than zero from any other RH at that level, a failure signal is passed to the SRM at the next level above in the hierarchy and the SRM at this higher level sets to zero the output of the RM at its level and in control of the lower level: the SRH then reevaluates. A complete reevaluation takes place at the completion of every HCU

output and can also be made under pri­ ority interrupts. This is a limited simulation of full parallel operation at the top level of the set of hierarchies.

Tracing through the information flow of a "Refuel" RM may serve to make Figure 5 clearer. Figure 6b illustrates such an Pl1 with "Difference bet·.,reen Full and Present Gas Level", the exciting stimulus and "Gas Station Seen" as an input to the Goal Selection Mechanism. \'lith such an arrangement, if the exciting stimulus exceeds some pre-set threshold, the "Refueling" Pl1 begins to compete with "Nest Building" and "Maintenance". ·If this should be the largest signal, the RH will be tentatively selected. However, if no gas station \'lere in view, its out­ put would be reset to zero and another R~ selected resulting in a different activity despite strong excitation.

PRO�~1IN� BY HIERARCHY ASSEMBLY

Using this system to produce a particular pattern of behavior requires the program designer to characterize expected states of the environment between initial and goal states by patterns or cues external to the given PM and discriminable by the robot. Such cues \Wuld include success­ ful completion of a previous robot act, permitting chaining of motor activity. It is not enough merely to find any set of cues. For successful operation, there should be included environmental signals that are highly improbable in any context except the desired activity.

Animals that rely heavily on instinctive behavior have developed just such dis­ tinctive cues in social interaction. For example, male stickleback fishes, normally white-bellied for camouflage, show a bright red belly during the mating season. Other males and females react to this color with complex behavior even when it appears on red-bellied dummies otherwise quite unlike a fish in appear­ ance!

A set of intermediate goal states \'lhich are highly likely to appear must be identified by the system designer. In addition, he must decide on the proper operations upon the environment to be effected by sequences of activated HCU's to pass from one intermediate goal to the next. A hierarchy of RH' s and SRH' s with the appropriate cues and calls to a succession of MCU's can then be most con­ veniently specified in a compact para­ meter notation. This formalized notation is supplied to an instinct "operating system" which assembles each set of ele­ ments as the environment and hierarchy selects them, Thus, the designer can spe­ cify an arbitrary hierarchy of stimulus­ response elements to solve any desired problem within the constraints of the ro­ bot design capabilities for sensing and acting and he can do this with a minimum of instructions. The operating system or executive will call the activated sets according to these instructions and auto­ matically provide the parallel multi-choice PANDE1·10NIUH hierarchies without further coding by the designer.

An example of the kind of system design possible may be shown by two different arrangements of the "Refuel" function as shown in Figures 6a and 6b. The arrange­ ment of Figure 6a means that only a single top level threshold determines whether the robot will engage in refueling activities, including searching for a gas station. The arrangement of Figure 6b permits two thresholds, At the higher level of fuel state the robot will refuel only if a gas station happens to be seen while it might be doing something else, If th'is should not happen and fuel level continues to drop, the "Seek Gas Station" RH will then preempt other activity.

Bes ides branching and subroutines, the other programming element, looping, is readily accomplished by an appropriate hierarchical arrangement of RM's. A su­ perordinate m1 can specify a goal state not necessarily immediately attainable by the action of its subordinate RH' s and MCU ' s . Nevertheless, a chain of events can be specified at the subordinate level designed to accomplish part of the task at each pass and such that it is a circular chain; i.e., actions at the end of the chain return the discriminable environment to the same state it was in at the begin­ ning of the pass. For example, if the task is to pile objects at a particular place, the superordinate exciting condi­ tion may be "less than · 6 objects at (x,y) ", 'rhis would cause subordinate R!1' s to acti­ vate first a seeking behavior pattern in the environment and then fetching opera­ tions to (x,y) for a first found object. Once this object was placed, the R!l for initiating search \·lOuld he excited again, as "less than 6 objects at (x,y)" is still true. Action ',vould continue till the goal s tate of six objects placed was reached, Note that action is automatically discon­ tinued once the robot has sensed that the goal state is attained even if the robot did not place the objects itself,

A LANGUAGE FOR HIERARCHY ASSEHDLY

The present status of the system is such that the program designer must insert RM designations in a table provided, specify which RM' s are subordinate to which by other tabular values, etc. The whole p ro­ cess is now formal enough to permit the construction of a special hierarchy as­ sembly language "lith conunands expressing concepts such as:

ADD INPU'r STH1ULUS xx TO RH yy. R!-1 z REPOP.TS TO RH a.- HCU-bb REPORTS TO PM c. FH daGOAL Pl\RAI'-!ETERS-GO TO SUBROU'l'INE Rl\1 ££_, etc.

Such a specialized language is of course possible in any computer application whenever the use justifies the effort of writing a translator, and the formal syn­ tax can be specified. Just such a use is foreseen in adding learning capabilities to the present system.

~~ APPLICATION TO LEAlli~IN�

Various forms of learning have been pro­ posed to make programs behave intelli­ gently. Samuel's method? of ch anging \.,reights has been the most successful to date, but it suffers from one defect common to all the really powerful arti­ ficial intelligence programs so far im­ plemented, Namely, the features or cues from the environment to which adjusted weights are applied must be recognized and supplied by the human program design­ er.

Suppose a learning program devoted, among other things, to determining previously unknown cues from the environment can be designed to operate in conjunction with a substrate of "instinctive" starting capa­ bilities. In other words, the robot has enough built in capability to give it a chance to learn. Once the learning mechanism had determined a ne\v cue, it would have the task of modifying the in­ stinct structure to incorporate this new input signal. If the learning mechanism had a hierarchy assembly language availa­ ble and a translator in operation for executing a chosen assembly command, such facilities would permit efficient hand­ ling of this part of the total learning task,

There remains the difficult job of se­ lecting the right command, It should not be necessary to resort to random mechan~ isms such as Friedberg used to do this,8 The context in which a new cue is isola­ ted will furnish heuristic pointers i­ dentifying with some likelihood the RM to which it might be associated, For example, let us assume that a constant association in time takes place between a previously unknown cue and one that is known because it is already used to ex­ cite an RM. Such an association makes that RM a likely candidate for adding on the newly discovered cue. In short, con­ straints are sought in syntactic relation­ ships between the unknown and various elements of the existing hierarchy as de­ termined by interactions with the environ­ ment. Only selection of such commands will be attempted as can be reasonably constrained to a limited set of possibili­ ties.

PRESENT STATUS

At present, minimal use is made by the in­ stinct system of updated internal memory. on the other hand the state of the ex­ ternal environment itself is used as a kind of memory of what the next action should be. This feature is reflected in instinctive animal behavior and was long ago noted by the ethologists 9 · It obvious­ ly results in a great economy of memory.

Because no planning programs have been implemented as yet, the strategies avail­ able to the present system are sequential; i.e., the robot tackles the environmental state in its immediate neighborhood, acts on it, considers the resultant state, etc. There is almost no look-ahead. On the other hand, the system exhib~t7 consider­ able flexibility and adaptab1l1ty con­ sidering the rigid strategies it must now employ. For example, the robo~ fi~ shows a search-fetch sequence 1n wh1ch ADROIT* sets out to pick up a nest­ building object. Hhen a similar object is placed directly in front of it, ~ROIT changes goals, stops moving towards the previous target object and immediately picks up the newly presented one. This happens because all goal ob­ jects in the field of view are being re­ vie\ved for optimality and the closest be­ comes the prime target. Similarly any goal states specified in the RH hierarchy and in a "state-of-being" before motor action is initiated are detected and motor action to get to those states is suppressed accordingly. This results.in "behavior" most unlike that of an ord1nary machine, and of the kind normally associated with animals.

This suppression of motor action usually taken by the program is an example of "unplanned" branching which is typical of ADROIT's functioning. \ve may regard the subordinate RH' s in the tree structure and the !1CU sequences they produce as planned branches or jumps from a super­ ordinate RH. Any time the program de­ parts from a normal string <;>f NCU' s c;-nd instead produces an MCU str1ng assocr~ted with a higher or lower level of the g1ven hierarchy or a completely different hier­ archy, this may be denoted as "unplanned".

• Acronym for the robot

It is characteristic of the program in operation that its action is full of such jumps. It has been our experience that once the program is debugged, these jumps appear appropriately natural and "life­ like".

ASSUMPTIONS ABOUT PATTERN RECOGNITION

The most ambitious of the robot construct­ ion programs to date is the SRI project. The project has focused on actual hardwa r e rather than simulation and has already achieved notable progress. It has de­ veloped some elaborate pattern recognition programs, particularly for the visual sense. Table 1 shows a comparison between the sensory and motor systems developed by SRI and the assumptions about pattern recognition capabilities of the simulated ADROIT systems.

The McCarthy project at Stanford Univer­ sity and the Minsky - Papert arm and eye project at HIT have made considerable pro­ gress in visual pattern recognition pro­ blems, as vmll as arm coordination sensing and motor adjustment. A greater pattern recognition capability than assumed for ADROIT's arm has been implemented by these teams.

CONVERGENCE OF TECHNIQUES

The SRI project has already made use of powerful mapping and planning techniques with great success. Nevertheless, SRI has introduced programs using unplanned se­ quential action techniques similar to ADROIT's for the case of l a rgely unknown environments, in order to avoid excessive computer processing associated with p l an­ ning. On the other hand, the total la.ck of planning even when the information is available is clearly a defect of 1\DROIT' s . Planning will be introduced into the ADROIT system at the earlie st opportunity .

The 11IT Instrumentation Laboratory p r o j e ct which is designing a robot for unmanned e xp loration of Mars, h a s simulated a ne urophysiological t heor y o f operation of t he reticular formation for its decis i on r.1aking system. This model appears t o s erve the same function as choosing a raajor goal hierarchy in the ADROIT be ­ havioral model.

The reticular formation simulation arrives at its choice by statistical rather than deterministic means. Although exploring in great detail hmv such a choice of mode is made, the published descriptions of the simulation do not discuss methods to arrive at appropriate motor action.3 Nothing is knmvn to us about how the pro­ ject plans to achieve detailed and coor­ dinated motor behavior.

The ADROIT program uses a kind of oper­ ating system executive to provide the pointers for the fully recursive capabili­ ty not available in JOVIAL and to assemble the hierarchy stipulated by the program designer . 'rhis executive is itself 1vritten in JOVIAL, as are all the other non-display routines, thus avoiding language incompatibility problems faced by the SRI System.

This is a somewhat misleading compari s on, however , because the motor control unit routines in the simulation are merely simple a l gorithms determining ';lpdated. position of the affected part 1n obey1ng a commanded movement, whereas in a h ard­ \vare system, the rapid action required demands assemb ly language as i mp lemented in the SRI system. rurthermore, nothing in t he ADROIT System corresponds to the SRI plan o f an english language commun~­ cation system between man and robot, w1th subsequent robot processing by 'first­ orde r p r edicate calculus. This has been programmed by the SRI team in LISP 1:5: It is premature as ye t to make a dec1s1on about the basic language to be used for ADROIT's learning mechanisms.

SUM.."1ARY

The effort to simulate a theory to ac­ count fo r the production of instinctive animal behavior has led to the develop­ ment of information processing module s useful in robot control design. To cre­ ate a working system, the modules are assemb led into larger structures possess­ ing the properties of any program; viz; branching, subroutines, and looping. In this r egard, the technique is similar to the biological processes of evolution whe reby various standard modules, the specialized cells, are assembled into organs. In our analogy, the genetic and developmental processes leading to organ assembly take the place of the program designe r and operating system executive leading to hierarchy assembly.

'l'here are certain advantages to this form of organization. Hany of the de­ s ired structural features of the program can be written in a compact notation stipulating only a few parameters such as module interconnections. An almost uniform formal structure has thus been es tablished for accomplishing any robot control task amenable to programming. This particular structure has automatic­ ally built into it certain interactions not standard to programming. Among these are a correlating process for es­ tablishing the most appropriate branching action, given the present state of the environment. The branching action auto­ matically works in backing up in the hierarchy to a higher level of decision 1vhen encountering failures, as vTe ll as moving down to motor action vli th success. The re is an effective "focus o f attention" action so tha t frequently irrelevant in­ puts \·Till be ignored . At t he same time, all the stimuli known to be relevant to a group of the tasks- a t-hand are actively sought and processed, Goa l states t hat normally require motor action to attain will be detected if t hey exist beforehand so that unnecessary motion is suppressed and corr.ect subsequent act ion automat­ ically is chosen.

A learning system at a higher l evel is planned, to operate in conjunction with the assembly of h ierarchies constituting the actual robot control. Not the leas t of the advantages of a uniform formal representation of this control sys t em is that it permits the introduction of a hierarchy assemb ly language and transla­ tor. The learning system may then use such a language to add to and improve on the l·lOrk of the program designer in adapting the control system to particular environments.