An Intelligent Robot with Cognition and Decision-Making Ability

1. Introduction

In order to complete an inte~rated auto­ mation system, a visual device to take the place of human eyes and a coordinated mechanical system which can simulate movements of human hands are necessary, Particularly , in machinin~ and electronic component industries, which mainly handle individual solid materials, automated control of production is extremely difficult, comparin~ with the situation in some other industries which handle collective fluidic matters.

Little pro~ress has been made of late especially in the techniques of automatic assembly, where numerous component parts are to be put to~ether into a complex unit and of product inspection, which still depends mostly upon visual screenin~ by human eyes, ·Hi phly complex co~nition ability of human visual system and intricately coordinated maneuvera­ bility of human hands have so far rejected in these processes a complete take-over by mechanically oriented systems, It is obvious, however, that the future industrial production will need more sophisti­ cated automation techniques, and the realiza­ tion of human visual co~nition and handlin~ functions by machines seems to be t he only breakthrou~h of current automation techniques that have already been saturated.

As the conversion of production proceeds from the currently prevalent method of producin~ a mass of uniform products to a method of producin~ varied products in small or medium quantities, in response to the increasin~ diversification of consumers' tastes, the necessity of the new automated techniques will more and more increase, It seems obvious in the future production processes that assembl y and inspection functions will have to be substituted by versatile systems wi th some intelli~ence, to cope with the diversified production lines and the chan~e of products.

There are several attempts to make machines capable of performin~ visual perception of three-dimensional objects (1),(2 ) , (3) and solvin~ the problems concernin~ robot behavior (4),(5). Our present work is also one of these attempts intendin~ to develop

useful and versatile robot svstems in future.

2. Outline of the intelli2ent robot

The intelli~ent robot bein~ described below is a prototype system developed to aim at basic researches in buildin~ up automated visual co~nition and handlin~ functions to be required in the future assembly processes, The robot, named HIVIP, is composed of t hree nuclear sub-systems; F.YE, DRAIN and HA~D (see

Fi~ure 1).

The sub-system F.YE is composed of two television cameras. Camera 1 studies the macroscopic instruction ~iven in the form of three-view drawin~ and reco~nizes the overall spatial confi~uration of the object shown in the drawin~, as well as the shapes, the number, the mode and order of assembly of the compo­ nents that make up the object. Camera 2 studies actual component parts separately but arbitrarily placed on an assembly table and confirms the shapes, the locations, the postures and the dimensions of the individual components. \,'hen the camera 2 finishes locatin~ the parts, it then finds the specific parts r .equired for each step of assembly sequence, which is automaticallv pro~ramed throu~h the analysis of the drawin~ by the sub­ system RRATN, a di~ital computer. Yhen these

recognition and decision-makin~ processes are over, the HAND mechanism is activated and starts the assembly process accordinR to the pre-conceived order,

3. Ma1or characteristics

Presently available industrial robots are lackinR in intellectual analyzin~ capabilities to control their movements in reaponae to various conditions of the objects or the circumstantial situation they are subjected to, Furthermore, they are still insufficient in versatility to comprehend human instruction and to adapt themselves to the new joba,

In Reneral, all the instructions of existinR machines have to be fed in microscopic form with one-to-one correspondence to each step of machine movements, as aeen in numeri­ cally controlled machine tools, rhe HIVIP, on the other hand, accepts macroscopic instructions Riven in the form of Rraphically illustrated plans, thou2h they are still too simple comoarin2 with the actual industrial drawings. ~~en the camera's acrutinization takes in all the information the plana offer, the comouter starts planninR detailed work procedures required to accomplish the instructed assign­ ment. Capability of studyinR and understandin2 macroscopic instructions may be a necessity in future intelligent production system where instant and autonomous response to chanRes in production procedures is required, The HIVIP see~s to be a simple prototype of this future Reneration robot, as it is capable of studyinP. and und er~tandin2 the ~raphic plans even they have never been exposed to before and of takinR nece s s~ry actions accordinRly,

4, Hardware svstem

The PIVIP is composed as shown in FiRures 2 and 3, The two vidicon cameras function as viewinR eyes and are operated by a scannin2 system which closely resembles that of the standard TV system, The imaRe area is divided into 76,800 picture elements; 240 vertical and 320 horizontal. The information of each picture element is transformed into 5-bit (32level) diRital information throuRh an analoR­ to-diRital converter and is transmitted into t he process control computer, HITAC 7250, ~h ich has 32,768-word 16-bit core memory and operates at 2-microsecond cycle time and 4,5-

rn icrosecond add time.

The computer can reRulate the TV cameras to hlock off an area of optional size of the lens view, permittinR ima2e information beinR viewed hy the remainin~ oblonR openinR of the lens, The computer can also instruct the camera to reRulate the 'fineness' of the photoRraphed ima2e bv thinnin2 out the matrix of the picture element layout in three modes, Mode 1 : rouRh viewinR. Every four picture elements on both verti­ cal and horizontal axes are fed in, Mode 2 : medium viewinR. Every two picture elements are fed in. ~de 3 : fine viewing , All picture elements are fed in, Ry reducinR the size of the area of the lens view, the camera narrows the eye and focuses on the object, rejecting all unnecessa­ ry objects from the field of vision. These visibility control functions, for instance, enable the camera to rouRhly scan the premises where the ob.1ects are positioned; then, when the object's locations are confirmed, the camera narrows its eye by blockinR out the lens area viewinR unnecessary objects in the premises, leavinR open the oblong window to take in the particular object for a closer study, and starts makinR scrutinizinR survey of the shape, the position and the posture of the ob1ect, The visibility control functions can also separate the photo ima2e into two contrastinR areas, bright and dark, bordering on a certain threshold instructed by the computer, or can differentiate the imaRe to represent the objects with only their outlines, The handlin2 mechanism of the HTVTP is an articulated arm with a parallel-jaw tvpe 2rippin2 mechanism, ~even de2rees of freedom of motion in all are equipped in the handlin2 mechanism which can be controlled simultaneous­ ly by seven independent servo-mechanisms, Tn handling control equipment, there are seven data registers, pulse distributors and diqital phase modulators that act as di2ital-to-analo2 converters and error detectors in the servo­ mechanisms, The control si2nals are compared with the feedback si2nals from synchro-resolver type position detectors in the arm, and are fed into thyristor-driven d,c, servo-motors,

5, ~oftware svstem

The software of this robot system is controlled by process or non-process monitorinP system (P~S, NPMS) of HTTAC 7250, It is classified into three maior pro2rams; a drawinv. reco2nition pro2ram, an ob~ect reco2nition

prov.ram and a handlinv. prov.ram,

5.1 Drawins recognition aleorithm

The initial instruction to the HIVTP is v.iven in the form of a rouv.h, macroscopic plan showin2 three sides of the re~uired a~~emhla~e (see Fiv.ure 4), The plan is, at oresent, composed only of strai2ht solid lines, accordinv.ly the assemblaRe must be a polyhedral structure composed only of flat planes, The robot first scans the plan and recov.­ nizes all the lines before making up a list of nodes, branches and loops. From the two­ dimensional information thus obtAined, the robot, then, checks the co-relationships amonv. them, calculates the spatial points, lines and planes and constructs three-dimensional configuration of the assemblage, The robot is also capable of suspecting and confirming whether a termination of a line is actually a point ed2e of the assemblage or just a false ima2e lookinR like a point. The assemblav.e shown in Figure 4 is made up of A grooved block (an octagonal prism) and two trian~ular pris~~ mounted on top, The point A in the front view of the assembla2e is a false point created by two crossing lines. The points B, C, D and E shown in the front view are not seen in side and top views, The robot is capable of speculating that the point A might be a false point and the line FA be actually a visible section of a real line FG. It can also imagine the existence of lines indicated only by points such as E, c, D or E,

~~en it finishes compiling a list of points, lines and planes in three-dimensional space, it then begins to figure out a structural image of the assemblage and in this process, eliminates all unnecessary infor­ mation, ~~en this is completed, it breaks down the created structural image into independent component parts and be~ins to examine the way they are assembled together to relay this information to the handling system, In order to break down the assembled image into ima2es of component parts, it employs a principle that a polygonal plane with a certain number of sides is linked with other planes with an equivalent number of arms, ~~en it finds that a line is co-owned by two planes, it classifies the method of the linkage as a tight link and when the line is co-owned by more than two planes, the linka2e is termed as a loose link, Thus, the robot disintegrates the set of planes into several subsets or 'shells'. Figure 5 1hows the way the assembled planes are disintegrated into a number of shells. After the planes are classified into several groups, the robot put together these open shells to construct closed 1hell1 and re2isters them in the lilt of parts. The surfaces to be co-owned by two parts are regarded as the contact surfaces when the assembly is carried out, Fi2ure 6 is the example of computer drawings on X-Y plotter after the computer completes the drawing recog­ nition process,

5.2 Object recognition algorithn

The object recognition is performed by utilizinR the brightness of light reflection from each surface of the objects on the table.

The brightness detected by the vidicon . camera is at first transformed into 5-bit div.ital information and fed into the computer which, in turn, recov.nizes the positions, postures, shanes and the sizes of the objects. The addition of color identification capabilitv would further make the object recognition easier. This, on the other hand, is not alwavs practical in the actual use, as the unusual coloring of component parts is required in lome cases. Since the environ­ mental situation seems to be regulated easily by simplifying the background and adjusting the lighting even in the future actual factory application, the HIV!P is equipped with only brightness identification capability so far and white colored sample objects are used in the demonstration.

The HIVIP can at present be applied to simple polyhedral objects composed of plain surfaces, especially polygonal prisms with the shapes easily processable with milling machines,

The first step of the object recognition alv.orithm is to spatially differentiate the image taken into the core memory and transform it into a hiv.h-contrast ima2e with the edge lines of the objects strongly emphasized, Then, one of the objects in the image is selected by maskinv. all other objects, and the position of each edv.e line of the object is detected by 'trenching' method and memorized in a list. Figure 7 shows the process of line detection drAwn bv the line-printer output device. The lines thus detected are sometimes unclear as shown in Figure 8, Therefore the robot rearranv.es them by carrying out a few procedure such as the elimination of unnecessarv twigs, Even when some of the edge lines are missinR, the robot is capable of fillin2 them in by correctly analyzin2 the spatial structure of the object, The information thus obtained finally determines the object's coordinate position on the table and its posture. Above processes are repeated for every object until the reco2nition of all objects in the ima2e is completed,

5, 3 Handling algorithm

When the robot completes the drawin~ recognition and the ob1ect recognition, it then starts thinkin~ the assembly procedure by evaluatin~ the relationships between the parts in the required assembly form. At first the object~ on the table are classified into two cate~ories; the necessary and un­ necessary objects for the instructed assembly. If there are excess parts with the same shape, only the required number of parts at the easier handlin2 positions and postures are selected and re~arded as the necessary parts.

Then, the assembly order is determined as the reverse of disintegration order by computin~ the possible disintegration sequences of the assigned complex into pieces, In general, there exists several sequential orders of disinte~ration as seen in Fi~ure 9, In this case, by evaluating the postures of the objects, the most preferable order is selected where the duplicate manipulation of the object by chan~in~ the ~raspin2 position is not necessary, or minimum in number.

As shown in Fi~ure 10, the robot further selects two parallellina surfaces of the parts for arasping, excludina the contact surfaces of the parts. Then, it determines the paths of the hand, the positions of assembly and the directions of approach by computing the vector sum of the contact surfaces, h'hen these deci­ sions are completed, it then calculates the an~ular displacement of each joint of the arm and dispatches the determined data to the seven data registers in response to the request from the pulse distributor in the handling control equipment, Thus, the seven different movements of the articulated handlina system are re~ulated independently and driven simul­ taneously with a continuous-path mode,

~'hen it is jud~ed at this decision­ makinR sta~e that there is the lack of parts on the table or that the hand mechanism is hard to reach the objects physically, the robot prints out the messaae on the typewriter, aives up the assembly and waits for a new instruction of t he human master.

6, Discussion and conclusion

As described above, the HIVIP is a proto­ type of the intelliRent r.obot that can under­ stand the macroscopic instruction given in the form of a three-view plan and is capable of makina necessary decisions to accomplish the assianment indicated in the plan, The development efforts of this robot have mainly been focused on the exploration of the processinR techniques that can deal with the i nformation of co-relationships between three­ dimensional objects in contact, as well as the information of individual objects. The drawina recoanition capability is especially a unique achievement of this effort and seems to be sianificant not only in ~iving a machine an intelli~ence to understand a macroscopic instruction but also in establishing an impor­ tant basis of araphic display technique which may develop into a sophisticated computer­ aided designing technolo~y.

As the major concern of the present work is the computer al~orithms, no special effort has been taken so far on the processor hardware, and the conventional digital control computer has been utilized, The entire software system of the robot, includina the operating system ~oftware and the workin~ area for visual imaaes, exceeds 400 kilowords of memory, and is stored in 512-kiloword magnetic drum memory as chain jobs, In addition, more than ninety percents of programs are written in FORTRAN of this particular computer whose speed no lonaer seems to be hi~h enough, These are the main reasons of takin~ much computation time as 240 seconds for image processin~, 20 seconds for drawing recognition, 50 seconds for each object recognition, 10 seconds for decision-making and 180 seconds for assemblv movements on the average, It may be possible in this case to reduce the time to one-tenth or even to one-twentieth by refining the program, yet the development of low-cost processors with totally new concepts, which may have at least the parallel processin~ capability, ~eems to be essential for the solution in future industrial appli­ cation. Thus, our present work rather posed the questions that must be answered in future, and the quantitative and qualitative improve­ ment of reco~nizable objects is one of the requisites that we should study next for intensifyin~ the problem-solvin~ ability and approaching to the intelligent systems,