A Knowledge-Based Interactive Robot-vision System

1. INTRODUCTION

This paper reports on a robot-vision lab for RandO and for experimentation of specialized industrial applications. The system combines many well-known methods of recognition with some own ones /a fast template method, a quasi­ parallel edge-finder etc./ but this combination provides a sophisticated goal-oriented approach for real-life problem solving.

2. PICTURE PROCESSING, RECOGNITION

The system is based on man-machine com­ munication; the goal is the recognition of 3D industrial objects whose number is limited to 10-20 for each task. The objects are taught either by building a geometric model or by a computer-aided transformation of the real situations viewed during the learning period to create a standard model description si­ milar to the artificial one. The camera input /Fig. 1/ is processed in overlap­ ping pixels by a simplified version of the Hueckel-operator using only two li­ near templates. An estimator of the fit coordinates a weight-attribute to the stroke. These weights and some fuzzy­ like parameters which are attributed to the higher picture-hierarchy are rele­ vant elements of the heuristic search. The hairy stroke-picture /Fig. 2/ is used both by the brute-force shape estimators which provide a first guess and, by the further picture processing.Nodes of the contour are supposed where the slope-variances of the strokes attain local maxima /Fig. 3/. Streaks along the contour lines are detected by a quasiparallel minimum-cost algorithm oper­ a ting in parallel between every sup­ posed node-pair connecte_d by paths.

This algorithm is based on a mathemati­ cal mode l of the expected streaks. The requirements with the paths are ex­ pressed by four graph-theoretic crite­ ria which mean that every contour-line should be optimally approximated by a s treak and each of them by only one streak. A potential-field growing algo­ rithm has been developed in order to find the streaks and it can be proved that the streaks prov ided by this algo­ rithm satisfy the criteria of the math­ ematica l model of the streaks. The streaks found using this algorithm are disp l ayed on Fig. 3.

The next preprocessing algorithm merges the streaks either into line segments by uniting the approximately identically directed consecutive strokes or into arcs by comparing the slope differences . This interpretation is generally weakt due to the uncertainty of the whole pro­ cess ; thus a probability value is as­ signed to each poss ible interpretation, consider ing that each streak may have various interpretations /Fig . 4/. This value serves for the ordering of hypoth­ eses in the linguist ic recognition.

Matching the processed picture with the model a nd the who le knowledge base, de­ cision on the ambigu ities due to multi- I ple or failing edges and areas are solved by linguistic methods. Three ac­ celerating heuristic tools are applied in directing the search on the problem­ graph~ a priori information, the wcights with the calculated fuzzy-like estima­ tion attributes of the picture hierarchy and the pistance measures between pic­ tures and models. The estimation attrib­ utes are calculated from the weights of the strokes, considering in each stage /node, streak, edge, picture part/ the reliability of the fit, decomposition and concatenatio~. Several distance measures are distinguished~ brute-force characteristics with their special task­ oriented combinations, distances similar to those used by Fu I 3 I , distances ob­ tained from statistics of learning, spe­ cial measures preprogrammed by the oper­ ator with the help of his personal expe­ rience.

The process of recognition includes the determination of orientation , namely the calculation of the transformation matrix which transforms the position of the model to the real positioR of the object. Model lfng is used not only in the ways initiated by Rober ts and later Baumgart and others but as a general tool for man-machine communication during i/ the system de s ign, ii/ the task learning and iii/ the robot operation . Each l eve l is different in inte lligence ; none o f the levels can disturb its superior one but each has a systematic downward compati­ ble interactive apparatus based on a graphic d isplay. The graphics is 30-20 oriented: the internal representation and manipulations re f er most ly to the 3D model /Fig . 5/, the different 20 pictures to be matched with the scene are derived from that using a fast hidden line elimination algorithm. This means that all usual 3D operat i ons are imple­ mented / r o tation , distortion , unifica­ tion etc./ , adapted specially to our requirements, i. e. r eal time, low storage capacity .

3 . EXPERIMENTAL RESULTS

Robo t c ontrol and hardware are usual not to be detailed here . The system use s a 16 bit 40 kB mini with a 2 us cycle-time /VIDEOTON R-10 licence aft~r the French CII- Mitra-15/ , minidisc , magtape, graphic display.

The who l e recognition procedure needs about 30-50 seconds to recognize an objec t shown e. g. in Fig. 1. The run­ ning times of the particular algorithms are the following. The hidden line algo­ r ithm is extremely f as t, it provided e. g . Fig . 6 within 2 seconds. The edge ­ f inding oper ator needs about 4 seconds for processing a TV picture input. The algorithm which assembles the strokes int o streaks requires 10~15 seconds. The grammatica l search ing procedure of find­ ing the possible faces and the possible ob ject drawings in the streaks' inter­ p retatio ns takes 5-8 seconds. The other on-line algorithms take less than 1 or 2 seconds altogether.

Experiments have proved t ha t the indi­ cated heuristic s earch results in a time requirement depending statistically li­ nearly on the ob ject'complexity, number of objects, number of ambiguities /edge~

interpretations , s hadows , partial obser­ vations/. A successful guess /e . g. a brute-force estimator/ can cut short occasionally all further trials. In u n­ fortunate cases the interaction of the operator is als o possible.

4. APPLICATIONS

The system is primarily assembly-orient­ ed. Other applications o f our r esults that already work are being realized re­ cently~ a .Tecr;>.gni tion system in a bus­ body factory identifying the meta l sheets for contro lling a painter robot and a man-machine system in neuro-biolo­ gical research which should detect neu­ ronal nets in microscope-sections.