Analysis Of Three-Dimensional Scene Images On The Knowledge Of Their Texture Features

Introduction

In recent papers on analysis of three­ dimensional scenes toe initial half-tone image is regarded as terturelsee e.g.1,2~ This 1s attributable to the fact that toe objects which the robot is to manipUlate may look like "spatial texture", {e.g. threading~ on a bolt or nut,an assembly with a large number of ia.entical !>arts). Also with toe resolution of the perceptin unit sufficiently high,tne aicro-pattern or roughness of toe material of which the are made may be mapped onto the image.The properties of the image texture depend on the object surfaces orientation relative to the perceptor axis of vision as well as on the material and the "spatial tex­ ture".Therefore different surfaces of the object are associated with more or less uniform areas with different structural properties in the image.

This paper will cover experimental study of the algorithm for identification of areas of uniform texture in a half-to­ ne image of a three-dimensional scene.One specific feature of the study is that the initial material of the experiments were images of real objects that were prepared by an "eye-computer"system.Another was that the ;area identification algorithm uses analysi~ of integral texture charac­ teristics on large areas of image.The image is first divided into fragments each treated as a separate texture then each fragment associated with a vector, which is a set of texture integral charac teristics.Automatic classification algo­ rithm divide the entire set of vectors into classes.As a result each fragment of the image is associated with the score of the class or the texture type index. A totality of neighboring fragments with the same indices is . regarded as an area of the same texture.

The paper consists of two sectios.The first will describe the integral texture characteristics used and the procedure for area identification.The second is devoted to deccription of experiments. confirming the validity of the method.

The experiments were conducted in col­ laboration with G.G.Vainshtein using an "eye-computer"system developed in the laborattory of D.S.Lebedev in the Insti­ tute of InformationTransmission Problems Academy of Scienee USSR.The system is described in (;).

Area identification algorithms Integral texture characteristics

Intuitively a texture image can be described in terms of the size and orien­ tatio~ of the grain.These features,how­ ever,have a sense only for binary images where one can identify "repeating"elemen­ ts of approximately similar shape,as do­ ne in (1 ).

We will propose a system of features intended for direct work with half-tone images.Following the above intuitive con cept of describing texture images the de­ sired features should be such that give good assessment of grains and their pe­ riod and direction of distribution in the textures to be analyzed.

A more straightforward definition of these features is obtained in using functions which describe the change of a certain measure of similarity between the image under study and the image obtaned from it by shifts depending on the step and direction of the shift.Indeed,the smaller, on the average, is the "repeating period"of the image in a certane direc­ tion,the stronger the decrease in simila­ rity as a fUnction of the shift.Consequen tly, the functions which describe the chan ge of the similarity measure with a , shift in different directions carry suf­ ficiently complete information on basic characteristics of half-tone images.

Assume that an analised image is spe­ cified on an MxN digital rastre.Consider a central M'xN' area which takes up al­ most the entire image.Compare it with an area of similar size which is shifted with respect to the rastre center by a distance t' in direction cp · The simila­ r! ty measure is a squared Eucledean dis­ tance between vectors of blackness distri­ butions of associated images specified on M' x N' rastre.

The system of feasible shift direc­ tions will be specified as a system of four directions:rightwards,rig~t and up­ wards , upwards, left and upwards. · Enumerate these directions as 1, 2, 3, 4. In each of these directions we will consi­ der a set of different shifts ~= 1, 2, ··· , n where n is small compa­

' red with min M ,N'.

· If necessary the number of shift direc­ tions can be easily increased. A crude four-direction system suffices, however, in many practical cases. Also, small shifts often give the same effect in simmetrical directions. This means that instead of two opposite directions only one can be studied.

Denote the blackening vector of the cen­ tral area as x ~d that of the shifted area as x (1(, f/1 ). The totality of values R!(:x, x (1r, 'f)- of squared Eucledean distances between the vector x and all vectors x ('t", 'f ) defined for rp = 1 ,2,3, 4 and 1r=1, ··· , n specifies a new vector xR. This new vector is the desired vec­ tor of the mainspace of features.

Note that characteristics of the half-tone image essentially depend on the contrast. Therefore identical initi­ al textures can be characterized by dif­ ferent vectors xa if they differ in the contrast. Therefore we propose that insted of using directly the vector xR one should use the normalized vector ~ obtained by the formula

XR = xR"«fa,

where dz is the variance of blackness distribution for an image.

Area identification procedure.

The image of a three dimensional scene to be analyzed was obtained througb an "eye-computer" system whose programs are described in ( 3 ) · This progr8JIS handle automatic focusillg and image input into the digital computer.

By a special program all images were divided into a totality of equally spa­ ced fragments. For each fragment the values of vector iR components were computed. The totality of these vectors was proces~ed through one of the auto­ matic classification algorithms descri­ bed in ( 4) · The processing resulted in an image in which the center of each fragment was marked by the symbol of the class with which the fragment was iden­

tified.

Area identification experiments

Figure I shows photographs of objects on which experiments were staged. Each such object or a totality of objects were placed in front of an "eye - computer" TV camera. Following automatic focusing the initial scene was introduced into the computer memory as an image specified on a 128 x 128 rastre with 64 blackness le­ vels. This image was observed on the display. Photographs of scenes shown on the display are given in Fig.2. Note that even where the object surfaces had a two-gade pattern (Fig. Ia, b) the image introduce~ into the computer was half­ tone (Fig. 2a, b) due to both the opti­ cal properties of the camera and the ac­ tion of automatic focusing.

The image was divided into 77 32 x 20 fragments. '.rhe centers of the frag­ men ~ s were spaced by IO rastre cells ho­ rizontally and I6 cells vertically. The windows which cut the fragments were 50% overlapping both horizontally and vertically. The values of the ~R vector components were computed for each frag­ ment on a Z9 x 17 rastre for four shift directions. In each direction the shift was by I, 2 or 3 cells. Consequently, each fragment was characterized by a 12-dimensional vector. To reduce the time for processing the values of xR the com­ putation was performed for roo points randomly selected within that rastre rather than for all points of the frag­ ments rastre. These points were different for various fragments. As special expe­ riments have shown, this reduction did not lead to substantial differences in classification.

The totality of all vectors obtained fro~ one image was divided into 4-8 clas­ ses.

The correctness of the processing results was estimated visually in terms of correspondence areas identified and the surfaces of objects.

Resylts

Fig.3 shows the results ot marking the images of scenes (see Fig.2) by symbols associated with the scores of classes.

In the image of two cubes whose surfaCes are patterned by lines and aquare spots (Fig.2a, b) all th~ thre~ sur­ faces were identified (Fig.3a,b). The processing results have a number of in­ teresting features. In each case the fragments on the meeting point of iden­ tified areas where picked out. These fragments (a fragment of the first class in Fig.3a and three fragments of the eighth class, Fig.3b) are in the vicini­ ty of the cube apex. Almost all frag­ ments of the eighth class in Fig. 3b incorporate a boundary of areas of dif­ ferent texture.

In the image of a pin with a nut screwed on it an area was identified which is associated with the thread surface (fragments of the third class in Fig.3c). Besides, background areas (fragments of the fourth class) could not be separated because the texture of these surfaces is too fine for the ca­ mera resolution. They can be separated by analyzing the blacknes intensity. Ho­ wever, as seen from the formula for com­ puting the vector xR its components do not change with the fragJtent mean black­ ness intensity.

In the image of a stockpile of similar nuts (Fig. 3d) an area associated with the internal thread surface (frag­ ments of the third class) and an area assotiated with the external surface (fragments of the fourth class) were identified. The background areas covered too small an area in the image. There­ fore in this case it is difficult to say how successfully the background areas (fragments of the first and second class) were identified.

In the image of the scene of Fig.1e details of both types (in Fig.3e frag­ ments of the first and second class res­ pectively) were identified. Fragments of other classes contain either parts of boundaries between the objects and the background or small partions of the background.

This leads to tbe .conclusion that the described procedure can be used as one of the algorithmical tools for design of a robot systems analyzing real three­ dimensional scenes ·