Between Regions and Objects - Surfaces and Volumes

Introduction

There are three major types of processing which lead to model construction at the region, surface, volume, object and frame levels: (i) curve fitting of region boundaries and analysis of contours in terms of primitive 2D shapes, (ii) hypothesizing surfaces and volumes using the 2D shape descriptions, the results of perspective, shadow, and occlusion analyses, and additional semantic information, and (iii) object verification using a spatial processor which matches 2D pro­ jections generated from a 3D description of the object against regions and contours from the image [Marr and Nishihara, 1976).

Representation

Objects in the data base are described in terms of primitive volumes. A generalized cylinder description is maintained in object-centered coordinates for each of the component volumes of an object. We are currently investigating the use of Binford's full generalized cylinder description [Agin, 1972; Marr and Nishihara, 1976). This description allows cross sections which are orient­ able in the plane (such as triangles, rectangles, and ellipses). Relationships between the primitive volumes, surfaces, contours, and regions are explicitly stored in the data base of long term knowledge. For example, the fact that a cylinder may project as a circle when viewed on end or as a rectangle when viewed from the side is explicitly stored in the knowledge base. These relationships provide some of the pathways between two and three

dimensions.

In addition to an object-centered coordinate system, we need to represent partial three-dimen­ sional information available from image data in a viewer-centered coordinate system [Duda and Hart, 1973]. A representation has been developed which allows a simple encoding of hypotheses about viewed object plane angles [Konolige, 1977). This representa­ tion provides a simple relationship between vanishing points and object plane angles, and a separate encodinp;

1 This work was supported in part by ONR grant

N00014-75-C-0459 and NSF grant DCR75-16098.

2 A. Hanson is affiliated with Hampshire College,

Amherst, Massachusetts 01002.

information available at that point and the reli­ ability of that information. One such strategy, which incorporates both data-directed hypotheses and the use of semantic clues, is given below.

Example Strategy

Often regions representing the ground and sky planes can be inferred from color characteristics and knowledge of where they appear in natural scenes. Contour analysis can find the strong straight lines in the image. These could be used by the perspective KS to fix the horizon of the ground plane, and by the shape KS to recognize the triangle and square. Knowledge of the 2D projec­ tion of 3D shapes suggest either a pyramid or a wedge for the image triangle; a wedge is hypothe­ sized if the roof line of the house is found. Then the spatial processor would orient the wedge to give the correct projection. At this point the shape KS could index into its data base of objects to suggest a house roof for the wedge; then a range for the distance of the house could be estimated from stored house size ranges. Finally, since the ground line of the house is above the hypothesized horizon, an upward ground slope to the house can be postulated (which in this case is a correct hypothesis) and computed using the estimated house distance. The desired results of this set of analyses are illustrated in Figure 2.