Knowledge Sources In Vision

ABSTRACT

This film is a tutorial introduction to the problems of computer vision. It begins with a motivation of why knowledge Is important to vision. Knowledge sources that humans take for granted are shown to be critically impor­ tant in pur understanding of what we see . The film then explains the necessary elements in a vision system, be it computer vision or human vision. This is illustrated by showing humans who use these elements to interpret images. The end of the film gives some detail abC\ut the ARGOS image understanding system [Rubin, S.M., "Natural Scene Recognition Using Locus Search," CGJP 13, 1980). The following is a brief overview of the two main points covered by the film: the importance of knowledge sources and the basic elements of an image understanding system.

FH ·'~ : KNOlfLF:DCF. SOURCF.S TN VISION

Steven M. Rubin

Bell Laboratoriest Holmdel. New Jersey 07733

Information acquisition, or learning, is also important for computers. Automatic learning, where the computer builds its knowledge sources from training images, is not well understood. Often, the inaccuracy of the training exemplars misguides the knowledge sources and makes the recognition process fail. Supervised learning is the most accurate way to build visual knowledge . The very fact that a human must assist the machine in this step Indicates that we are not fully aware of our own visual knowledge sources.

KNOWlEDGE SOURCC3

The interpretation of images is not a simple process for machines or humans. We are not aware of the abundance of special visual cues that we use to rriake sense of images. Of course, obvious cues such as color and shape are important in distinguishing objects, but so are many more subtle cues. Each of these is a source of knowledge about the scene. For example, the shadows cast in a scene tell much about the three-dimensional structure. Shadows also indicate the position of light sources which can tell much about the con­ text of the scene such as location and time. Thus, It is very important to understand visual knowledge sources. They make the difference between a clear recognition of a scene and simple image processing .

IMAGE UNDKRsrANDING SYbiEMS

Ouc" kuowl"dg" sow·ce::; 1111ve ueeu luenli(t.,c.J. , ll 1::; necessary to assemble them into an Image recognition sys­ tem. Trte idenliJcation oi knowledge ~c,w· c es IS actually lhc tlrst of four steps that must be taken to achieve this goal. The other three steps are the determination of a proper representation for the knowledge, the learning of the knowledge, and the application of the knowledge in inter­ preting images .

Knowledge representation is very important to com­ puter image understanding systems. Humans do this sub­ consciously and the method is not understood. For comput­ ers , there are many alternative representations, most of which manifest themselves as a set or constraints that are applied to parts of an image. Each constraint guides the interpretation locally, and can be combined with all other constraints to produce a global analysis of the image .

t Author's current address: Fairchild, Palo Alto, Cali!omia 94304

The final step in building an image understanding sys­ tem is the actual use of the knowledge sources in recogniz­ ing images. Many techniques exist for application of knowledge sources. Often. the technique is dependent on the representation used. Ultimately, all true image under­ standing techniques must apply the knowledge sources in a global way. What this means is that knowledge applied to any part of the image helps in the recognition of other parts of the image . Without this global evaluation. the system is not recognizing the image. but simply processing it. one piece at a time.

The ability to apply many constraints to an image in a globally consistent manner requires a complex search tech­ nique that can find the proper constraints which give the desired interpretation. For this reason, the application of visual knowledge is viewed as an A.J. search task . Many heuristics are employed to reduce the search time since images tend to be highly complex and ditricult to explore . Tl.u::; IS c.J.u., ill JJI!.l'L ~o Lito: muii..l-Ullll"U"lOUI1illy of lh" SJ.glW.l. It is also due to the relatively small amount of knowledge that is typiCally brougt.t tn bea;- on the ;·e,·ogmtion. Smt.c most vision systems use a small fraction of the available knowledge, it is not surprising that the results are weak compared with other signal processing tasks (such as speech) where the signal semantics are better understood. Thus, it remains critically important to bring more knowledge to bear in the interpretation of images.

CONCWSJON

Much effort in computer vision is focused on the proper search techniques for analysis of images. There are also many quests after the proper representation of knowledge . The theme of this paper and the associated film is that knowledge sources are also important. Many of these sources of knowledge are not fully exploited in vision sys­ tems and this leads to weakness of recognition. A proper understanding of Visual knowledge will lead to representa­ tion and search techniques that will Improve computer interpretation of images.