The Use of Color in Highlight Identification

1 Introduction

The appearance of objects in the real world is related to the complex ways they reflect the light impinging on them from the light source present. Different materials reflect light in various ways, but one thing common to most mate­ rials is the fact that they are neither totally matte nor to­ tally mirror-like; it is a combination of these two modes of reflection that governs the appearance of most objects. In short, it is the sum of diffuse and specular reflections that determines the total reflection of light off surfaces. Based on this statement, we observe that a shift in color occurs under some conditions, which leads to the development of an algorithm designed to identify the color changes which are due to specular reflection.

The motivation behind the quest for highlights origi­ nates in two opposing properties highlights possess. On one hand, highlights in images provide important infor­ mation which can be used to determine shape, size, or curvature of objects {1,2,3,9). On the other hand, high­ lights cause numerous problems for algorithms which re­ quire correspondence between consecutive images, such as stereo or motion algorithms {6,8).

2 The Shift in Color

Earlier studies into models of reflection pointed out that the total reflection of light striking an object is actually a sum of two components: specular and diffuse [4,7). This combination determines the appearance of the object ev­ erywhere, be it shadow, highlight, or any other region. The most intriguing aspect of this statement is the fact that specularity is an additive process; that is, the regions which appear glossy have the color of the illuminant added to the diffuse color of the surface for nonmetallic objects. The reason for this transition is a change in the weights (magnitudes) assigned to the diffuse and specular reflec­ tions. A realistic formulation of the physical processes a shift of the C-values towards the same direction. In terms of the C-space, this direction is actually pointing towards the values of a perfect reflector, since highlights are supposedly a realization of mirror-like surfaces. This observation was analyzed numerically in the following way. C-values of a perfect reflector were generated under the ?-Bs~m~tion t~at a perfect reflector would reflect the light tmpmgmg on It at all wavelengths. The resulting C-values for a perfect reflector in wavelengths A E (400, 650] were: Ct = 1.340, C2 = -0.176, and C3 = -0.009. This claim is illustrated in Figure 1 with a (dashed) line segment start­ ing at the origin and ending at the C-values of a perfect reflector, as given above. This line segment represents the C-values which reflect light equally at all wavelengths, for reflectances from 0 to 100%. We can see that one of the legs points towards the end of this line segment.

· Aleo, Canadian Institute for Advanced Research .

3 The Algorithm

The algorithm designed to detect possible highlights is based on the observations made in the previous section. It is a two step algorithm: first, it requires the segmentation of the image into regions, and compilation of information about regions and neighbors, and secondly, it attempts to find a "dog leg" relationship between every adjacent pair of regions. The major steps of the algorithm are:

• Chromatic-based segmentation of the [R,G,B] image and aggregation of small regions into larger ones.
• Transformation of the (R,G,B] values into C-values.
• Least squares approximation of the C-values in each region to a line in 3-d.

For each pair of adjacent regions:

• Verification whether the lines ("legs") are parallel - if so, a new pair of regions is tested.
• Computation of the point of intersection between the two lines and verification that the two lines in­ deed meet.
• Detection of the region in which most of the C1 val­ ues are beyond the point of intersection, and labeling it as a candidate highlight. If both regions pass the test - a new pair of regions is tested.
• Verification whether the line representing the candidate region is in the direction of a perfect reflector.

The algorithm was applied on the images presented in Plates 1 and 2, with the results of the regions considered as highlights depicted in Plates 3 and 4. In all the images, the highlight regions are coded in a grid pattern superimposed on the original image. Note that the shape of the regions does not precisely fit the shape of the highlights, due to the nature of the segmentation.

4 Discussion

One assumption which was not dealt with was the num­ ber of light sources and their spectral power distribution. Since the algorithm makes use of color constant descrip­ tors of the image, and thus discounts one spectral power distribution of illuminants, the same assumption regard­ ing the light source holds. Therefore the algorithm can handle more than one light source in the scene, as long as all the light sources share the same spectral power distri­ bution. The fact that there are no assumptions regarding the geometry of the objects allows the treatment of many light sources, since the algorithm is based on chromatic properties only. The scatter plot resulting from the in­ clusion of more light sources will not change since there will be no change in the chromaticity of the light sources, and consequently, in the colors of the diffuse and specular regions.

There remains one problem with the technique em­ ployed in this paper, namely, highlights on materials which share the same color as the light source. A simple exam­ ple can be a highlight caused by a white illuminant on a white object. The problem, of course, is that there will be no color shift, since the diffuse and specular reflections share the same spectral power distribution. This results in the fact that there will be no "dog leg" in C-space since the only difference between the two regions will be the strength of the values of C1· ·This problem remains open at this stage.

5 Summary

This paper discussed the problem of identifying probable highlights in real images. The solution we have proposed is based on the physics of interaction between light and materials, as formulated in models of optics and reflec­ tion. Based on these models, a prediction as to the be­ havior of highlights, or the specular reflection, was made, and was observed in real images. Based on the observed phenomenon, namely, the shift in the color of objects from the diffuse reflection to the the specular one, we have de­ veloped an algorithm designed to detect these shifts. The algorithm requires a segmentation of the image into re­ gions, so that each region can be related to its neighbors to verify whether they share the shift in color. Since the color shifts can be characterized by a "dog leg" behavior in color space, the algorithm was designed to detect such relationships between the regions, and augmented with rules that employ information about the expected color of highlights in the scene. The algorithm was tested with real images and produced good results in detecting the location of probable highlights.