Color Picture Processing By Computer

1. Introduction

Picture processing by computer has been of considerable interest .in computer science and robotics research. Most efforts in pic­ ture processing have been concerned with monochromatic pictures. But, as everyone realizes, actual scenes and pictures all contain color, and color information plays important roles in human perception and recognition of scenes and pictures. This suggests the usefulness of color inforoa­ tion in picture processing by computer, In Ito(l973) we have discussed some basic problems and a conceptual frame,mrk for color picture processing by computer. We stated there the need of an experimental computer system for color picture process­ ing and emphasized the importance of "com­ puter-oriented chromatics".

This paper contains an overall description

COLOR PICTURE PROCESSING BY COMPUTER

Takayasu Ito Central Research Laboratory Mitsubishi Electric ~orporation Ama~asaki-City, JAPAN

of our cocputer system for color picture processing, the principal idea of color analysis, the use of color information to picture processing and some aspects of their applications.

2. A Computer System for Color P!.c'ture Processing

A computer system for color picture pro­ cessing should be equipped to facilitate computer acquisition of color pictures, computer analysis of color information, feature extraction by effective use of color information, structural analysis of color pictures and interactive processing complex color scenes and pictures.

In developing our system ,.,.e initially en­ countered the lack of efficient real-time input devices of color pictures · . Thus we started froc constructing efficient real­ time input devices which include a high­ speed multi-head color film reader and a digital color TV scanner. In this section \te introduce our computer system: for color picture processing with several color pic­ ture input/output devices.

2,1 A Computer System for Color Picture Processing

Our computer system for color picture pro­ cessing consists of a computer complex of MELCOM7500 and PDP-11 and several I/0 de­ vices. A set of interactive color software are available on PDP-11, and f.lELCOl17500 enables us to use a picture compiler, pic­ ture preprocessine routines, list process­ ing languaee LISP/M, SL deduction theorem prover .·

Figure I shows the main body of our com­ puter system for color picture processing , which works as a computer complex under a control of network handler.

2.2 Picture I/O Devices and their features

As is shown in Fieure 1 our system has several picture input/output devices in order to achieve an interactive picture processing capability. Among them the foll­ owing devices are principal tools of our picture processing system:

1. Multi-head color film re,derf
2. Digital color TV scanner
3. Pseudo-color picture displal
4. Gray-scale film writer

\-There the devices with #-mark are made by us.

2,2,1 Multi-head color film reader Various kinds of color film readers are possible in principle, but we have develop­ a high speed multi-head color filn reader devised by this author. Figure 2 shows its basic idea and mechanism.

Figure 3 shows an experimental Multi-head color film reader and pseudo-color disp­ lay unit installed in the Central Research Laboratory,Mitsubishi Electric Corporation

2.2.2 Di3itnl color TV scanner

The multi-head color film r eader can r ead filmed picture ~ata, so that it i s often ino.ppropriate as an on-line picture in,nut clevice. For such a s i tun.tion 1·re have deve­ l oped a dic i t uJ. color TV scrmner vi th the characteri s tics of Fip,ure 4 and Table 2.

~. Pseodo-color Picture Display and Gray-scale Film Writer

I ~ pseudo-color picture display and an input tablet nre used in order to achieve an e~~icient interactive processing. The display characteristics is listed in Table 3.

Table 4 sho1·rs the characteristics of the .~ray-scale film \·rri ter.

3. An Interactive Color Picture Analysis and Di splay System

Ho\·1 we proceed to expl ain the applications of our computer system to interactive picture proces sine ,color analysis and the use of color information to picture pro­ cessi ng , In thi s section ~~e introduce an interactive feature of our system,callcd "Color Pfcture Analys is and Display

System". The follm!ine devices are integ­ rated to form this system:

1. a mini-computer(PDP-11 or li!E.L..COM-70)
2. Multi-head color film reader/Di.gi tal color TV scanner
3. ( 3) Pseudo-color picture display with an input tablet
4. DISK and MT
5. Standard I/0 devices of the minicomputer.

This system is desiened to achieve an eff­ icient interactive processing of a consi­ derably laree amount of color pfcture data The functions o~ this system consists o~ (1) I/0 control proerams (2)pseudo-color display pro.grams (3) color analysis pro­ grams ( 4) data tran·sformation programs (5) conversational programs. The following are some o~ the system programs:

color analysrs

TRIT trianele representation o~ color

HSB compute Hue, Sa turation,Brigh tness

ADM Adams chromance speci~ication

color picture feature extraction

COI\IPD compute areas 1-1i th specified colors

CLASS

IIIST

CLRSP

classify by specified density ran~es

compute density hist.n; ram

color separation by hue

color picture monitorfng

4, Methods of Color Analysis and Specification

Our computer system for color picture pro­ cessins enables us to process color pict­ ures in an interactive manner. Some of elementary functions are already .mentioned in the preceding section, In this section we discuss more details of color analysis and specification, and the next section explains some aspects of effective use of color information in picture processing. Before ~oing onto computer processing of color pictures, 'lore need to analyse .and recognize color information obtained from color picture devices like the r.tulti-head color film reader and Digital color TV scanner,because the correct recognition of color is important in the effective use of color information to picture process1ng. A number of color analysis and specifica­ tion methods are knO\m in the classical theory of color science. These methods are also useful in our situation, but ve need some new considerations to adopt them, since they are hunan-oriented.We aim at computer-oriented chromatics based on our computer system and picture devices, but they are not--perhaps,should not be--diff­ erent conceptually from the classical one. ~ Basic Concepts of Color Analysis and

Specification by Computer The computer analysis and specification of color information may vary in each appli­ cation, But the foll0\Jin0 methods will be applicable as the standard procedures in various applications:

color analysis

• l)color comparison procedure This is the color identification method of comparing with the standard color chips. This method corresponds to the color appearance system.
• 2)chromaticity analysis procedure This is the ~ethod of analysing color by chromaticity diaerams.

color specification

• l)color quantities---hue,lightness,chroma
• 2)XYZ specification
• 3)Triangular· representation
• color band

5)Color difference chart All of these methods are some modified versions or e:=teneione of the classical methods in color science. Some features of these methods will be explained below. However the readers should notice that we are not concerned with strict and pre­ cise analysis of color,but we are interest­ ed in approximate and convenLmt methods which are useful in correct color recogni­ tion necessary for picture analysis and recognition. According to our experiences a single method is not sufficient for var­ ious applications, so that we listed a number of typical methods developed and used by us.

The filter characteristics and light source of Color Film Reader and Color TV Scanner are different from the standard ones in color ccience,..tlu.t i f ·.ve kno\·T the optical characteristics of the ~iven filters and the light source, we can derive the stand­ ard stimulus value. This is also possible in an approximate sense, since those chara­ cteristics is known approximately in our situation. Taking account of the theoret­ ical backgrounds from color sCience, our color analysis software is developed both on PDP-11 and MELCmi-7500.

4.2 Color Specification Iviethode We explain our color specification methode implenented in coJ.or analysis software. They c.re classified into the numerical spe­ cification and the diaeram specification. (i) numerical specification

In order to express the color information as numerical quan.tities,we use the follow­ ing quanti ties:

• a)color quanties

The hue,lightness and saturation are three standard color quant!ties.These quantites will be computed by the regu­ lar triangle for color specification. This is shown in Fieure 5 below.

• b)XYZ specification

In a rough approximation the optical density from picture devices may be used as a first approximation to XYZ values.

The precise values corresponding to CIE standard will be also computed, since \·Te know the CIE-XYZ char acteri stics of the Wratten filters.

( ii)dia:-; ram specification

The follo1'1ing dineram specifications are proposed and implenentcd in our color ana­ lys is soft\mre. They include some non-st­ andard ones.

• a)standard triangle s r ecification There are the re,~lar triangle method and the rieht-angl ed triangle method as in Figure 5. B(0,0.1)

H=ll

• Pi~re 5 Color triangle b)multi-triangle specification In color picture processing w~ often like to represent the lishtness in tri­ angular diagrams. For this purpose ,.,e propose the multi-triangular specifica­ tion as in Pigure 6.

Fi~re 6

Color

Multi-triangle

c)color band

The hue circle can be expanded into the color band,and it is ex tended as Hue hist ~ram of Fi ~ure 7.

Figure 7 Color band

d)color differnce chnrt

The color difference and its distributi on are good illeasures to know color structure and balance .Adams chro!nance space method seems to be very convenie~ for our pur~ose among various methods. Fi t>:ure 8 shovTS an example of Adams chro­ mance space method.

• e)dynamic projection of color space The diagram specification is intended to see the projection of color distri­ bution in color space. T~1e use of high performance ~ra~hic display will allow us to see the projection of color space dynamically.
• ~ Elenentary operations on color space (i) scalar operation Zxample: Y-aR+bGt cB
• (ii) color vector arithmetic Every color \'fill be represent ed as a vee-;. or in a color space, so t'1 at ,.,e can cons-;. ruct a system of color ari thmeti c.

.!ES;r: vector addition of 1f and r

! e ;r: vector subtraction of .! and .I

.!C9.I= vector multiplication of.! and I

,!~;y: vector division of ! and .I

,.,here ! and .I are color vectors .

5. Use of Color Information in Pfcture Processing

There are many possibilities for effective use of color information in picture pro­ cessing. They range over from simple cases to complex and difficult cases. Most re­ searchers ,,rould have interest in rather difficult and complex problems such as recognition of color aerial photoGraphs, recoeni tion of color block ~rorlds, identification of human faces,etc.Their develop­ ments are interesting, but they will not meet direct applications in an industrial situation according to the present state of picture processing technology. Here we introduce several simple but useful ideas A with direct applications.

A different application is the computer analysis of colored maps of land utility. Also the detection of color irregularity in color printing and color painting can be done as a simple application of color separation.

~Color Effect Method and its Application The color effect method is an effective method devised by this author. This method was originally proposed to detect some de­ fects of mask patterns by an effective application of colo~ information to picture recognition. Figure 10 is an example to explain this method.

2L! Color Separation and its Applications Using our color ~1ulysis software we can separate and identify colors,and this fun­ ction is often needed in various applications of color picture processing. The color separation itself finds several di­ rect applications. Fieure 9 shows an ex­ ample in the automation of color textile printing.

ori~inal color pattern AccordinG to this example the detection of pattern defects can be performed by detectinG so~e specified color information (vithout handling any complex picture). This method has found several interesting applications in computer checking of vari­ ous mask patterns~ solving registration problems by simple techniques.

dark brown

Fi.nure 10 An example of color effect method

This color effect method can be extended to multi-color effect cases. If ,.,e use three colors, we can expect color majority decision, \·rhich is explained in Figure 11. Pigure 12 explains the principle of color

Figure ~~Vector representation of color effect method

effect method by color vector analysis in the color space.

• ~ Conversational Retouching of Color Pictures

The conversational retouching of color is useful in picture reproduction and digital film writing by computer, since it elimin­ ates the wasteful process and feedback \·rhich require optical treatments. The foll­ owing functions are necessary to implement the conversational retouching system.

1. sharpness
2. compression
3. ( 3) gradation
4. masking
5. color removal
6. smoothing by interpolation or extrapolation
7. computation of color· density
8. color separation and composition
9. display of color picture
10. {10) picture editing
11. ~ Pseudo-color Displaying and Picture Measures

The pseudo-color display of pictures is successfully used in analysing remote sensing data and in designing structured patterns.The assignment of pseudo-color to each pixel and its evaluation are im­ portant in this case. There are many case studies in remote sensing. But no ~eneral procedure is known to find optimal pseudo­ color assignments. Also the complexity­ measures and some degree of picture beauty are necessary in color pattern design to evaluate picture quali ty.The degree of beauty B 0 in color science can be effect­ ively used from the standpoint of color harmony. This Bb can be defin ~~d as

B 0 =W/Q

\·rhere W is the degree of order and Q is the degree of complexity.This quantity can be computed by Moon-Spencer formula in color science.This kind of measures will be intersting from a theoretical standpoint. There are several possibilities to extend this idea, using information num­ bers and co~plexity measures in information theory and pattern recotjni tion. The follo'!oT­ ing are some of their examples:

1. color entropy based on color hist;ram
2. topoloeical number in color pictures
3. heuristic picture complexity measure
4. computational complexity oeasures
5. hl Domo.in Identirication by Color '£he domain identification is often import­ ant in scene ~nlysis. According to the suggestion of human perception the use of color information often helps to solve this problPm if the domains are identified by color~ The domain identification of color block rTOrlds are often difficult in gray scale treatment when the shado'" pro­ blem is comprised in a ~iven situatio~.The chromaticity analysis and the minimum dis­ tance clunterin,~ are useful for this pro­ blem. In Ficure 13 1o1e show a simple do!i!ain identification, based on the chromaticity analysis and its effective application.