An Automatic Position Recognition Technique for LSI Assembly

1. Introduction

Recognition of object position and posture has been recognized as a vital technique in the machine perception field. Many studies have been conducted to date in this field and have led to the development of practical techniques such as bolt-tightening robots(!), printed-circuit board drilling machines(2), wheel-fitting machines(3), as well as others(4)(5}.

One critical area in which machine percep­ tion research is expected to prove especially valuable is automated semiconductor assembly. This is because semiconductor devices have gotten steadily more sophisticated and have been miniaturized to the point where human operation is becoming increasingly problematic. Along this line, success has already been realized in developing a new transistor assembly system(6), the first practical application of a group­ controlled vision system.

This paper deals with an approach to the assembly of other semiconductor devices, especially LSI chips. It has been developed in parallel with the transistor assembly system by the same machine perception group. To date, almost all LSI assembly processes have already been automated, the notable exception being the position measuring station for wire-bonding. The position recognition technique in this study i s designed to rectify this situation, opening the way for fully automated LSI wire-bonding systems and, consequently, more reliable LSI

2. Outline of LSI Wire-bondi~

An example of an LSI lead-frame prior to assembly is shown in Figure l(a). The close-up photograph in Figure l(b) shows bonding pads distributed in the peripheral portion of the LSI chip. The wire-bonding process is where gold or aluminum wires are bonded and stretched between the bonding pads on the chip and the outer leads on the lead-frame. This process is carried out by numerically-controlled high-speed wire-bonding machines at a speed of 0.2 seconds / wire by using previously-measured positions of bonding pads. However, the measurement is still made by operators; either by manually pointing out a typical corner of the chip on a projected image screen or by manually aligning the chip image with the two cross-lines on a magnified view.

The posi t ,ion· recognition technique developed here uses two TV cameras to scan images of two different peripheral portions of a single LSI chip fed automatically into the measuring station. The position of a bonding pad in each field of vision is then determined approximately by a special image processing hardware and a minicomputer. The computer then calculates positional data of all bonding pads and records them on magnetic cassette tape for subsequent bonding.

(b) LSI chip ( Bonding-pad are ) distributed in the peripheral portion of the chip.

Production Engineering Research Laboratory, Hitachi Ltd.

Initial positional errors, due mainly to previous die- bonding processes, of ±l50 ~m in the X and Y directions and I30' in an angular direction are inevitable and must be tolerated at the measuring station. The position recogni ­ tion technique must compensate for the above loading error. It must also have a position finding accuracy of less than ±lO ~m to ensure reliable wiring at all bonding pads. Further­ mo r e, the recognition time and recognition rate should be less than 0 . 3 seconds/chip and more than 99.9 %, respectively, to accommodate various production condition .

3. Position Recognition Method

The image input system for LSI chip position recogni tion is shown in Figure 2. In order to assure adequate resolution, two TV cameras and microscope lenses were installed above the LSI chip, which was illuminated by a l amp . Shading and geometric distortions were a l most completely eliminated by compensation circuitry. The size of each visual field was set to 800 ~m horizontally and 600 ~m vertically. For our purposes, the size of an LSI chip ranged from 3 to 6 mm square . To accommodate various LSI types and sizes, two independent XY tables were actuated to adjust the camera positions and distances whenever product changeovers, e.g. from 3 to 6 mm chips, become necessary.

In the process, a frame-feeder feeds LSI chips one by one, into the system's field of v1s1on. An analog video signal containing target pad information can be obtained from each TV camera. The video signal from each TV camera is sampled and the effective scanning field is divided into 320 x 240 picture elements . Thus, one picture element corresponds to 2.5~m square. The image processing technique adopted here has two major functions.

One stably thresholds the analog video signal into a binary image. This thresholding is particularly important in practica l image processing . This is because a very simple and high-speed image processor can be realized if stable binary video signals can be assured in spite of a wide r ange of illumination inten­ sities, reflection ratio changes for different chips and other variations.

The other function detects the position of the bonding pad in the image with high accuracy and re l iability. This position detecting algorithm consists of two successive steps i . e. (l) macroscopic and (2) microscopic . I n the first step, the positions of the bonding pads are detected approximately by using 80 x 60 picture elements sampled in the X and Y directions . More accurate positioning is accomplished in the second step with the or iginal picture element resolution . Finally, a geometric check is performed for the posi ­ tional data of an arbitrary pair of pads between the visual fields. When a reasonable pair is found, all other pad positions are calculated from known geometric relations of the pads in the chip.

4 . Thresho l ding Algor ithm

Images of an LSI chip from the TV camera are show~ in Figure 3. In general, the analog (gray- scale) image shown in Figure 3(a) has four major brightness levels whose absolute values differ from one chip to another and have over­ lapping brightness distribution ranges . The brightest part corresponds to a l uminum, and the square- shaped po r tion at the image center of Figure 3(a) is a bonding pad . The second brightest par t in the image is Si, the third is Si02 and the fourth is Au-Si. The problem is to discriminate the bonding pad from the surrounding Si0 2 .

(b) Binar y i mage

Fig , 3 Images of peripheral part of LSI chi p .

The binary image obtained by the "ba l anced threshold method", developed here, is shown in Figure 3(b) . The a luminum bonding pad and other parts are discriminated clearly in the figure.

The "balanced threshold method" determines a thr eshold by successive approximations . First, an arbitral .threshold level is chosen in advance, then the average values higher and lower than the threshold l evel are calculated . Secondly, utilizing these average values, the threshold level is modified to a new one. This process is continued N times until an approximate equilib­ rium threshold level is attained; where N is a constant. This process is expressed as the following iteration equations:

For n=0,1,2, ... ,N,

where f(t) is the analog video signal from a TV camera, B'"'is the n-th threshold level, !if~"' and li f ~"' are the n-th average values higher and lower than 8"' 1 , fJ and 1- fi are constants that relate to the weight of L!f,:;'' and Llft>, and T ~·' and T ~"' are time periods where the image signal is greater and less than e'~'.

This algorithm can be explained more concretely by using a simple signal model to illustrate the meaning of these equations. A signal model with only two brightness levels, f w and f 8 , whose time periods are T w and T6 respectively is shown in Figure 4. Note that those time intervals are closely related to areas of the scanned image. When f 8 '-e''~ f.., and when the second and third terms of Equation (1) are rewritten as:

Lltl" 1 for this particular signal model becomes:

Let 9* be the equilibrium threshold, that is, (}'"'so that ilti~o in Equation ( 5). Solving Equation (5),

If Li&'"'becomes comparatively small after several iterations, the threshold given by cEquation (1) approximates to the equilibrium threshold.

The relation between 8'%' and .1B'~'for this model is illustrated in Figure 5. Among the values Ll8w, J(} 8 , ;jf}~ and tJB; at the discontinuous points in the figure, the following equations hold:

Thus, the following relation can be obtained from Equation (7):

Therefore, O*is unique regardless of the initial threshold value, as shown in Figure 5.

However, the images of LSI chips are not as simple as the model because they usually include various brightness levels. Suppose that the image of an LSI chip involves m major brightness levels. Then it can be shown that, at most, m-1 equilibrium thresholds exist.

However, illumination and reflection are stable enough that the brightness of the aluminum is always between an upper bound S bound ~ . Thus the time periods T;' should be changed into T '"' and T '"' 5~f(t)~9'"J and ?~ fn)~ 6l '""':~ and a lower and T;"' , where respectively.

In this case, the video signal for an LSI chip can be approximated by the model in Figure 4. A stable binary video signal, which can cope with the various changes in practical production conditions e . g . illumination changes, reflection ratio changes for each LSI chip, sensitivity drift of the TV cameras , can be obtained by the above "balanced threshold method".

In order to apply the above algorithm to this system, Equations (2) and (3) are converted into sums :

where T '~' , T ~';' ar e the digitized time periods for~~ 1. ~ e'''and >? ~ f;~ ecn)' respectively. The numerators and denominators in Equations (9) and (10) ar e easi l y obtained by counting picture elements as image scanning proceeds. The division and multiplication steps in Equations (1), (9) and (10) are executed by software within the blanking period.

An actual example of the relation between ()'.,.,'and LIB'"'' for an LSI chip is shown in Figure 6 . Here, the video signal is quantized into 64 l eve ls and typical parameters for this case are shown.

This thresholding method is essentially equivalent to the method which utilizes the histogram(7) of the LSI chip brightness distri­ bution. However, the processing speed is much faster than that of the histogram method .

5. Position Detection Algorithm

The bonding pad of an LSI chip is typically 120 ~m square. This corresponds to 48 x 48 picture elements in its image . Adjacent to the square portion, the pad has a protruding part on one side edge that joints to inner patterns . In addition the pad usually has a sc ratch on its surface, as shown in Figure 3, caused by the needle probe used in the previous wafer inspec ­ tion process.

The position detecting algorithm should accommodate these typical features of bonding pads and stably detect the position of the pads . The algorithm consists of two par ts: the macro­ scopic and microscopic processes . In the macro­ scopic process, the roughl y- sampled image is used , and a pattern matching method is applied . In the microscopic, several r ectangul ar domains are gene r ated, and the white areas in the domains are calculated to determine the position more accurately.

5.1 Macroscopic processing

In the macroscopic, the binary video signal of an LSI chip is sampled every four elements to yield an 80 x 60 e l ement image . In t his case, a pad corresponds to a 12 x 12 element . A pr e­ processing noise-elimination step is performed on this roughly-sampled image. A template for this procedure is illustrated in Figure 7. When the number of white picture elements among P0 , P 1 , P 2 , ··· , P 8 is l ess than four, P o is regarded as black, otherwise it is regarded as white. Thus a simpl e binary image is obtained in which uneven boundaries and isolated noises are eliminated .

Next, the position of the bonding pad is detected by a pattern matching method, as illustrated in Figure 8. Four templates A 1 , A 2 , A 3 and A 4 are scanned in parallel over the input image pattern to determine whether the input pattern is matched. The size of each template is 5 x 5 picture elements. Its standard pattern is equal to each corner of the bonding pad to be detected. Let Uij be a standard pattern, Dij be a "don't-care" pattern and Iij be an input pattern corresponding to a domain Ak (where k=l, ... ,4). The matching result Mk is given as follows:

This equation means that the Mk in the template Ak is "l" when Uij matches Iij except for Dij="l", otherwise "0".

However, at the preceding wafer inspection stage every bonding pad is scratched by a needle probe. Consequently, the pattern matching does not always hold for all four corners. Thus, regarding logic variable Mk numerically, the following numerical sum M is calculated:

If M ~ 3, the pattern is regarded as a bonding pad. Therefore, the pad is easily detected, even if one of the four coners lacks its normal shape as shown in Figure 8.

5.2 Microscopic processing

Strictly speaking, the macroscopic processing finds only pad candidates. This is because there can be other patterns in the image which resemble pads.

The first microscopic processing step checks and classifies the type of pads by investigating the joint terminal direction of the candidate for every picture element . In the process four rectangular domains are generated around the candidate, as illustrated in Figure 9(a). In the process, four rectangul~r domains are generated around the candidate. White areas S1 , S2, S3, and s 4 are then measured. In the example of the figure, the following conditions are checked:

where SB and Sw are appropriate threshold values. Only when relation (13) holds, the candidate is accepted as a bonding pad.

• (a) Direction finding of joint terminal of candidate.
• (b) Position modification of every corner.

Fig. 9 Microscopic processing.

The second step detects the position of the pad . For this purpose , sever a l minor r ectangul ar domains ar e set ar ound t he corner s as i llustrated i n Figure 9 (b), depending on the r esults of the previous step . Thus , the X and Y positions of the cor ne r s ar e dete rmined by calcul ating the white ar eas in the doma i ns at the r eso l ution of the original picture . For example, when area Sy is detected at a domain whose center position in the Y direction is YMas shown in Figure 10, the precise Y coordinate becomes :

Thus , t he position of the co r ners , except fo r any missed corners , ar e dete rmined .

Lastly, the center of the bonding pad is calculated . Here, the joint terminal and the position of t he sc r utch inside the pad ar e taken into consider ation. For example in Figur e 9 , the center coordinates (xc ,yc ) of the bonding pad are calcul ated as follows :

Her e (x 1 , y 1 ) and (x z , y 2 ) are coordinates of corne r s Q 1 , and Q 2 , and y f is the Y coordinate of corner Q4 in the figur e . The second term in Equat i on (15) means 1/ 2 of the pad size, in picture element.

6 . System Configuration and Resul ts

6.1 Recognition system

A position detect i on system based on these method has been constructed (see Figures 11, 12).

----- - ---- -----l

Vislon-8: ~~ P.SP

6'l0

It consists of four measuring stations involving image input TV cameras, a recognition device, a mini-computer HITAC-10 ll (16 bit, 16 KW) and cassette tape-recorders, as shown in Figure 12. A maximum of four measuring stations can be controlled in parallel by a single recognition device and the mini-computer .

In the recognition device, an integration circuit for thresholding calculates the four integrated values in the numerators and denominators in Equations (9) ann (10). This circuit basically consists of an AD converter and four counter s . The noise elimination circuit and the pattern matching circuit are the macroscopic processing circuits, and send positions of pad candidates to the computer. In the pattern matching circuit, several line memories are used to pick up a local image in the real-time mode.

In accordance with the positional data of the candidates, the computer sends several sets of rectangular domain positions to the two groups of white area counters which do the microscopic processing. The counted data are sent back to the computer, which checks the terminal direction and .calculates the position of the bonding pad. These information transfers are executed by the interface circuit using Direct Memory Access (DMA) as well as multi-level interrupting functions. The selector is used to determine the appropriate measuring stations, TV cameras, and image processing circuits depending on the processing sequence. Some of these are selected in parallel to enable simultaneous processing.

A time chart of the image processing is shown in Figure 13. As illustrated, three fields are devoted to thresholding. Pattern matching is done by one field. Direction finding of the joint terminal i s accomplished by one field. At the same time, position adjustment of each corner is effected by one field. One field is equal to 16.7 ms. Thus, total image processing time becomes 133 ms. In addition, however, there is some lag due to the diminishing after-image effect of the TV camera and other software computing time. Ther efore, overall position locating time becomes approximately 200 ms at maximum. This is well within the initial target time of 300 ms. The results of each image processing sequence is shown in Figure l4(a) to (d).

6.2 Test result

The r ecognition methods discussed here are evaluated from three points of view: (1) effec­ tiveness of the thresholding method, (2) recogni­ tion rate, and (3) positional accuracy.

1. In practical production lines, various disturbances must be considered, especially the effects of:
2. (a) intensity changes of the illumination,
3. (b) positional change in the image field, and
4. (c) lens focus.

The system was tested by changing illumination ± 15 % from its normal intensity. To det ermine positional dependence, position ar e a ltered t 150f-J-- m a l ong both X and Y directions. To investigate the effect of lens focus, the vertical position of the chip is varied ±1 20p.m along the opt i cal axis .

The r esults are shown in Figure 15. Each cr oss - sectioned r egion gives corr ect operation of the r ecognition a l gorithm as long as a thresho ld value in the r egion is adopted. Thus, each median axis of the region corresponds

(d) Calculation of

cent er of bonding pad.

Fig . 14 Example of each image processing .

(b) Positional change (c) Lens focus(~m) in image field (fm)

Fig. 15 Threshold change due to various disturbance factors.

to the threshold with the highest margin fo r pad r ecognit ion. As a r esult of the evaluation, the optimum thresholding parameters are follows: N=J, a=64, b=l5, when anal og video s i gnals a re quantized into 64 intensity levels. The optimum value of ft is determined to be 0.55 by finding the minimum standard deviation f r om the median axis . The relation between ft and the corresponding standard deviation is shown in Figure 16 .

1. After determining the thr esholding para­ meters, the recognition rate was measur ed for several types of LSI chips. The results shoWn in Table 1 demonstrate that the expected recognition rate of 99.9 % has been achieved. The single misrecognition was due to the defective chip whose large white domain happened to resemble a bonding pad.

,.....

1. For positional accuracy testing, three representative types of LSI chips were selected. The results are shown in Table 2. The recogni­ tion accuracy was ±5.9pm, well within the initial target of ± 10 ~m.

8. Conclusion

Position recognition methods for LSI chips were discussed, as applications of image processing techniques. The methods developed here depends on a stable thresholding algorithm and a two-step position finding algorithm. The recognition rate and accuracy for these methods were well within the target ranges. The comple­ tion of this system makes high-speed automatic assembly of LSI packages feasible, and avoids human errors encountered in previous positioning techniques.

9. Acknowledgment

The authors wish to thank Jun Suzuki, Department Manager of Equipment Developing Department in Semiconductor and Integrated Circuit Division, Hitachi Ltd., for his insights invaluable arrangements. Thanks are also due to Dr. Hiroshi Watanabe, General Manager, Dr. Jun Kawasaki, Department Manager, of Central Research Laboratory, Hitachi Ltd., as well as many others who contributed to the realization of this system, for their help and encouragement in carrying out this study.

10. References

• (l) T.Uno, M.Ejiri and T.Tokunaga: A method of real-time recognition of moving objects and its application, Pattern Recognition, Vol.8, pp.20l-208, 1976
• J.A.G.Hale and P.Saraga: Control of a PCB drilling machine by visual feedback, Proc. of 4th IJCAI, p.-775, Tbilisi, 1975
• J.T.Olsztyn et al.: An application of computer vision to a simulated assembly task, Proc. of lst IJCPR, Washington D.C., 1972
• P.Saraga and D.R.Skoyles: An experimental visually controlled pick and place machine for industry, Proc. of 3rd IJCPR, Coronado, 1976
• M.L.Baird: An application of computer vision to automated IC chip manufacture, Proc. of 3rd IJCPR, Coronado, 1976
• S.Kashioka, M.Ejiri and Y.Sakamoto: A transistor wire-bonding system utilizing multiple local pattern matching techniques, Trans. of IEEE on Systems, Man. and Cybernetics, Vol. SMC-6, No.8, pp.562-570, Aug. 1976
• A.Rosenfeld: Picture Processing by Computer, Academic Press, New York, 1969