Perceptual Problems in Analyzing Industrial Radiographs

PERCEPTUAL PROBLEMS IN ANALYZING INDUSTRIAL RADIOGRAPHS*

0. Firschein Lockheed Palo Alto Research Laboratory Palo Alto, CA 94304

M. A. Fischler SRI International

Menlo Park, CA 94025

• *

ABSTRACT

Given recent advances in image processing and analysis techniques, it seems that it should be within the state of the art to perform automatic analysis of industrial radiographs to locate defects in manufactured items. However, human inspectors are routinely able to detect certain subtle defects that are beyond our current capabil­ ities in image processing. This paper describes a specific radiographic inspection problem (analysis of the filler in an artillery shell), indicating the difficulties that arise in trying to detect subtle flaws, and especially flaws close to material boundaries. The relationships between human perception and the computer algorithms are indicated, and unexpected differences are pointed out (for example, the boundary determined by the human observer differs from the boundary position determined by most algorithms using a change of

intensity criteria).

INTRODUCTION

There are various methods of inspecting objects whose interiors cannot be seen directly, such as ultrasonic sensing or x-ray radiography. Such methods are used in medicine [1,7) and in industry. In industrial inspection, the images resulting from these techniques are inspected to locate voids, cracks, foreign body inclusions, separations, etc. Although one might expect that such inspection could be readily automated using present-day image analysis methods, there are certain classes of defects that, though readily seen by human inspectors, defy automation. The paper will focus on the results obtained in one particular study, the inspection of artillery shell

radiographs such as shown in Fig. 1 [2,3).

In the artillery shell inspection problem, we are concerned with the detection of defects in the explosive material used to fill the shell. This material is poured into the steel shell and, as it

solidifies various defects can arise:

* The original radiograph inspection study, AXIS, was performed by the Lockheed Palo Alto Research Laboratory for the U.S. Army Advanced Research and Development Command. The present paper represents

independent work carried out by the authors.

• cracks develop in the filler
• air bubbles cling to the interior of the shell, causing voids
• the filler may not flow into narrow portions of the shell, leaving voids
• the filler separates from the base of the shell as it hardens
• foreign bodies may have fallen into the filler material as· it was poured

Both gross and subtle defects must b~ found despite changes in radiographic film density due to the cylindrical shape of the shell and variations in imaging geometry. This is s non-trivial problem since the intensity difference between a defect and its surroundings is often of the same order as the geometry-caused intensity differences between pixels. In addition, one must deal with problems caused by internal physical structures,** and noise that arises in the radiographing and computer

scanning processes.

PROBLEMS AND PROCESSING STRATEGIES USED

We will first discuss flaws in open regions, a problem that can be dealt with using present-day processing techniques. Then we will describe the problem of finding boundaries, showing that the boundary found by the human observer differs from that found by the usual computer-based boundary finding algorithms. Finally, we will indicate two problems currently dealt with in only a limited manner by present-day techniques: flaws on a boundary, and flaws in narrow regions having boundaries very close by.

A. Flaws in Open Regions

Fig. 2 shows an intensity plot of a section (normal to the symmetry axis) of the radiograph of a shell. Note that because the shell is cylindri­ cal, we obtain an intensity curve that is parabolic in nature. Flaws show up as perturbations that "ride" on this trend line. A flaw that occurs in the "open", away from a boundary, can be dealt with using current image processing techniques. The intensity trend can be removed by fitting a polynomial to the inte~s 1 ity curve and subtracting the fitted curve from the actual intensity curve. Residual perturbations that remain after "field flattening" include the flaws that are to be detected.

**The problem of defects on or near internal structure also arises in medical radiography, e.g., tumors and lesions sometimes overlap the rib cage [4).

Some cautions must be observed:

• some way of ignoring large defects must be used to avoid the influence of the defect on the curve fit.
• the polynomial curve must not have its end points too close to a boundary, otherwise the steep intensity gradient will affect the fit.

The problem of the effect on the fit due to large defects can be handled by testing points against the fit obtained in the previous row to identify and eliminate from the curve-fitting procedure any deviant points. Another approach can be based on the RANSAC paradigm [5), in which random sets of points are used to obtain trial curves which are then tested for validity as models of the back­ ground intensity. Once the trend has been removed, low pass filtering followed by appropriate thresh­ olding (sometimes variable over the image) can be used to eliminate isolated noise pixels while retaining the candidate defect pixels. A final decision can be made by performing a connected component analysis on the candidate defect pixels followed by size and shape analysis of the indivi­ dual clusters of contiguous defect pixels.

Figure 3 shows the results obtained by processing a test image in accordance with the above approach.

B. Finding Boundaries

The shell boundaries are typically found (using computer based techniques) by examining the intensity curve for the maximum slope change. As will be explained below, this,method of finding the boundary will produce a different result than that obtained by the human visual system.

An informal series. of experiments were carried out to determine where the human observer sees a boundary in the shell inspection problem. Fig. 4 shows global and local intensity profiles normal to both the base and side wall boundaries of the shell; arrows indicate where the human observer sees the boundary. In spite of the minor varia­ tions from case to case, the basic pattern is one where the intensity gradient in the explosive charge region near the boundary is either constant or slightly increasing, followed by a rather sharp decrease or even gradient sign reversal as the shell casing region is entered. However, the visual boundary does not occur at the intensity gradient breakpoint, but rather is offset by a distance bf at least 2-4 pixels before the break­ point (in the direction of the explosive charge).

A number of our experiments demonstrated that different human observers will, with remarkable consistency (and in spite of the variations in boundary profile shape), position a cursor within one pixel on either side of their averagedresponse. Therefore, the boundary offset phenomenon is not due to experimental error in pointing to the visual boundary.

Though the objective was to devise algorithms which duplicated human performance, it would be interesting to know where the physical boundary appeared in the imagery. Unfortunately, this information was not provided with the radiographs, nor could it be easily determined due to uncontrolled variations in the relative positioning of the x-ray source, shell, and film.

C. Finding Defects on Boundaries

As indicated previously, because air bubbles can form on the boundary wall as the filler is poured, voids in the filler will occur there. This type of defect eludes present day processing techniques because the slight change in image density rides on an extremely steep intensity gradient due to the existence of the boundary. As explained earlier, fitting a polynomial to such steep gradients causes poor fit elsewhere on the curve. One approach that we have tried is to enhance the signal caused by the flaw by averaging along the flaw. In the case of a boundary separation, this direction is known because the flaw follows the boundary. Therefore, the intensity is averaged within 'layers' parallel to the boundary, as shown in Fig. 5. Averaging is carried out over a sector having a width comparable to the length of typical separation flaws. This process is conceptually similar to a curved moving window average whose height is one pixel and whose width is the sector width. In this sense, the operation is analogous to a low-pass filtering operation. The measured average intensities are fitted using a least-square, second degree polynomial. Points in the region where the separation can occur are not used in deriving the polynomial approximation; this prevents flaws from influencing the fit. The resultant curve for an image with boundary separation is shown as the dashed line in Fig. 6. A significant deviation between the actual and the fitted data signals the presence of a boundary separation.

We are able to detect some of the more prominent boundary defects with the above approach (especially when the defect is not flush against the boundary), but this non-adaptive averaging technique cannot come close to matching human performance. It would appear that part of the 'magic' of human vision is in knowing how to find the appropriate averaging windows in spite of the immense number of possible alternatives.

D. Finding Defects in Narrow Regions

Certain shells have an internal geometry that forms narrow regions of filler surrounded by boundary, as shown in Fig. 7. The procedure described above for separations cannot then be used since there is insufficient room to construct a grid parallel to the boundaries. There are too few points for intensity trend removal, and window operations cannot be carried out due to the space restrictions.

RELATION TO HUMAN VISUAL PERCEPTION

Many inspection problems involve the detection of an anomaly -- something that is 'different' from its background based on intensity level, shape, texture, or pattern. The human perceptual system seems to be able to analyze intensity gradients and can recognize the characteraistics of the background in order to carry out the inspection task very effectively. Specifically, in the shell i nspection problem, what stands out dramatically to the human eye as a defect often turns out to have no l a rger an intensity gradient than that of the i ntensity gradient due to the geometry of the s he l l. Thus, the eye is able to detect and suppress the global intensity trend due to geometry. Spot-type defects 160 mila in diameter* a nd cracks of 30-40 mils having a length of 1/2 inch can be readily dealt with, despite the fact that they represent leas than an 8 level intensity di fference from the local background in an x-ray f ilm with density that ranges from the lightest to darkes t regions in the ratio of 1:20,000 (i.e., 20, 000 intensity levels).

ln order to handle separations between the s hell wall and the filler, the human inspector must be proficient at knowing what a boundary is and in bei ng able to find it. While exact statistics are not available, it appears that human inspectors can detect a base separation defect having a width of only 5 mils if the length exceeds 1/4 to 1/2 inch.

In examining the question as to why the visual bounda ry is offset from the gradient boundary, the dat a shown in Fig. 8 may offer an explanation. If the eye sees a gradient which is averaged over some interval exceeding that due to the difference in in tensities between adjacent pixels, then the eye will not see a step function change in slope at a true break-point (betwen two regions having uniform, but distinct intensity gradients), but will see a smooth transition with the maxima and mini ma occuring displaced from the break-point by a few pixels, as is the case in Fig.8. Here, experiments show that the eye places the boundary a t t he local maxima adjacent to the break-point, even though there is another, slightly higher gradient maximum further into the explosive charge r egi on.

Other possible explanations for the visual boundary offset phenomenon are related to the visual broadening of the gradient transition region due to a Mach band effect (6], and to the contextual information the eye uses in locating the boundary which cannot be captured by the local i ntensity variations.

It is probably the case that the human i nspector brings to the problem much general perceptual knowledge, i.e., a lifetime of exposure t o va rious types of physical surfaces. Thus, the i nte r nal boundary of the shell has characteristics t hat t o the human, are distinguishable from those of a defect. When faced with the need to decide when there is a flaw on the suface of the boundary, the human can summon all of his previous experience with surfaces. This general perceptual experience is difficult to express in algorithmic form.

* To put these measurements into proper context, we digitized the film over a range of from 2-20 mi l s /pixel. Average film grain dimaeter was.2 mils .

DISCUSSION

Two pervasive problema in scene analysis arising in a radiographic inspection problem have been discussed: (1) finding edges or boundaries , and (2) finding objects that are anomalous with respect to their surround.

In an industrial context, locating edges is a critical first step for almost all mensuration and inspection tasks. The experimental results presented here demonstrate again that a simple edge model using only local information is inadequate to deal with precise delineation of real edges. Certainly, our computer models do not correspond in these situations to those invoked by humans engaged in the same tasks. It would appear that 'understanding edges' is a task that goes well beyond any local model we might construct; a sophisticated reasoning ability using global information is probably an esse'ntial ingrediant.

The second problem, finding small isolated regions of an image which are anomalous with respect to their surround, is often more difficult from a computational standpoint than from a conceptual one. The underlying requirement in many versions of this problem is that of selecting the appropriately shaped region of the image to compare with its neighborhood. However, when we are blocked by boundaries from introducing global comparisons, as in the case of the narrow region of thee shell with internal structure, then we cannot apply the currently available techniques.