An Application of the Photometric Stereo Method

AN AI'I'LICATION OF THE PHOTOMETRIC STEREO METHOD.

K<tt~u<lll lht'IICIII 'Arlllflcl;tl lutdlt;·;o.'IICC LaiJoratory Ma«.lciHI<crtl lmtuutc of Trchnology ~1'1'· Trrllnnlron ~q. C.1mhndg P. 1\ I~ ~<:Jcllusrm 021:l9

Ben hold 1\ . P. Horn tnirtfiCI~IIntclhgcnce Laboratory M;j ~~:J(hmrm lmtttutr of Trchnology ~1'15 To'rhnnlCl~Y Sq: Cambncl t:e. Massachmetts 02139

The idea of photnmrt11c ~trrro 1s to drtermine the wrface onrntation of the points of an object in a pacture using several views of an obJPCt t;jken with varymg ilhrminat1on, but from the samt viewing direction. We can implement this method using reflectance map techniques. In order to calculate the reflectance map, we consider primarily the specular component of surface reflectance, smce most industraal materials are very specular. The technique has been demonstrated by experiment.

0 Coudu\inn

The tPcllllii"Jilr nf photomrtric stereo has been proven by an expenme1it whme schema is shown in Frg. I. Orientations of surf:w:- patchrs on a sphere obtained using this technique are shown Ill Fig. 2.

Thr ~dtrllt;j k·; two p;nts of JObs. o11e IS a11 off-ltne JOb, 1 rpt r-<rut r rl hy In nkr11 lmes . Th" other JOb IS one-line. lllt.l) .. " htt~ hlnr<~ is nht .ltnrcl hy w.mg a TV camNa . We ;(' ; ,,~, h lor m.t:>-IIIHim v.tlur< of l·r· r.~ htnrss 111 rach array. l'.r1:.hll···ss .nr.tys ;tre ,1 lw ul1br.1trd and normalized . A lookup t<~IJir:o hH IJI'CII comtructed off -line from the trflr< t1urr nnp hy usmg Nrwton's method . The lookup tablr 1< u~rrl In nlot.1111 Sill f:~(r nrirnt:~twn .

lhi~. ,- ,p<:wt rlc:.crihes rc~earch done at the Art.ilici ,ll lrttell igence Lahoratory of the 11.1 ~. · .. 'JriiiJ' PIt <- In" I i tule of TechnoiO~I'.J· Support fnr 11,, . ( .J I· c·r.llor~t's ,cu-t ificial intelligence ,-,.".· ;,. rl· i·. prnvirlr-cl in pilrl hiJ the Advance Prn j<·c l A<t r·nc·.t of th~· Ocp.3rtmcnt of Or::fcnce under (lffr r:ro nf Ni JVdl He st'arch contract N00014-75-C­ fjf.l, '.! .

I lh·flectancl' Map and Photo-Mrtric-Strreo

Wc 011 rxprn~ the geometric dependence of the reflectance ch:n~ctcn.<ttrs of a snrf~ce 111 terms of the slope components p and q. 11srcl a~ ~xr~ in grad1rnt spacr [1).

If w·: t~kr the dHrctlon from the surface to the v1ewer as the ditdtton of thr 1-ax1~. thr:n the r<:'fkctancl! properties of a surf.1cr p~rrh drprnci on ( p.q ), thr rlH<:'rtion of the surface norm~ I <tnd ( I'!··CJs ). the clnrrllon to thr sourer [1). Each p0111t ill e:radiCIIt sp:tce, COt'l'elpOildS tO il partiCUlar SUrface onc11tat1on (b.unl 011 the direction of the vacwer) .

If wr ~ . nnw thr rdkct~ncr chH~ctNI~tiCS of an Objf?Ct, we can C:!lrllblr flow h11ght ~ WI f~ce rkmmt With that OrientatiOn w11l .11'1"'.11". Tim longhtne~~ rl1~tnhution on thr gradient $pace ~~ do:llott:d hy R( p.q ). and is called as "the Reflectance t-1 a p".

Usml~ 1hr rrtlrct~ncr map. the ha~1c 1m~gmg equation is

wlwr"' EP( :-:. .y ) ~~ thf" lmchtne~~ (1m:~gE' 1rradi:mce) in the 1111<~ ~_-.o: - lurllliii P, syst('lll at t~lf" poult ( x,y ) 111 the 1mage plane, prov1drd th:~t we c~n use the olihographic projection; m orhrr II'Oicl, thr olorrct i~ sm~ll compared with thr. d1stance to thP ~oi11 cr :.nrt thP image-forming sptf'm. This equation cont .liiH two unk11own v;~mhlr~ p. q. If WE' changE' (p 5 .q 5 ) kr.ep111g the (othr.r condit1on as samr, wr can $Oive for p and q fmm thr. rr.whing system of (non-linear) equations at the po1nt (x ,y); n~mrly, 1f we take more than one 1mage wJth V<HYIIli~ 1llummation cond1t1on, but from thE' samE' direction, we (.Ill oiJt .lill :1 sud.lU' Ollf'llt:liOn :Jt the point (x,y), This is the (JI'IIICII'le of photometric stereo (2].

2 Tnt:~l '\chema of the System

The tPchn1qur contains two kinds of tasks; one is the off· lme( pre computm~ ) job and thP. otlu·r 1s the on-line( real­ tim(' ) JOb whKh is rather s1mple comp:urd with the off-line jClb. Thr:' sunplicity implies rapid calculation, as drsired in any h:mcl·ryr ~ptr:'m . Thr:' off·linr:' 1ob comim of making the rt>flrct .1nrr map. constmctlng a lookup table, and a c,lllhr:·tinn t:1biP. of hnghtness. ThE' on-line job consists of re<HIIIIp; the lll)af;E' bn t~ htness and dt·t~·rminmg onentations of a Hll bcr patch IJHrd on the lookup tablr. F1g. I shows mfnrmatiCln flow brtwrrn thr on·hnr JOb and the off-line job.

3 Cakulation of tht' RPflt'clanrt' Map

One of th(' ma111 points oi our d1scmsion here is that we com1dcr only sprcul;u componr.nts of reflectance when we ~:Jicnbtr the rdlrrr~nce m~p. simr most industrial materials are IWHIP of metal and have strong ~prcuiHily with very we.lk l.11nloerti.1n ch~1~cterist1cs. We c:uanot trl'at such kind of nutnuls :H a mu.lll~mtJrltlan model. t.IE'~nwhile, ratios of sprwl.u comporn('IS to lambnmn compornt:'ls arc usually ahn11t ·,n to lt'l(l. Th1s situation allows m to consider only ~prcuhr rnmp0111t't~

For ,1 >pa:cul:n surf~co.· .wei .111 l?~tl?mlt·cl sourcE', we can obtam hll):htnr>s d1~tnhut1on on thr r;rad1cnt space from the drl1111ti0n of thr h1 ·durctwnal rdlcctance d1stnbution funrt1nn (1\R DF).111d rhr 11~1Hformat10n from the local coordua.He ~y~tem to thP. Vlf'WPr coordmate sy~tl'm [1).

R ( l'n·'ln ). L,( p,.q, ), ( :1 ) whcrr L, 1s the cl1stnbut1on function of the source rad1ance and

Pn~ (IT/.cl:<,

q" <·Tidy

Thus . evrn thou p: h the source cl1stribut10n may be compli c;IINI. only thE' contnbunon from a s1ngle d1rrct1on (p,.t'b) need br conmlered and onr nred not be concerned with the effect~ of other art'~s of thr d1stnbuted hght sourer.

4 Comideration of light Source

We ""' :1s a so11rce planE' a lamhertian ~urface that is illumlllated hy a hnrar lamp shown in Fig ~ - Though a sphr:'rtC.11 shapr source rastly can covl'r dm·ctwns more than ninrty clf:'~tN' S . 1t 1s d1lf1cult to m~kc an idral bnghtness dmnburi~n . It 1s possible to cover angles of more than nmrty ctq; rrrs by making a hox -hkl' ~ourcc . In that cHe, however. we luvp to treat each planr sepa rately becausE' the surbcr nurnul IS not d1fferent1:~ble at thE' 1ntrrsect10n of two pb nrs. Thus. wr considrr one plane surface wluch has the lambrrt1an cha l~ctrnqics and is 1llum111ated by a hne source a~ ~ rrprrwnrative case.

Bra f~ htnrss dastaabutaon on the surface is calculated by Eq. i . LPt f ht? the flux rate j)er unit source lrngth ( watt/(m sr)). tot:~larracla .1nce E [ watt/m 2 ) is

whcrr ro~() 1 ~ .;x 2 ·1o 2 1r ilnd cosil2=1olr and 1 0 is the distance from the l.lmp and the source pl~ne.

Wc J'lll <til object just under the point (x,y). (0.510.0.0). If we drnotc thl' dJst~wce from thl' surfacc to the object as 11,

5 CoJntrurtion of the Lool.up Table

Tllf' mnq ronvrnarnt mrthod for convcrting a triple of bng htnr~s to an onrnt~taon is farst to make a lookup table from the reflf:'CLince map. We can lookup the table by using a tnpk. Ltch entry of the table contains a paticular surface onl'nt~wat correspondmg to a tnple.

'We romtn1cr thf? lookup table by ming the Nrwton method. In the C.Hf' of a l:tmhertian reflector and point sources an ek~· .int method exists(~). In our case, however, we have to ol~ul:itl' 11 l'll1111C'tlcally. Nam'C'Iy, we solve three expressions of F.q . G with rrsprct to (p.q}. The three expressions are ciaffrrrnt 111 thf'll ~omce darrctiom (p 5 .q 5 ) .

From thl.' lol·kup t~IJie we ca11 get the surface orientation. Tltr luol.up t.li.Jie is actually two tnanp11:tr two·cllmensional m.1111Xn Although it as possable to make a three claml'nsional lookup table an which each dimension con<'>J"l·llds to thf' hnghlness of a surface patch under three sourc··~ . Jhe wc.1k~· st t.ng·htness likely contams measurement enor ;oad we only use 11 to det·~rmme a solution between two <1 hrrn~ 11 vn c<~mrcl by the non-hneanty. Thus, the lookup t :1 h le r:1n br two d amrnsiona I.

G Information Flow in the On-line Job

lm~ g r bn ;:htue~s as obtaint?d from thr. TV camera. Actually, wr 1n0k morr th:on OIH' picture prr hg.ht sourer, and arrays cnnP<JH)Jtrlm~ Jo thr ~:~me hght smarce were averaged. The rl'sultu, ~, . thJt··: lnae htnrs~ arr~ys. one for each light source, wcrl' IIIJ.tlll to the photometnc stereo system.

Wr sr:nchs rach .array for its 11\Hamum bng·htness. If objrcts are comp~n and the visual angle JS wade enough, the ima f; e HI'.1Y alw:tys contams the bnghtness corresponding to the -nuximum source brightness. So. by using these values, we can cancel the effect of varying albecio and can identify the m~ximum value in an image array With the value 1.0 in the reflcnance map.

We look up an entry mmg the two l~rr;rst brightness values. To inuease the accuracy of computation and economy of memory use, the first dimension represents the largest image bnt:: htues~ . The second dimension repre~ent~ to the second lart·.cst. ::.1nce the uoH-Iille~rity [;1ves me to two solutions for each bnghtncss p~ir. rach mrsh of the matnx contains two a ltrrn:~ ti \'e solutaons. Eaclt a ltrrn~ta vr solution conta ms the corrrsponrling surfa\e mirnlation and thr smallest image bnc htness . In order to decide which one of the two altcna,ttlv··s JS the r~C.ll ouc, we compare the distance between the ~ctu:~l thnd 1map;e bnghtncss and the element of the mataax. ~nd choo~e the ~olutaon concsponding to smaller dasr.1nce :~mong the two solutiom.

7 Experiment and Di\CII\ .~ion

ExperJillt'IH:~I rewlts are shown in Fag. 2. The algorithm rc:~rls thr t:lblr twace hy rxch;1ngmg s~cond brightness and thnd hnghtness and drtennine.< the ~olwion by avNaging.

A clirrct appiiCiltJota of om trchniquc is ati industrial hand­ eye sptrm that picks up an obJect out of a JUmble of m~trrial . Although tim technaquc cannot detect the wrface onrnJ :ttaon whrn there is mutual illumination. the technique rf'cor.: niH·s this conclat1on and doe~ uot produce an erroneous result. This is not so harmful H the result of mis­ dctcnnma 11011 of the orientatiOn. Brca usc the nus-understood gu1ci .1nce to a hanci may cause the hand or materials unrPcovrr:~hiP physic:ll defects.

Anothl'r <·ppllcatwn of our tP.chnaque is to 111spect surface con clition of meta Is. If a surface has a crack, stain, or finger prmt. the annge hnghtnes~ tnplc y1clds inmtent values for thr ~urf.1cr nnrnt.1t1on in the arr:l of the blemish.

REFEitENCES

(I) 11om. nand Sjoberg, R. "Calculatmg the Reflectance 1\hp." /\1-l'vlcmo -198, AI Lab, MIT, October, 1978.

(~) Hona. B. Woodham, R, and Silver, B. " Determining Sh~pe ~ud Reflectance Usmg Multiple Images." AI-Memo i91\ :\1 Lab. MIT, August,l978.