Speech Training Systems Using Lateral Shapes of Vocal Tract and F1-F2 Diagram for Hearing-Impaired Children

1. INTRODUCTION

If hearing-impaired children can see their ar­ ticulation and compare it with a reference ar­ ticulation in order to bring their articulation closer to the reference, or if they can see the formant frequency locus for their utterance on a Fl-F2 plane, where the regions of the vowels are shown, they might be able to understand how they articulated or how they should articulate. In addition, teachers might be more able to teach them how to articulate or know where their articulatory defects exist. Therefore, we have constructed three speech training systems using a minicomputer and a microprocessor. For vowel articulation the first system estimates a later­ al shape of the vocal tract from the speech sound and displays it on a storage CR-tube. And it can compare the estimated shape with the ref­ erence one and produce instructions with animat­ ed cartoons which show where any articulatory defects exist. For consonant articulation we cannot yet estimate a vocal tract shape from the speech sound, but our second system can display successively the lateral shapes of the vocal tract for the articulation of any phoneme se­ quence containing consonants. Our third system displays the first two formant frequencies ex­ tracted from a speech sound as a spot on the Fl­ F2 plane, where the regions of the vowels are also shown in color. It can also display pitch

frequencies.

2. SYSTEM !

This system estimates shapes of the vocal tract for vowels by evaluating PARCOR coefficients and instructs where any articulatory defects ex­ ist and also indicates how a child should cor­ rect his articulation. Fig. 1 shows the main flow of our procedures. The operation of this system is simple, and one has only to type a number or YES/NO in reply to the system inquiry " = " or " ? " shown in the state c::J of this diagram. Then the system proceeds to the next state executing the action described along the line, types the characters shown in the new state and waits for the next reply. Referring to the marks in the diagram we will explain what this system can do chiefly.

• (a) In order to avoid disturbance from the room noise, you can specify a threshold value in re­ ply to the inquiry "Cut level=" at which the system begins to store the input speech wave for 3.2 sec.
• (b) The system displays the reduced input speech wave on a storage CR-tube as shown in Fig. 2 by typing "N" at the state (ii) IAgaJ.n?l Therefore you can examine the speech wave enve­ lope and select the places where you want to obtain the vocal tract shape.
• (c) At the state (iii) you can enlarge the wave form at any specified place and find its pitch period as shown in Fig. 3.

. Jif~· ~ ... I I I

• (d) Two storages labe lled "1,2" for s t o ring the estimate d shapes are prepared and you can re­ spond with one of them to the inquiry "Patte rn=" at (v). Whil e , in the third storage labelle d "0" you should store the s hape which you want to use as a refe rence .
• (e) For the inquiry "Display s hape= " at (vi) you can s peci fy a ny s toraqe containing the s hape . Then the s pecified s hape i s di s played.
• (f) At the state (viii) !Refere nce - O, ·· ·? j, if you input "0" then the e stimated shape i s com­ pared with the content in the s torage "0", and if you input "1" or "2" then the es timated s hape is compared with the standard r eference s hape which was s pec ified at the state (vii) jPresent vowel- I . Comparison be tween the es timated s hape and the refere nce one is done for the two fea­ tures: the place of articulation and the lip­ opening. If the difference between the places of articulation of both s hapes i s over a certain

threshold value, an instruction such as "nraw back your tongue" i s s hown and the estimated lateral shape moves gradually toward th<· rc·f,·r­ ence shape as shown in Fig. 4. lind if till' clif­ ference between li p-openinqs of both s hap<'>; i ,; large , an instruction ,;uch .1:; "OJ>l'll your I i p::" is written and the lips repres ented by ull l'l­ lipse move gradually toward the refL·rence open­ ing (Fig.4). If the difference is a little, the instruction "OK" i s written as shown in Fig. 5. The refe r e nce s hape stored in the storage "0" can be the prepared s tandard shape or the l':>ti­ matcd shape which has be en es timated from tilt' teache r's voice through the above-described t:ou­ tine .

(g) Go ing along the route 3 by inputting "3" at the s tate (i) jLink order- J, we can estimate and di splay successively the vocal tract s hapes of the vowels at the pre-specified places .

Fig. 5 /In t!Xampl ,. of practice . One can .Jr­ t i c ula te seeing th<· t: e ference shape shown by broken line . In thi s case both in­ s tructions for articu­ lation of /a/ are 11 0K".

For practice, at first you should display a standard shape prepared previous l y or a shape estimated from your own utterance. Then a chi l d can articulate seeing the reference shape , and re-articulate comparing his own articul ation with the refe rence shape (Fig. 5). The practice shapes of the child should be s tored in either storage "1" or " 2", then he can successively compare h i s articulation with the previ ons ar­ ticulation or the reference shape , if he wants, l ooking at instructions .

We have prepared a system which displays syn­ the tic articulatory movements for monosyllable u tterances or words containing consonants, be ­ cause it is very difficult to estimate a vocal tract shape from a speech wave for any of the consonants. The lateral shape of the vocal tract f or Japanese phon eme sequences are produced by our speech synthesizer u s ing an articulatory model [1]. The synthesized lateral shapes are stored on a drum storage and displayed with ar­ bitrary speed . A child can easi l y l ook at the articulatory movements containing stop or frica­ tive consonant s . Fig . 6 shows the synthesized shapes in the sequence /ika/ .

4. SYSTEM 1

This system displ ays the fir st two formant fre­ quencies extracted from speech as a spot on the Fl-F2 plane , where the r egions of the vowels are shown in col or as shown in Fig. 7 . It can also di splay pitch frequencies . Thi s system uses a microprocessor and a color TV set.

Thi s system is useful by itself , and also help­ f ul in order to know the quantative vowel qual ­ ity.

5 . DISCUSSION AND CONCLUSION

Our estimated shapes of the vocal tract are not accurate, but we can approximate the place of articulation for vowels and point out the de­ fect of articulation and, o f course, the sys tems function properly for hearing-impaired children. But an infant i s unable to under s tand instruc­ tions s uch as "Draw back your t ongue body", so these systems should be used to of f e r informa­ tion to a teache r .

We also hope that these systems are used by o ld­ e r persons for self-teaching .

Though we have not yet evaluated the effect ive­ ness of these systems , p r eliminary use of t he se devices has shown that they complement each o th­ er to form a more effective system of s peech training. Especially it is instructive to use the systems 1 and 3 s imultaneously , because we can get articulatory information in both articu­ latory and freque ncy domains.

Authors would like to express the ir gratitude to 0 . Tanaka, T . Nishide , Y. Furuya and other students in our laboratory for the ir he lp.

REFERENCE

[1 ] Shigenaga, M. and Ariizumi, H. "Articulatory Movements by Rule " in Modeles Articulatoires e t

Phonetique ed­ ited by Carre , R. et al. (G . A. L.F. Groupe de la Communica­ tion Parlee) , 1977, pp . 193202 .

(c) from /k/ to /a/

Fig. 6 Synthesized lateral shapes of the vocal tract for /ika/ . In our synthesizer the shape of the vocal tract begins to move from a uniform tube , therefore in (a) it starts from the shape paral l e l to the upper wall and moves toward the reference shape of /i/ .

An output in the system 3 for /aoi / .