An Infinite-Connected Words Recognition System for Male Speakers Using Time-Space Dynamic Programming

AN INFINITE-CONNECTED WORDS RECOGNITION SYSTEM FOR MALE SPEAKERS

USING TIME-SPACE DYNAMIC PROGRAMMING

Ryu-ichi Oka Pattern Processing Section Information Sciences Division Electrotechnical Laboratory 2-6-1 Nagata-cho, Chiyoda-ku Tokyo 100, Japan

A new algorithm of pattern matching for time-space speech feature using Dynamic Programming is proposed for recognizing infinite-connected words spoken by male speakers without adaptation of speakers before the utterance. The new algorithm carries out frame-~isely two kinds of normalizations : the time normalization for removing speed variat1on of speech and the space normalization for removing speaker-dependence. This algo:ithm is realizable by a simple cellular automaton. The recognition rate 85.7% is obta1ned for 50 categories, h male speakers and 1140 samples (connected words).

1_. INTRODUCTION

The purpose of this paper is to propose a recognition system based on a new dynamic programming algorithm for recognizing continuous speech spoken by male speakers. This system is realizable as a simple cellular automaton. Recently, recognition systems dealing with isolated words spoken by many speakers are proposed [1], [2 ). Other systems dealing with several connected words spoken by a speaker are also proposed [ 3), [ 11]. The following abilities are desirable for a system; 1) dealing with infinite connected words in an utterence, 2) allowing for many speakers without speaker adaptation before the utterance, 3) working in real-time. \Ve propose a system with these

abilities.

~· RECOGNITION PRINCIPLE

The principle is based on a new pattern matching algorithm using dynamic programming called Time-Space Dynamic Programming (TSDP) . With regard to speech feature with the time and space axes such as spectrum and vocal tract area function, TSDP is applicable for normalizing two kinds of variations occured in time-space axes of the feature. The one is time normalization for removing speed variation of speech and the other one is space normalization for removing speaker-dependence. Two normalization parts of TSDP are curried out frame-wisely without segmentation of a continuous speech. Segmentation problem of continuous speech is

maintained to the final stage of recognition process. Two kinds of normalizations of TSDP are separable.

l· OUTLINE QE..:!.!!! SYSTEM

The recognition of words is carried out frame by frame. We can divide this frame-wise recognition process into three stages as follows.

1. Extraction of a vocal tract area function.
2. Application of Time-Space Dynamic Programming.
3. Frame-wise decision of category.

!:_. ~ ~ ~ ?UNCTION EXTRACTION

We use a vocal tract area function as a parameter. The reason is that a vocal tract area function is more stable for different male speakers than other parameters suet as spectrum. The pre-experimental results revealed this conclusion.

The extraction technics of a vocal tract area function from speech waveform have already developed by Wakita [6) and Nakajima et al [7] by using inverse filter technic to remove glottal wave. We used an improved technic of Nakajima's one that is a frame-wise inverse filtering method of second order critical damping with a positive or a negative pole. The used one is a frame-wise inverse filtering method with the only positive pole. This method is more suitable because it does not carry out over inverse filtering for the frames in the

consonant part of speech waveform.

The analysis condition is as follows 10 kHZ

sampling, 20 msec window for analysis, 10 msec frame interval, 12 sections for a vocal tract area function and a vocal tract area function is normalized with regard to its volume.

.2_. ~-~ DYNAMIC PROGRAMMING

.2..·1 Time Normalization ~

Let l f(t,x) : 1 ~ x ~ LJ be a vocal tract area function of the time t, where x indicates the space axis, i.e. vocal tract, and L is the length of vocal tract. Let { Z(( ,x) : 1 ~ t ~ T , 1 ~ x ~ L} be a standard pattern representing a word, where T indicates the length of the standard pattern and Z' corresponds to the time axis of speech feature. We define a function A(t) representing the temporary minimum distance between a past input pattern with the optimum length and a standard pattern, i.e.,

()j,Z'j) j=l,2, ·· ,T where Kj= ~. - 5 + [.. 1 -i·, ~~ 2'. ~ .. "2 · ·· :2 )d 2 · · 2 ); }. o, J- 1 I' I (,2l'~."2 ···2'Zd'2 ·· zTr::.l,

and J*(t) is the sum of Kj's determined from the optimum pairs ( 5* j, z'F j) from j equals one to j equals T. We will propose an algorithm for calculating A(t) frame-wisely using Dynamic Programming. When we calculate A(t) by the following method, we call it the time normalization part of TSDP. This idea is suggested by so-called Max-Product Parallel Machine proposed by Y.Isomichi [5]. Three variables are necessary for describing the time normalization part of TSDP. The variable Q(t,~) indicates the local distance, P(t,t) indicates the optimum value obtained from integration of the values of Q(· ,·)'s, and C(t,L) indicates an integrated value of Kj's which are embedded in the value P(t,z). The variable C(t,t) is necessary for normalizing the length of the optimum pass.

• P(t-2,Z'-1)+2Q(t-l,t)+Q(t,'t) (d)

C(O, (')=C(-1, ()=0 (1 ~ Z'S T) C(t,l)=2

• C(t,2) ! C(t-2,1)+3 = l C(t-1,1)+2 if if (a) (b)
• C(t,l)+l if (c)
• C(t-2,£:'-1)+3 if (d)
• { C(t,Z') = C(t-l,f-1)+2 if (e)
• (3.fZ~T) C(t-l,'l:'-2)+3 if (f)

where Q(t,T)= ~I f(t,x)-Z([,x)\ and M is the fixed value. Then the following equations hold.

The detail discussion about these derivations is described in a paper [8].

.2..·~ Space Normalization ~

The space normalization is applicable to the determination part of the value Q(t,~) of the time normalization algorithm.

If we introduce a new variable g(t,"t,x,~) and suppose 1~ ~ -1 f x ~ .; +l~L, then the space normalization algorithm is described as follows.

g(t,"t,l,l)=2 I f(x,l)-Z(t,l)\ g(t,Z,x,~)=min[ g(t,!,x-l,~-2)+2lf(t,x)-Z(t,~-l~ +If( t ,x )-Z('f .~ll. g( t, T ,x-1,~-1 )+::V( t ,x )-Z ('t .~)l ,g(t,t,x-2,~-l)+~(t,x-1)-Z(Z',~~ lf(t,x)-Z(T,~~].

If the time defined by algorithm is indicates the respect to the equation holds normalization part has Q(t,T) Q(t,T)= g(t,T,L,L)/2L, then the called TSDP. The value Q(t,Z') local distance normalized with space x in f(t,x). The following about the output of TSDP A(t).

A(t)= 1 in~I.Kj(m~n ~ Wijf(t-)j,xi)-Z(C'j,~i~\1

I

(}

2L J* ( t) ( J

"

()j,(j)j=l,2, ·· ,T where Kj = )j-1 - SJ + (j-1 - (j , Wi = xi - xi-1+ ~i -~i-1, xl=l, xL=L, ~1=1, ~L=L.

~. CATEGORY DECISION ALGORITHM

Let { Ai(t): i= 1,2, ·· ,n} be n values determined from TSDP, where i is the category number. Let tF be the end time of a word in a continous speech, We never use tF in the recognition system. Section 5. reveals that the value of Ai(tF) is the smallest among those of Aj(tF),j\'i, Ai(t) (t(tF), and Ai(t ) (t)tF). We define il*(t), i2*(t) so that Ail*(t) (t)= min Ai(t) S:..A. , Ai2*(t) (t)= min Ai(t) ~

Ail*( t) ( t )+h ~ i\. for i'\=il *( t), where A. and h are the positive constants, Then we define the decision category number i*{t) by

where Lil*(t), Li2*(t) are the lengths of the standard pattern il*(t), i2*(t). If il*(t) does not exist, we set i*(t)=O. The variable i2*(t) is the candidate category number of the second minimum distance. In general, further algorithm is necessary since there is a time sequence whose frames indicate the same nonzero recognition category number.

-1· EXPERIMENTAL RESULTS

Recognition experiments were carried out for 50 words making weather forecast sentences ( 25 Japanese-city names and ·25 Japanese words about weather forecast ). The standard patterns are isolated utterances of the words spoken by a male speaker who is out of the tested 4 speakers. The uttered connected words take a form of a sentence with natural speed, Three types of error existed, i.e., confusion: the system decides an uncorrect category at the time when a word must be recognized, missing: the system decides no word at the time when a word must be recognized, ghost: the system decides a category .at the time when no word must be recognized. Table 1 shows the two kinds recognition results of connected words (application of TSDP and the time normalization only of TSDP).

8. CONCLUSIONS

A new speech recognition system has been developed and evaluated from the viewpoint of continuous words recognition for male speakers without segmentation and speaker adaptation. The recognition system is realizable as a simple cellular automaton. We obtained 85.7 % as the recognition rate for 440 connected samples, 4 speakers and 50 categories using the standard patterns spoken by a male speaker who was out of the tested speakers.

ACKNOWLEGEMENTS

The auther would like to thank Dr.S.Mori and Dr.T.Nakajima and the menbers of the Speech Recognition Section for supporting this work.

TIME-SPACE DYNAMIC PROGRAMMING

R.O R,O

9 males R.O

500

48

466

30

15 19 28 93.2 %

5 12 7 62.5 %

ONLY TIME NORMALIZATION PART OF TSDP