Improvement of Marko's Model Toward the Inter-Locutional Communication Theory

1. Introduction

Shannon's model of a communication channefll is useful for the efficient and reliable trf2~­ mission of information, while that of Marko may be more appropriate for treating the case of actual communication between two living creatures or between one creature and his surroundings, as Marko insisted. The author also agrees with the latter in the point that such a new model should be promising for developing an inter-locutional communication theory in the future. However I have a different opinion in some minor points, those which are the main parts of this paper. In short, the information flow between two nodes should be unidirectional during a small interval of time and it may ue considered bidirectional only when the time intervals are sufficiently long. From this standpoint, I would like to insist that the Shannon's model is not a special case of Marko's model and the new model should include both.

Before considering the new model, it will be helpful to review briefly both models.

2. Shannon' Model

IS,T,r.H,R,D and N, shown in Fig.l, respec­ tively express information source, transmitter, communication channel, receiver, destination and

IMFROVEMENT OF MARKO'S MODEL TOWARD THE INTER-LOCUTIONAL COMMUNICATION THEORY

Hideo Seki

University of Electro-Communications Tokyo,Japan

noise source.

Let x 1 , x 2 , ··· , xk, ·.· x 0 be representation elements(e.g. letters) at the transmitter side and the set of these elements be written as X. Similarly, Y is the set on the receiver side, including y 1 , y 2 , ·.. , yi'''' y . Then the self entropies may be expressed as follows.

where P(xk) or P(yi) is the probability of oc­ currence. P(xk..' yi) and P(xk\ yi) are joint and conditional prooabllities respectively.

As is well known, the mutual entropy is used for the estimation of channel capacity and is relat­ ed to other entropies as follows.

where

(5)

The conditional entropy H( xiY ) is called equivocation, while H( Ylx ) is channel noise entropy. According to the notation of Marko, the latter is written as o 12 or o 21 and called by him as discrepancy. The net information at the receiver side is, of course, the difference be­ tween the tranmitted information and equivoca­ tion. This difference is I( X;Y) itself.

3. Marko's model

The·notations in Fig.2 are slightly modified from those shown in Marko's model in the origi­ nal paper so as to facilitate the reader's com­ parison with Shannon's model, i.e. H( X ) and H( xj Y ) are used here instead of H 1 and n 21 re­ spectively . However, I( X;Y ) cannot be used for T 21 , because of the unsymmetric character of T 21 ana Tl 2 . In this case, I( x~Y ) was used for T 21 , whi e I( X~Y) for T 12 .

Nofe that represents the set of transmitting elements in I( X·Y ), while the same letter X denotes the set of received elements in I( X~Y ). As will be shown later, symbol I( X;Y ) may be used for the sum equation (15).

In this model, X is an information source as well as a destination and Y has also two meanings . For this reason, X and { 3 yan conven­ iently be called inf ormation nodes .

Let the message composed of the message elements x 1 , x 2 , ... xk · .. xn in node X be

and that composed of y 1 y 2 ... yi ... ym in node Y be

Then, according to Marko's theory, the message element x followingq not only depends on the elements Ql ~2 .. . ~ut also on &1 82 ... a . Let us represent thi~ CDnditional probabilitynon the side of node X by P( x!xn , yn ) . Similarly, the conditional probability on the side of node Y can be defined by P( Y\ yn, xn ) . Consider also the j oint conditional probability P( x, ylxn,yn) which is the probability of joint occurrence x and y due to the preceding n events on the side of nodes X and Y respectively.

Then the total subjective entropy on X side can be defined by

and that on Y s i de by

On the other hand, the free subjective entropies on X and Y sides can respectively be defined by the foll osing two equations.

be written down by

In contrast to the Shannon's model, where the equivocation was simply expressed by

and the channel noise was

as in equation (3), the corresponding quantities should take the following forms in Marko's mod­ el.

Note that the free subjective entropies seem similar as above but have basic differences as follows

The directive subjective mutual entropies defined by equations (11) and (12) are, of course, non-symmetric . But their sum becomes symmetric as follows

Thus this sum may be understood as the total mu­ tual entropy and be written as I ( X;Y ), usi ng the same notation as in the case of Shannon's model. Then the definition of total mutual en­ tropy may be written down as follows.

The following relations also are deducible from equation (14).

The directive subjective mutual entropy through the left-going communication channel should be expressed as (11)

and, therefore, the similar quantity through the right-going communication channel can analogously

P( x lx,._n_,_>.,......,-} Such condition was denominated by Marko as "suggestion", where the information received by node X from node Y shall be returned without any transformation. However, in the general case of

"stochastic degree of synchronization" or "de­ gree of perception" was defined by him as fol­ lows.

is satisfied, the coupling between two nodes become maximum and the reletion

results, as can be seen easily. But, in general,

results from equations (14) and (15).

Fig~ 3 shows the possible values of the stochastic degree of synchronization under dif­

ferent conditions.

4. Improved Model

Marko's model seems perfectly general so as to be applied to the inter-locutional communica­ tions between the animals and machines. However, it seems to me that a fine-structural refinement should be necessary for further development of the bidirectional information theory. The reason is that the telephone communication between two persons, data communication between two computers and many other communications are unidirectional as far as a short interval of time is concerned. TASI system is an example -that such substantial characteristics in speech were properly used. This means the model should be Shannon's, at least, witnin a short interval. Reversing of the direction of information flow occurs only when the storage and processing of the received mes­ sages are completed. This idea necessarily re­ quires the model to be modified to a new model as Fig. 4 where the situation of information source and destination alternatively reverses from time to time. In Fig. 4, IN means information node which becomes information source or destination

depending on the instant of situation.

Now, the model of Fig. 3 is a uni-direction­ al communication channel and the equations from (1) to (6) hold exactly, so far as the change­ over switch is set on the fixed side. As a con­ sequence, H ( XIXY ) in Marko's model becomes simply as H ( X ) , or

Thus, all relations which were deduced in the unidirectional communication model may be apl­ lied to the new model. In a rigorous mathemati­ cal sense, the notation n~·in (22) and (23) may not be true, because of the finite duration of time of the change-over switch position . But, it is not a serious problem in practice.

Let us consider next the case of two commu­ cation channels of opposite flows of information between two communication nodes. In such cases, the continuous flows of signals may occur in both directions, and the model cannot be consid­ ered to be unidirectional if even a very short interval is taken so long as we pay an attention to the middle point of communication route. Even this model also may be regarded as a unidi­ rectional flow, at least, at both terminals of communication route where two change-over switches are inserted as shown in Fig . 5, which is the result of fine-structural consideration. Thus, considering the network inside the dotted square as a communication channel, you can see that the Fig. 5 exactly coincides with Fig. 4.

Now, anyway, the changing over of switches should occur always when, at least, a whole sen­ tence is completed. In other words, the revers­ ing of direction of an information flow will al­ ways occur when a sentence '"\ a: 2 .. . an or sen­ tences composed from representafion elements x 1 x ··. is over. Let this sentence or series ot sentences originated from node X be~. Then, node Y will send back another sentence or series of sentences choosing from his repertoire, con­ sidering it as a most suitable one for a special purpose in question or for performing a special task. And let P (~ 1 1§.> be a probability that Y chooses when he receives 51 from X. Then the probability that X sends out ~ 2 after receiving~~ from Y, may be written as P CE,_)~, ').l. Using the abbreviated symbols~ instead· of~ . ~ 2 ··· ~ n. and t instead of?· ( 2 ··· , equation (7), (8)", (9) and (10) may be rewr tten in the following new form as

(27) Thus you can see that the Marko's entropy equa­ tions become necessary at this level of sen­ tences. Similarly, the equations from (11) to (17) may be rewritten in the same manner as above, but may be omitted here for the simpli­ fication.

In short, the improved model coincides with Shannon's model at letter ( representation ele­

ment ) level and with Marko's model at sentence

( message ) level. Representation elements have no meanings, while the messages have.

Communications having a fixed purpose among nodes might have considerable effects on their forthcoming behavior. If the relation between the messages and bahavior could be estimated quantitatively, a value anylysis of information may also be possible.

5. Multiple Inter-Locutional Communications

It seems to me that the stochastic degree of synchronization (18) proposed by Marko is a measure of persuasion. But the purpose of com­ munication among multiple nodes may be flexible. For example, it may be consultaion, discussion, quarrel or any other type. In such cases, value of information or measure of learning may become necessary instead of the stochastic degree of synchronization.

The old proverb says, " two heads are better than one ". The similar proverb in Japan says, " three heads constitute Manjusuri ". This sug­ gests that the value of information can be raised up by communications among multiple nodes . Al­ though the quantitative analyses of such informa­ tion processing are extremely difficult at this stage, still we can roughly analyses, in princi­ ple, according to the model of Fig . 6 . Fig. 6 (a) is an abbreviated diagram of Fig. 4, at the same time inserting third node z. (b), (c) or (d) of Fig. 6 show states, respectively, when X, Y or Z is the information source and the remain­ ing two are destination.

No~, let the probabilities that Y and Z choose from their repertories after receiving the message ~ 1 from X in (b) be P(~,l~ 1 ) and P(),l~ , ), these two being generally different from each other. Changing to state (c) from (b), we can again assume the message choosing proba­ bility of X as P(~,j5, .~ 1 ) and that of Z as P(~ 1 \~, . ~ 1 . ~), whilel at state (d), these probabilitfes become P~1~1,)1·?,·'/.> at X and P(~~~. ·(, ·52 ) at Y. Returning to the state (b), X wlif send message tJ' which excite Y and Z by the probabilities 1S (~~,{1,?,·t.· f l) and P <~! f,,~3 .?,·)r' )) . Note that the ~~bsc1pi numbers ar~ not necessarily in order.

After n times of trnsi tion, the probabilties

may be c~nside!ed and finally expect that the probability of some special message becomes very large, or

Fig. 7 shows this tendency. Before starting inter-locution, the figure showa that the proba­ bilities P of choosing any message are uniformly dis t ributed. Assuming the abscissa is the order number of probabilities arranged from higher to lower ones, the pea~ of the curve gradually rising up after repeating inter-locution and finally approaches unity .

• In Snannon's mathemetical theory of commu­ cation, the average amount of information mes­ sages instead of spec.ial messages was a most im­ portant quantity. But; in our case, probabili­ ties of same messages are rather important and this very quantity may be considered a measure of the value of information.

Another suitabt~)measure for the group as a whole is learning , i.e.

where m is total number of possible messages and H is an average amount of information per mes­ sage, example of the form of estimation being

The number of nodes also can be extended t o more than three. Christie and others have dis­ cussed the case of nodes ·5 and got some experi­ mental results under proper constraints.

• A brain also can be taken as a system con­ sistinfg of many localized information proces­ sors or nodes and communication channels between nodes. Thus even a brain ordinarily 'considered as one node in the case of inter-locution should be taken as an ensemble of nodes. If so, the above mentioned new model probably become one of the important suggestions for treating thinking processes of the brain.

6. · Conclusion

In concluding my discussion about the new model for inter-locutional communication , it seems better to summarize some important points for further studies as follows: ·

1. Though the node in an inter-locutional com­ munication net is able to be an information source or a destination, it must remain to be either of them within a short interval of time , i . e. the model must be uni-directional .
2. b . Statistical relations between representation elements follow the mathematics of Shannon's model at the level of uni-directional communi­ cations.
3. Statistical relations between messages, should follow the mathematics of Marko's model at the level of inter-locutional communications.
4. Though continuous information flows to both directions may occur in a 4 wire communication channel, it still may be regarded as a revised model, because two change-over switches are necessary just in front of the information pro­ cessing units.
5. The new model can be extended to the inter­ locutional communications between more than two nodes by considering conditional probabilities of choosing particular messages. Thus the def­ inition of the value of information and the learning of the group may become possible .

Finally, the author would like to offer many thanks to the members of I.E.C.E.J. for their suggestive discussions.