Toward Flexible Intelligence: MITI's New Program of Real World Computing

1 Introduction

The Real World Computing (RWC) program is start­ ing this year on a large scale as the successor to the Fifth Generation Computer project which formally ended in 1992. This is the next Japanese national project launched by MITI (Ministry of International Trade and Industry) with a budget of over $ 500 million for ten years

The reason why we dare call it "program" is that we intend to pursue more fundamental and generic technol­ ogy in a new and wide framework of research and de­ velopment different from the traditional MITI projects which have mainly devoted themselves to the develop­ ment of machines. Through the program, which is open to the world, it is also intended that Japan makes a greater international contribution to basic science and generic technology which should be the common prop­ erty of mankind.

The primary objective of the new program is to lay the theoretical foundation and to pursue the technolog­ ical realization of human-like flexible and intelligent in­ formation processing as a new paradigm of information processing towards the highly information-based society of the 21st century. Contrary to that the Fifth Gen­ eration project pursued the aspect of logical (symbol­ based) information processing (or intelligence) of hu­ mans, the RWC program is rather pursuing another as­ pect of intuitive (pattern-based) information processing as a new framework and foundation of information pro­ cessing, and also aiming at unifying both aspects in a bottom-up manner .

In this paper the outline of the Real World Comput­ ing program is introduced, referring to the background, concepts, and contents of the research and development of the program.

2 Background and Objective

Supported by the remarkable development of computer and communication technologies, information technol­ ogy is producing an innovative change in the society, not only in industrial activities but also in the qualita­ tive improvement of our way of life. It is foreseen that information to be handled will explosively increase to­ ward the next century because of the increasing needs of multimedia information processing and the expansion of new application domains. It means not only the increase in quantity but also the increase in quality and variety of information.

Such social and technological needs are starting to re­ quire a new paradigm of information technology, not as a simple linear extension of the conventional one, but as an essentially new underlying framework . In other words, it is necessary to make computers more friendly and easy to use by providing them with human-like flexible and intelligent capabilities in order to assist and collaborate with humans in the diverse information environment of the real world.

Looking back the history of computing, modern elec­ tric computers originated from mechanizing calculation with logical circuits. In the first stage, they developed along the line of conventional von Neumann architec­ tures in the fields of numerical computation, document processing, management of data-base storage and re­ trieval (see Fig. 1), since those application fields have clear algorithms and are suitable for processing by con­ ventional computers.

The second stage of development was directed toward manipulation of symbols and logic, casting light toward the intelligent thinking process of humans such as logi­ cal (deductive) inference. The research fields of AI and knowledge engineering contributed in providing comput­ ers with the capability of handling symbolic representa­ tion of knowledge and inference rules. In 1981, Japan proposed the concept of the Fifth Generation Comput­ ers as an approach to new generation computing based on the logic programming. The goal was to provide com­ puters with the powerful logical inference capability and to open the door to the world of large-scale knowledge information processing.

Today, · computers have come to possess enormous computing power and far surpass human ability in solv­ ing well-defined problems such as numerical computa­ tion, document processing and logical inference in pre­ assumed ideal worlds where the algorithms for solution exist and can be clearly stated in programming lan­ guages. Nevertheless, computers are still inferior to hu­ mans in many areas such as pattern recognition, problem solving under incomplete information, and learning. The framework of information processing by modern comput­ ers is still not so flexible compared to human flexibility in information processing in the real world where many problems are ill-defined and hard to describe in algo­ rithms. It might be said that current information tech­ nology is still immature in the so-called intuitive aspect in contrast to the logical aspect.

Therefore, in order to cope with such real world prob­ lems and to open a new horizon in information process­ ing technology, it is essential to pursue the fundamental ways of human-like flexible information processing, by casting light on the intuitive or subsymbolic level of hu­ man information processing, and to embody them as new information processing technologies on the basis of the developing hardware technologies.

Given these circumstances, the concept of "Real World Computing" is proposed as a new paradigm of infor­ mation processing which aims at furnishing the real­ worldness (or flexibility) of human information process­ ing to information systems. Development of informa­ tion systems which have human-like flexible informa­ tion processing functions and can cope with real world problems is now one of the most important demands common to various fields, such as pattern information processing, knowledge information processing, intelligent robots and friendly man-machine interface, which aim , 0 at further advanced intelligent information processmg.

3 Key Concepts of RWC

Here let us consider the notion of flexibility which is required for Real World Computing (therefore, which is lacking in the conventional information systems) . It will be summarized into the following functions:

• The function to integrate a variety of complex and intricately related information containing ambiguity or uncertainty and to reach an appropriate ( approx­ imate) decision or solution within a reasonable time,
• The function to actively acquire necessary informa­ tion and knowledge, and to learn general knowledge inductively from examples,
• The function to adapt the system itself to users and changing environment.

Those can be regarded as corresponding to robustness, integratedness, real-timeness, and openness, which are required in RWC systems.

Those functions can be further summarized into the following two essential functions:

• The function of information integration,
• The function of learning/self-organization.

Thus, ''flexible information processing" is the key con­ cept to indicate the functional aspect of RWC, which is necessary for information systems to work without fail­ ure in the real world environment that is full of uncer­ tainty and change. Actually we humans, ourselves "real world computing systems", have such flexibility, and it is realized and supported by the massively parallel and distributed information processing in the neural network of the brain.

Therefore, "massively parallel and distributed process­ ing" is the key concept to indicate the computational aspect of RWC, which supports flexible information pro­ cessing. Here it should be noticed that the terms par­ allel and distributed are different from the conventional usage in the computer science in the sense that a variety of information is represented redundantly as distributed patterns and processed in parallel in the dynamics.

Hence, RWC or flexible information processing can be characterized by parallel distributed and learning type information processing, while conventional information processing can be characterized by serial localized and procedural type information processing.

The primary objective of the RWC program is to real­ ize flexible information processing as a new paradigm of information processing toward the 21st century and to establish the theoretical and technological foundations for expanding the application fields of information pro­ cessing to the real world.

In the traditional information processing, humans (users) are forced to adapt themselves to accessing coi?­ puters using hard logic. In the future, computers will be expected to get close to humans and support human intellectual activities, collaborating in the diverse infor­ mation environment of the real world. For that purpose, it is necessary to provide computers with the ability to flexibly deal with and process a variety of information in the real world, such as images, speech sounds and languages, just like humans do. In this sense the RWC program is aiming at foundations for new generation in­ formation processing or AI which might be called Com­ puter Renaissance.

4 Contents of R&D

In order to accomplish the objectives, it is, first of all, im­ portant to explore theoretical foundation which under­ lies the real-worldness and flexibility of human informa­ tion processing by casting light on the intuitive aspect . Upon the theoretical foundation, novel functions for ap­ plication, such as flexible recognition/understanding, in­ ference, problem solving, and control, are pursued, and by integrating those novel functions, realization of new information systems dealing with real world problems is explored. As the computational bases for those, it is nec­ essary to develop new computer architectures including neural networks and optical computing devices.

Therefore, the research and developments in the RWC program are divided into the following three mutually related subjects:

• Theoretical foundation,
• Novel functions for application,
• Computational bases.

The three-storied structure shown in Fig. 2 is the fun­ damental framework for the organization of research and developments in this program.

In the following sections we shall outline respective research contents and topics in more detail.

4.1 Theoretical foundation

The research objective is to establish a new theoretical foundation for flexible information processing.

For this purpose, it is first of all necessary to expand and generalize the conventional framework of informa­ tion processing in all aspects of information represen­ tation, processing and evaluation. In that generalized framework, it is important not only to continue in-depth study in the related research areas, such as pattern recog­ nition, multivariate data analysis, probabilistic and sta­ tistical inference, fuzzy logic, neuro-computing, machine learning, regularization and various optimization meth­ ods, but also to theoretically clarify the principle or "soft logic" which commonly underlying these research fields, toward aiming at construction of a new unified theoreti­ cal base for RWC [Otsu, 1991] . There, probabilistic and statistical formulation of problems and nonlinear dynam­ ics will be a key approach.

Research topics and key words are the followings .

• Flexible representation of information: patterns vs symbols, spatial vs temporal, probabilistic, topolog­ ical, or hierarchical representation, and knowledge representation as constraints, etc.
• Evaluation of information and processing models: from True/False to degree of certainty, information criteria such as MDL and AIC, energy function, etc.
• Flexible storage and recall of information: associa­ tive memories using similarity, probabilistic reason­ ing, Bayesian networks, non-linear dynamics, etc.
• Integration of information and of processing mod­ ules: constraint satisfaction, neurocomputing, mul­ tivariate data analysis, etc.
• Learning/self-organization and optimization meth­ ods: statistical learning, multivariate data analysis, neurocomputing, GA, etc.

In particular, integration of multi-modal information (and of heterarchical processing modules) and learn­ ing and self-organization (optimization and adaptation) are the most fundamental issues, and how to imple­ ment these in a framework of massively parallel and distributed information processing will be a major key point in developing novel functions for application, such as flexible recognition, inference and control.

It will also be important to learn and get inspiration from nature, namely to take into account new findings in scientific research into the brain, evolution process of creatures, and ecological dynamic systems.

4.2 Novel functions for applications

Since flexible information processing intends to expand the abilities of information processing beyond the con­ ventional limitations, the range of expected application fields are quite wide.

Research and development on novel functions for ap­ plication should be directed toward investigating elemen­ tal novel functions which are essential for realizing the flexible information systems which solve a wide range of real world problems. Those functions are the following:

• Flexible recognition and understanding of multi­ modal information such as pattern information (im­ ages , speech sounds, etc.) and symbolic information (natural languages) in the real world,
• Flexible inference and problem solving based on a flexible information base which admits direct treat­ ment of multi-modal information and has learning and self-organization capabilities,
• Flexible human interface and simulation enabling smooth man-machine interactive communication through enlarged multi-modal information chan­ nels,
• Flexible and autonomous control and integration methods to interact with real world environment, such as sensory-motor association, etc.

The novelty of the functions should emerge from new concepts of theory or algorithm suitable to RWC, closely cooperating with the theoretical foundations. The bottle-necks of the conventional information pro­ cessing should be broken through by new kinds of flexi­ ble functions such as integration of symbol and pattern and learning/self-organization. Merely combining con­ ventional technologies or to make ad-hoc systems for specified tasks is not what is desired .

Some typical real world problems should be tackled by exploring the way of integration of these elemental technologies and for demonstrating their effectiveness. The following two are the important directions to pursue for realization of integrated systems of novel functions:

• Real-world adaptable autonomous systems,
• Information-integrating interactive systems.

The former means flexible autonomous systems that can understand the environment and actively and adap­ tively interact with the real world for partial replacement of human activities in the real world, while the latter means flexible systems which can support and enhance human intelligent capabilities such as problem solving and information creation through enlarged communica­ tion channels between humans and systems.

4.3 Computational bases

RWC involves processing of large volumes of spatia­ temporally distributed information at a high speed, while taking into account their mutual interactions. As a new computational basis to support it, computing sys­ tems which can exploit parallel and distributed process­ ing at several processing levels should be developed. For this purpose, research and development from the follow­ ing three perspectives is important, and the way of inte­ grating those technologies will also be investigated.

• General-purpose massively parallel computing sys­ tems
• Neural computing systems
• Optical computing systems

4.3.1 Massively parallel systems

Implementation of RWC applications is likely to need a computing system which consists of many modules ex­ ploiting parallel and distributed processing at several levels both within and between modules. RWC will probably be realized by some combination of computing paradigms such as concurrent object-oriented process­ ing, dataflow, dataparallel, or neural network, probabil­ ity based processing, etc. These observations show that a massively parallel system is necessary RWC to support the computational power, and it must also be general­ purpose to efficiently execute the multi-paradigms.

The massively parallel system should be flexible itself, adapting itself to application environments for optimal performance while minimizing the workload on the users. The research and development will include the topics in the following:

• Massively parallel architectures,
• Operating system for massively parallel systems,
• Languages for massively parallel systems,
• Environment for system development and program­ ming.

4.3.2 Neural computing systems

In recent years, neural networks have been receiv­ ing attention for their capabilities of learning/self­ organization and many types of flexible information pro­ cessing. However, those networks are still limited to small scale applications, because the neural models used there are very simple, and the learning is mostly based on the back-propagation technique and requires large com­ puting time.

In the RWC program, the possibilities for large scale neural networks will be explored to create flexible infor­ mation processing systems that can operate in the real world. The research and development will include:

• Research on new models,

• Hardware architectures and software environments,
• Development of a large scale prototype system,
• Integration with the massively parallel systems.

4.3.3 Optical computing systems

Light is expected to be a new information medium, because of its extended transmission capacity and mas­ sively parallel processing capability. Optics will provide new device technology as well as new architectures and algorithms in the RWC program which aims at flexi­ ble information processing using massively parallel dis­ tributed processing. The device technology is, however, still immature and has many things to do.

Research topics will be classified into the following categories:

• Optical interconnection,
• Optical neural systems,
• Optical digital systems,
• Environment for system development.

In particular, optical interconnection is expected to be able to eliminate information transmission problems in electronic systems, such as propagation delay, line-to­ line crosstalk, space factors of wiring and mounting, and large power consumption.

5 Organization of R&D

So far, two years of preliminary study have been done in 1989 through 1990 under the research committee on the New Information Processing Technology (NIPT) and some working groups, with participation of more than one hundred researchers in various fields from universi­ ties, national institutes like ETL, and companies. The activities resulted in the final report [MITI, 1991]. FY 1991 was devoted to a feasibility study under the new name of Real World Computing (RWC) program, to­ ward making a master plan [MITI, 1992] . During this period, we had three workshops (Dec. 1990, Nov. 1991, and Mar. 1992) and one international symposium (Mar. 1991), which were open to foreign countries.

5.1 Fundamental Policy

The RWC program is quite a fundamental and challeng­ ing project to open a new horizon of information pro­ cessing toward the 21st century. Therefore, in order to accomplish the ambitious goal, interdisciplinary and in­ ternational cooperation by industries, national institutes and universities is necessary in the related wide research fields. It is also important to secure a flexible scheme of R&D which is adaptable to the development of R&D. For this, the program is to be managed under the following fundamental policy.

• Formation of flexible research organization: Re­ search themes are appropriately allotted so that common-base (such as computational bases) or sys­ tem integration oriented researches are performed in the central laboratory while individual or elemental researches are performed in the distributed labora­ tories, and an organic and flexible link between both parties is secured.
• Introduction of competitive principle: The program introduces competitive principles in the first half­ stage, taking various approaches, and selects the re­ search themes to be investigated in the second half­ stage on the basis of the results of evaluation after the initial five years.
• Interdisciplinary and international cooperation: The program promotes interdisciplinary and inter­ national cooperation to fulfill its basic and chal­ lenging aims, supporting joint researches with na­ tional institutes like ETL and universities, etc. and inviting subcontract applications from domes­ tic/overseas research organizations such as univer­ sities, etc.
• Publication of research achievements: The progress and results of R&D are to be reported and published in domestic/foreign conferences, etc., actively hold­ ing symposia and workshops as well.
• Establishment of infrastructure for research activi­ ties: A high-speed computer network is to be estab­ lished as the infrastructure for internationally dis­ tributed researches, and formation of a flexible re­ search organization as well as exchanges of research results are supported.

To manage the total RWC program, MITI has formed the Evaluation and Promotion Committee (EPC) whose members are from universities and ETL.

5.2 Organization scheme

The organization scheme of R&D in the RWC program is illustrated in Fig. 3.

The RWC Partnership (RWCP) was founded in July of last year (1992) . At the present (at the moment of preparing this manuscript), 13 Japanese companies in­ cluding almost all major electronics firms have joined through the reviewing process by EPC. More that 30 themes were selected as contract research topics.

In October 1992, RWCP founded its own central lab­ oratory, Tsukuba Research Center (TRC), near ETL in Tsukuba City, expecting close cooperation with ETL and inviting about 20 researchers from distributed labo­ ratories of each company.

ETL, which belongs to MITI and has been playing an important role in concept formation of this program, will continue to support and lead the program, sending some researchers to the main positions of TRC and also carrying out its own basic and leading research on RWC with a big group of about 60 researchers.

Like the previous 5G Computer project, MITI will provide a similar amount of total budget (about $ 500 million for ten years). The main part of the budget will be allocated to RWCP, and about 10% to ETL and do­ mestic universities, and not less than 10% to foreign re­ search institutes to promote international cooperation.

There is a modality for foreign researchers to partici­ pate. Foreign companies and non-academic (corporate) organizations are permitted to directly join RWCP, while foreign universities are able to participate either as sub­ contractors or by collaborating in joint research with ETL and RWCP. In the cases of joint research, basically there is no budget flow except the information exchange.

Actually, GMD (Germany) joined as a partnership member, and some other foreign corporate research in­ stitutes in EC and Asia have applied and are under the review process. Unfortunately, U.S. researchers can at present only participate in the field of optical comput­ ing in this RWC program, since the decision was made by the U.S. and Japanese governments based on the sci­ ence agreement which requires large-scale international projects to be coordinated through government channels.

The first call for subcontracts was announced to over­ seas in February this year (1993), and the deadline was the 5th of March. It was in a small scale this time by several reasons due to the first year. More chances for subcontracts are available after this year. Detailed in­ formation about participation is available at the Interna­ tional Relations Department of RWCP (hirairwcp.or .jp ).