Remote-Brained Robots
Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence. Invited Speakers. Pages 1593-1605.
© 1997 International Joint Conferences on Artificial Intelligence
Abstract
We introduce our research approach to investigating real world intelligence by building 'Remote-Brained Robots'. The key idea is that of interfacing AI systems with real-world behaviors through wireless technology. In this approach the robot system is designed to have the brain and body separate, both conceptually and physically. It allows us to tie AI directly to the world, enabling the verification of high-level AI techniques which could previously only be used in simulation. For robotics research, this approach opens the way to the use of large-scale powerful parallel computers. For AI, this approach allows experiments with realistic agents, an essential step to the application of AI in the real world. In this presentation we introduce the remote-brained approach, describe some remote-brained robots and discuss experiments.
- [Brooks and Stein, 1993] R. A. Brooks and L. A. Stein. Building Brains for Bodies. MIT AI Memo 1439, 1993.
- [Brooks, 1991] Rodney A. Brooks. Intelligence .without reason. MIT AI Memo No. 1293, 1991. IJCAI
- [Hirose et al., 1983] S. Hirose, M. Nose, H. Kikuchi, and Y. Umetani. Adaptive gait control of a quadruped walking vehicle. Robotics Research : the First Inter- national Symposium on Robotics Research (ISRR1), pages 253-369, 1983.
- [Inaba and Inoue, 1989] Masayuki Inaba and Hirochika Inoue. Robot vision server. In Proceedings of the 20th International Symposium on Industrial Robots, pages 195-202, 1989.
- [Inaba et al., 1993a] M. Inaba, T. Kamada, and H. In- oue. Rope Handling by Mobile Hand-Eye Robots. In Proceedings of the International Conference on Ad- vanced Robotics !CAR '93, pages 121-126, 1993.
- [Inaba et al., 1993b] Masayuki Inaba, Satoshi Kagami, and Hirochika Inoue. Real time vision-based control in sumo playing robot. In Proceedings of the 1993 JSME International Conference on Advanced Mechatronics, pages 854-859, 1993.
- [lnaba et al., 1994] M. Inaba, S. Kagami, K. Sakaki, F. Kanehiro, and H. Inoue. Vision-Based Multisen- sor Integration in Remote-Brained Robots. In 1994 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pages 747- 754, 1994.
- [Inaba et al., 1995a] M. lnaba, F. Kanehiro, S. Kagami, and H. Inoue. Two-armed bipedal robot that can walk, roll-over and stand up. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robot.~ and Systems, volume 3, pages 297-302, 1995.
- [Inaba et al., 1995b] M. lnaba, F. Kanehiro, S. Kagami, and H. Inoue. Vision-Equipped Apelike Robot Based on the Remote-Brained Approach. In Proceedings of the IEEE International Conference on Robotics and Automation, pages 2193-2198, 1995.
- [Inaba et al., 1996a] M. lnaba, Y. Hoshino, K. Na- gasaka, S. Kagami, and H. Inoue. A Full-Body Tactile Sensor Suit Using Electrically Conductive Fabric and Strings. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 450-457, 1996.
- [Inaba et al., 1996b] M. Inaba, T. Igarashi, S. Kagami, and H. Inoue. A 35 DOF Humanoid that can Coor- dinate Arms and Legs in Standing up, Reaching and Grasping an Object. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 29-36, 1996.
- [Inaba et al., 1996c] M. lnaba, S. Kagami, F . Kanehiro, K. Nagasaka, and H. Inoue. Mother operations to evolve embodies robots based on the remote-brained approach. In Preprint of Artificial Life V, pages 278- 285, 1996.
- [Inaba et al., 1996d] M. lnaba, S. Kagami, F. Kanehiro, K. Takeda, T . Oka, and H. Inoue. Vision-Based Adap- tive and Interactive Behaviors in Mechanical Animals using the Remote-Brained Approach. Robotics and Autonomous Systems, 17(1-2):35-52, 1996.
- [Inaba et al., 1996e] M. Inaba, K. Nagasaka, F. Kane- hiro, S. Kagami, and H. Inoue. Real-Time Vision- Based Control of Swing Motion by a Human-Form Robot Using the Remote-Brained Approach. In Pro- ceeding.~ of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 15-22, 1996.
- [Inaba, 1993] Masayuki lnaba. Remote-Brained Robotics: Interfacing AI with Real World Behaviors. In Robotics Research: The Sixth In- ternational Symposium, pages 335-344. International Foundation for Robotics Research, 1993.
- [Inaba, 1996] Masayuki lnaba. Extended Vision with Robot Sensor Suit: Primary Sensor Image Approach in Interfacing Body to Brain. In G. Giralt and G. Hirzinger, editors, Robotics Research: Th e Seventh International Symposium, pages 499-508. Springer, 1996.
- [Inoue and Inaba, 1983] Hirochika Inoue and Masayuki lnaba. Hand eye coordination in rope handling. In Michael Brady and Richard Paul, editors, Proceedings of the First International Symposium on Robotics Re- search (ISRR1}, pages 163-174. MIT Press, 1983.
- [Inoue and Mizoguchi, 1985] Hirochika Inoue and Hi- roshi Mizoguchi. A flexible multi window vision sys- tem for robots. In Hideo Hanafusa and Hirochika In- oue, editors, Proceedings of Second International Sym- posium on Robotics Research (ISRR2 ), pages 95-102. MIT Press, 1985.
- [Inoue and Sato, 1994] Hirochika Inoue and Tomomasa Sato. Humanoid Autonomous System. RWC Technical R eport, pages 109 -llO, June 1994.
- [Inoue et al., 1992] H. Inoue, T. Tachikawa, and M. In- aha. Robot vision system with a correlation chip for real-time tracking, optical flow and depth map gener- ation. In Proceedings of the 1992 IEEE International Conference on Robotics and Automation, pages 1621 1626, 1992.
- [Inoue, 1985] Hirochika Inoue. Building a Bridge be- tween AI and Robotics. In Proceedings of IJCAI-85, pages 1231-1237, 1985.
- [Inoue, 1993] Hirochika Inoue. Vision based robot be- havior:tools and testbeds for real world AI research. In Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), pages 767 •773, 1993.
- [Kagami et al., 1996] S. Kagami, M. Inaba, and H. In- oue. Design of Real-Time Large Scale Robot Soft- ware Platform and its Implementation in the Remote- Brained Robot Project. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1394-1399, 1996.
- [Kanehiro et al., 1996] F. Kanehiro, M. Inaba, and H. Inoue. Development of a Two-Armed Bipedal Robot that can Walk and Carry Objects. In Pro- ceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 23-28, 1996.
- [Kato et al., 1974] lchiro Kato, Sadamu Ohteru, Hi- roshi Kobayashi, Katsuhiko Shirai, and Akihiko Uchiyama. Information-Power Machine with Senses and Limbs(Wabot 1 ). In First CISM-IFToMM Sym- posium on Th eory and Practice of Robots and Manip- ulators, volume 1, pages ll -24. Springer-Verlag, 1974.
- [Kitano, 1993] Hiroaki Kitano. Challenges of massive parallelism. In Proc. of 13th IJCAI, pages 813--834, 1993.
- [Koike and Tanaka, 1989] Hanpei Koike and Hidehiko Tanaka. Parallel inference engine pie64. bit :Paral- lel Computer Architecture, 21(4), 1989.
- [Kuniyoshi and Inoue, 1993] Yasuo Kuniyoshi and Hi- rochika Inoue. Qualitative recognition of ongoing hu- man action. In Proc. of 18th IJCAI, pages 1600-1609, 1993.
- [Kuniyoshi et al., 1989] Y. Kuniyoshi, M. Inaba, and H. Inoue. Teaching by showing: Generating robot pro- grams by visual observation of human performance. In Proc. of 20th International Symposium on Industrial Robots and Robot Exhibition, pages 119-126, 1989.
- [Matsui and Inaba, 1990] Toshihiro Matsui and Masayuki Inaba. EusLisp: An Object-Based Imple- mentation of Lisp. Journal of Information Proce. 9sing, Vol. 13, No. 3:327-338, 1990.
- [Matsui and Sekiguchi, 1995] Toshihiro Matsui and Satoshi Sekiguchi. Design and implementation of parallel euslisp using multithread. Journal of In- formation Processing Society of Japan, 36(8):1885 - 1896, 1995.
- [M.Inaba et al., 1993] M.Inaba, T.Tanaka, S.Kagami, Y.Ishiwata, and H.Inoue. Committed-Choice Lan- guage Application for Remote-Brained Multi-Robots (in Japanese). Proc. of JSME Robotics and Mecha- tronics '93, pages 791-796, 1993.
- [Miura and Shimoyama, 1983] Hirofumi Miura and Isao Shimoyama. Dynamical walk of biped locomotion. Robotics Research : the First International Sympo- sium on Robotics Research (ISRR1 }, pages 303-325, 1983.
- [Moribe and others, 1987] H. Moribe et al. Image pre- processor of model-based vision system for assembly robots. In Proceedings of IEEE International Con- ference on Robotics and Automation, pages 366-371, 1987.
- [Nilsson and Tanaka, 1986] M. Nilsson and H. Tanaka. Fleng prolog -the language which turns supercom- puters into prolog machines. In In Proc. Japanes e Logic Programming Conference, !COT Tokyo, pages 209-216, 1986.
- [Sugano et al., 1985] Shigeki Sugano, Yoshi- haru Tanaka, Toshio Ohoka, and Ichiro Kato. Au- tonomic Limb Control of the Information Processing Robot. Journal of Robotics Society of Japan, 3( 4 ):81- 95, 1985.
References were extracted automatically from the PDF and may contain errors.
Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.
1 Introduction
Robots of the future will act in the real world. To re alize robots with the common sense to do this will, it is generally believed, require ma,ssively parallel processing. This is a problem for those of us who want to do experi ments with robots in the real world today - it is hard to build active, limber, situated robots when they have to carry along heavy brains. Our answer is "remote-brained robots" [Inaba, 1993].
A remote-brained robot does not bring its own brain with the body. It leaves the brain in the mother environ ment, by which we mean the environment in which the brain's software is developed, and talks with it by wire less links. In this 'remote-brained approach', we can de sign and implement a robot with both a large-scale pow erful brain and a lightweight limber body. This opens up the possibility of working with intelligent behaviors of a multi-limbed robot, such as a whole-bodied humanoid, as yet not much studied.
In this framework, a body is defined as the real inter face for a brain to the outer world. The channel from the body to the brain defines all the input to the brain.
The brain is defined as a program which receives the input containing all the sensor information and sends the output for actuation to the body. The research task regarding intelligent behaviors with remote-brained robots is to design and implement a brain program based on a well defined interface between the brain and the body. The remote-brained approach divides the time consuming work of building an integrated robot for in telligence research into interdependent work on body, in terface, and brain.
To put the brain inside the mother environment pro vides us with convenience and important advantages. It allows us to observe the internal states of the brain dur ing experiments. If the robot is a mobile robot, this facility allowing us to probe the brain behaviors is indis pensable for effective debugging of brain software. The brain software is developed in the mother environment which is inherited over generations. It can benefit di rectly from the mother's 'evolution', meaning that the software gains power easily when the mother is upgraded to a more powerful computer.
The research topics with remote-brained robots in clude the mother environment for raising brains, brain software based on different architectures, brain imple mentation on parallel computers, manipulation in han dling flexible objects, multi-robot coordination, recon figurable robot system with distributed modular bodies, adaptive behaviors of quadruped robots, apelike robots and humanoids, and so on. Figure 1 shows the idea of the remote-brained robots. This paper introduces some of them with real experiments.
2 The Inherited Mother Environment
In 1983 we built a hand-eye system, COSMOS (COgni tive Sensor Motor Operation Systems/Studies), which could perform rope handling tasks, such as insert rope-into-ring and tie-knot, using a single arm and a camera with a stereo viewer [Inoue and Inaba, 1983; Inoue, 1985]. Our rope handling experiments allowed us to study some of the essential functions for the integrated hand-eye system. Our computational environment was based on a 16 bit mini-computer with a frame memory and GPIB interface. The visual guidance program was written in terms of look-and-move sequences. While the
I
l
\


robot arm rnoved, the vision system could not watch the scene because of the limited power of the mini-computer. This forced us to develop attention control strategies based on local image processing - by projecting the predicted situation resulting from arm motions, we could determine in advance where vision should watch, making it much more efficient.
As the successor to the COSMOS system, we built COSMOS-II, using networked workstations and a vi sion server. The vision server [Inaba and Inoue, 1989] provides the workstations with visual functions such as multiple parallel-processed visual windows [Inoue and Mizoguchi, 1985) and high-speed line segment detectors [Moribe and others, 1987). The programming language was the object-oriented Lisp, EUSlisp [Matsui and In aha, 1990), which was designed specifically for the needs of robot programming on workstations. This allowed us to demonstrate a new, language-free, robot program min"i approach, "teaching-by~showing" [Kuniyoshi et al., 1989j. These experiments also showed that multiple flex ible visual trackers are essential for observing flexible structured objects, such as the human hand, and for im plementing ongoing action recognition [Kuniyoshi and Inoue, 1993).
We found, however, that a system based on Unix work stations was not adequate for intensive sensor interac tion. After COSMOS-II, we have extended the real world interface of the workstation with multiple real-time pro cessors, in the form of a transputer. A transputer is a micro-processor which can communicate with other transputers through four serial high speed communica tion links. It supports programming facilities for parallel process execution. We have developed a new tracking vi sion system [Inoue et al., 1992) implemented it using the transputer and a correlation chip, and a general-purpose transputer board for motion control. All the worksta tions of the COSMOS-II system now have access to such a 'real world interface'. We can connect such vision sys tems and motion controllers to every workstation and build flexible configurations with transputer networks. This programming environment has allowed us a variety of vision-based behaviors [Inoue, 1993).
The remote-brained approach aims not only to keep such extensibility for both the robot system and the re search environment but also to provide real robots with mobility free from the workstations. The remote-brained robot is intended to freely support growing the research platform and to utilize the software inherited from pre vious versions and to be developed in future extensions of COSMOS.
The research environment for the remote-brained ap proach inherits the software developed in the COSMOS II environment, with many improvements. The robot vi sion server which includes high speed line-segment find ers and multiple visual trackers for attention control is replaced with multiple transputer vision systems. Eu slisp [Matsui and Inaba, 1990) now includes powerful facilities for robot programming: modeling robot body structures, model-based object recognition, network in terface over workstations, transputer interface, window based programming, parallel programming with multi threads [Matsui and Sekiguchi, 1995) and so on.
3 The Remote-Brained Robot System
The advantage of the remote-brained robot approach is that we can use any computer as the brain hardware and apply any software platform for developing the brain software. The design and the performance of the interface between brain and body is the key in the remote brained approach. Our current implementation adopts a fully remote brained approach, which means the body has no computer onboard. The key to simplifing the configuration for multisensor integration in the remote brained approach is the method for multiplexing differ ent kinds of sensor signals on the wireless link. A method for integrating multisensor signals into a video signal is to use an image superimposer which multiplexes several sensor signals into a two dimensional sensor image[Inaba et al., 1994]. The sensor image is defined as all the input from the outer world to the brain.
Figure 2 shows our environment when we started the remote-brained robot project in 1992. Nine different robots are presented in this figure. The tower in the center is the brain interface including a parallel vision system and a motion controller. The cubic box to the left of the tower is the box of transputers which includes 48 T800 transputers. It is connected to the tower through transputer links. The tower and the transputer box are connected to the workstations.
Figure 3 shows the hardware configuration of the sys tem built in our laboratory. The brain base consists of three blocks of vision subsystems and the motion con trol system. Each block of the vision system can receive video signals from cameras on robot bodies through the wireless video link. Two of the vision subsystems are parallel sets each consisting of eight vision boards. The vision board consists of a transputer augmented with a special LSI chip which performs local image block match ing [Inoue et al., 1992]. It can provide a robot body with real time image processing capabilities such as pattern matching, motion detection, and object tracking. The motion control subsystem can handle up to 280 actua tors through 35 wave bands and send the reference values to all the actuators every 20msec.
In remote-brained robot approach, the body design is a question of designing mechanical hardware using mod ular components. The key idea in body development is to use broadly available components to build robot body. When we try to acheive new behaviors with a real robot and it breaks, we can repair it by just replacing the bro ken part. The main electric components of the body of a remote-brained robot are joint servo modules, a con trol signal receiver, a battery for actuators, a camera, a video transmitter, and the battery for the vision systems. The servo module, receiver, transmitter are available for radio hobby toys. The servo module is a geared mod ule which includes an analog servo circuit and receives a position reference value from the motion receiver.
As the brain base system consists of workstation and transputer networks, we have to coordinate the non real time system software on the workstation with the real time system software on transputers. The system re quires a flexible connection between the workstation and the transputer network. We have designed and imple mented the software platform where we can test any type of connections between them[Kagami et al., 1996].

4 Multi-Robots Applications
The wireless connection of a remote-brained robot allows us to build an environment of multi-robots.
4.1 A Multi robot application on a super parallel computer
In the Japanese national project on Fifth Generation Computer Systems, logic programming languages on parallel computers have been investigated intensively. An aim of remote-brained robotics is to allow such so called advanced computer technology to be incorporated into and allow the development of advanced robots.
We have used one of the Committed Choice Lan guages, FLENG [Nilsson and Tanaka, 1986], which arose from the debate and discussion of the Fifth Generation project. This language was designed as a kernel lan ~uage for a fine-grained parallel computer, the PIE64 [Koike and Tanaka, 1989].
What we first wanted to use a parallel computer for was brain software for multiple robots at work. We took soccer as the example. Each robot could have its own brain sofware, and each change in the situation was han dled as an event for each robot body [M.Inaba et al., 1993].
A major advantage of remote-brained robots is that things based on simulation of the natural world can be ported to real robots' brain software without change.

Thus the brain, operating in the real-world-simulating mother-environment, can do well.
4.2 Coordination of Hand Modules and Eye Modules
One of the advantages of the remote-brained approach is the ability to use a mobile body with many DOF. In order to organize so many actuators and degrees of freedom of actuators without decreasing flexibility, the body should be designed to be physically reconfigurable to perform many kinds of tasks.
Our first design was to build a distributed modular robot which has eye modules and arm modules. The concept of the robot is that it can gra..c;p where it wants to grasp and it can watch where it wants to look. Figure 4 shows two eye modules and two arm modules and a view of an eye module observing a hand module. Eye modules have a camera set with pan-tilt mounted on a mobile base with two wheels. Arm modules have a gripper at the end of a 3DOF arm mounted on the same mobile base. The total number of actuators is twenty.
Working with reconfigurable robots raises many prob lems such as identification between eye and hand mod ules, navigation of the hand modules by eye modules, co ordination between hand modules, and so on. As a first step, we have started with a model and vision based ap proach without complete calibration between modules. The model works as a memory to control visual attentions. Each eye module can verify the memory and up date it. During visual guidance of a gripper, each eye module can guide it by sending relative motion command such as left/right or up/down to the control process of the hand module.

Figure (c) of 4 is a view of the mother software which shows the inside of the brain. The view at the left lower shows the predicted image which the eye module is look ing at such as in (b). The brain of this reconfigurable robot has to update the memory according to the visual information. The tracking vision can use the memory to recover from failures of tracking. The identification process of the vision for updating the memory requires such memory based tracking processes.
The software for the brain of the reconfigurable robot requires such memory management. It is hard for each module to have memory storage and maintain it in its own body. During the development of software for recon figuarable robots, the environment can show the memo ries of every mobile module and also the result detected by each eye module to the developer, enabling faster in software development. This is another advantage of the remote-brained approach.
4.3 Coordinations in Rope Handling
Handling flexible objects requires a robot where sensing and action are tightly coordinated. We have developed two sets of mobile hand-eye robots. Each robot has a camera, an 3DOF arm with a gripper and two wheels mounted on a base. The robot is very compact, with the size of the base being 20cm by 20cm. A team of two of these robots can succeed at rope handling [Inaba et al., 1993a].
The remote-brained approach affects the design of the arms. The gripper is small enough that vision can ob serve the target object without occlusion by the grip per. As the wires making up the gripping effector are flexible, the picked object may change orientation. In general, such a gripper is not considered to be an appro priate robot hand, because the robot can't estimate the location of the grasped object. However, a vision-based robot can tolerate such flexibility because vision allows it to know the actual location of the object, not just the gripper location. Thus, an approach based on intensive powerful vision allows us to design simpler robot bodies.
5 Vision-Based Behaviors
The framework of remote-brained robot allows us to carry out different kinds of robotics research especially vision-based experiments. So far, we have developed the realtime tracking vision and applied it for many experi ments with our robots [Inaba et al., 1996d].
5.1 Vision-Based Action Selection in a Quadruped Robot
Another advantage of remote-brained robots is that the bodies can be lightweight. This opens up the possibility of working with legged mobile robots. As with animals, if a robot has 4 legs it can walk. We are focusing on vision-based adaptive behavirors of 4-legged robots, ex perimenting in a field as yet not much studied.
Moreover, we are considering a quadruped robot where two of the legs can be used as arms. Rather than assign ing distinct functions from the start, we want a robot to decide to use legs as arms if required by the situation, as kind of an emergent property (the development of in telligence); this is another problem for futher research.
A quadruped robot is designed to interact with human by catching a ball. It can chase a ball and grab it. Figure 6 shows a sequence of motions and the image which the robot sees. Vision is the only sensor in this interaction. It detects the target ball by template matching, analyzes the relationship between its body and the target ball, and verifies the picking up action.
The robot can show several vision-based adaptive be haviors. When the robot chases a moving ball, it controls the stride of both sides oflegs to change its walking speed and direction. In order to keep tracking the ball in the middle of the view, the robot continuously controls the neck orientation in parallel with walking motion. When the target detection fails, a looking-for-action is per formed and the robot · tries to find the target by block matching by moving its head around.
When the robot tracks the target, it keeps watching the target and measures the distance to the target. If the distance to a static target does not change when walking, it means some obstacles are in the way. If the walking direction does not point to the goal, it means that one of the legs is probably being hindered, thus avoid and climb actions emerge to give adaptive capability in chasing the target.




5.2 Vision-Based Balancing in Humanoid Robot
Figure 7 shows an experiment in balancing. In this ex periment, a human tilts the ground board on which the robot is standing. The robot vision tracks the scene in the front view. It remembers the vertical orientation of an object as the reference for visual tracking and gener ates several rotated images of the reference image. The rotational visual tracker [Inaba et al., 1993b] can track the image at video rate. If the vision tracks the reference object using the rotated images, it can measure the body rotation. In order to keep the body balance, the robot feedback controls its body rotation to control the center of the body gravity.

5.3 Vision-Based Swing Control
When a robot sits on a swing and swings, it requires a fast visual tracker and a free body. The remote-brained design allows us to make an apelike robot perform vision based dynamic control of a swing [lnaba et al., 1996e]. Figure 8 shows the experiment. The robot vision can measure the frequency and the phase of the swing using optical flow generation. In order to accelerate swinging, the robot controls the height of the center of gravity of its body using the visual measurement. Although a hu man can write a control program based on the theory of parametric oscillation, we have succeeded in automatic generation of swing motions on a neural-net based con troller where the weights of the neural network are generated by a genetic algorithm. This is a first step to automatic action acquisition for mechanical animals. As the GA method requires long time trials, we developed a simulation system for the swing world. The simulation program is written in Euslisp and simulates not only the physical motion of the robot and the swing but also vi sual processing with time-base filters. Generally, it is hard to generate accurate optical flows. We adopted a Butterworth-filter to get the motion velocity of the robot body from the optical flows. The visual process with a time-based filter is indispensable to get stable motion control in dynamic actions.

6 Human Robot Interactions
The remote-brained approach encourages us to try build ing whole-bodied robots which interact with humans. We present the principal experiments on interactions done in our platform.
6.1 Improving Robot through Human Robot Competition
We have done an application for improving robots through interactions with human-controlled robots. The example is the computer-human sumo wrestling. Two robots fight until one of them is pushed out of "dohyo" ring or falls down. In this experiment, as shown in Fig ure 9, we control one robot by a computer for fighting against a human controlled robot. It is an experiment of human-computer competition for a reactive game in the real world [Inaba et al., 1993b].
In such a mutually reactive process, a human operator observes the behavior of the computer robot and reacts against it by means of adequate fighting strategies. At the same time, the computer also reacts against the be havior of human controlled one. As a human develops his skill, the computer program must be improved so as to compete with the opponent. When the computer enhances its skill, a human operator must try to find bet ter strategy. In such a way, both sides improve their own strategy through the exercise of mutual reactions. The remote-brained approach helps the developing cycles as the brain of the robot is inside the mother development environment.
6.2 Visual Guidance
Interaction with a human through vision is an essential task for a vision-based robot. The robot has to recog nize the human intention by seeing and performing an action. Figure 10 shows a experiment using visual track ing function. The robot is programmed to follow the target object which the human shows. The actions the robot can select from are waiting, standing up, go for ward/backward, and turn left/right. One of the action is selected by observation of the target object. As the prepared actions include primitive locomotion in any di rection, the human can navigate the robot to anywhere by visual guidance [Inaba et al., 1995b].
6.3 Learning Instructions

Another project is the interactive teaching of gesture learning. Figure 11 shows some gestures and the scheme of learning a mapping between visual patterns shown by a human and the gestures. As the interface with a human is only vision, it is assumed that the robot knows the predefined meta signs, Yes and No, to know whether the requested gesture is right or not.


In the experiment, a human shows a gesture sign to re quest a gesture. When the robot detects a gesture sign, it starts performing a gesture. After the robot performs a gesture corresponding to the presented sign, the hu man verifies the correspondence. If it is bad, the human indicates No by waving the sign horizontally.
7 Behavior Development in Humanoids
Across the ages, one dream of intelligent robotics has been humanoid robots [Kato et al., 1974; Sugano et al., 1985; Brooks and Stein, 1993; Inoue and Sato, 1994]. The remote-brained approach allows us to develop a whole-bodied humanoid. Our approach towards hu manoid robots will be via apelike robots, "apoids". How to evolve from an ape's use of arms and legs to bipedal lo comotion, taking our clue for our 'mother environment for software development' from development over long time spans in the natural world, is another goal of our research.
Figure 12 shows apelike robots. They have different body structures. The apoids in the bottom picture cor responds to (a) .. (d) from left.
The apoid (a) has 16DOF (3 in leg, 4 in arm and gripper, 2 in head). The apoid (b) has the same number
( .......................................................................................................................................... ........... ......................................................................................... 1

of total DOF of the body but the freedom of the leg is extended, each leg has four DOF. The extended degree is added at the ankle and it gives the robot the ability to tilt the body to left and right while standing, which allows the robot to move the center of the gravity of the body over each foot. Thus the robot can perform biped walking in a static way. The apoid (a) can not do such static biped walking. It just performs static locomotion by knuckle walking.
The apoid (c) has 22DOF (4 in arm, 1 in gripper, 5 in leg, 2 in head). It can change walking direction during biped walking because of the extended DOF of the legs. As the arms have more DOF, it can carry an object by holding it horizontally [Kanehiro et al., 1996].
The apoid (d) has 35DOF (6 in arm, 4 in gripper, 6 in leg, 4 in head) [Inaba et al., 1996b]. As the number of DOF in each arm and leg is six, it can control its hand and foot in any position and orientation in three dimensional space, even if it can't change the orientation of the body orientation. This robot can move on a chair and manipulate objects on a desk. One of the goal tasks of this robot is to retreive an object that has fallen off of the dask.
The goal here is to know how the brain for apoid (a) should be raised to accept the body of apoid (b) and so on. Another goal is to know whether there is a brain which accepts all body structures, and whether there is a general method to build such brain. In order to approach these question, we have tried to share the software envi ronment in the mother environment.
7.1 Operations Study for Development
Once we can have a rich platform to investigate various robotics themes by developing real robots, we can do research on the operations for development of a robot [Inaba et al., 1996c].
Figure 13 is shown to explain the relationship between a robot and a mother and the operations done in the mother. Preview simulates the situation of the real environment and Body behaviors according to action values given from Brain. It shows simulated results to Brain. Observe monitors behaviors of Body through real sensors of the mother. Evaluate receives observed behaviors from Preview and Observe and selects ap propriate operations from Teach, Bear, Build.
7.2 Bearing a Brain Through Preview Operation
One example of Preview operation is a case of building a brain to perform vision-based dynamic action of an agent. In the swing experiment as shown in Figure 8, the swing and the virtual body are modeled in a three dimensional geometric model. The Preview ticks away the states of virtual body and environment according to the trajectory given from the brain. The Previewed result is given to the brain as sensing data.
At the first stage, the builder of the apoid body Eval uate the previewed data and Bear the brain based on the model of parametric oscilator. The brain structure is based on a feedback loop architecture. The attention selection is done in both space and time. The watching region is given by the last position of the visual tracker. The apoid controls the angle of neck to view the target successfully. The monitoring time to generate the refer ence value for action is defined by the frequency of the swing. The frequency is also measured by vision.
As the next stage, the builder changes the Evalu ate process and the Bear operation with a GA system. Evaluate sets the fitness of all population of the genes. Bear operation is operation on all population of genes. Brain receives the map from the perception to the ac tion. The perception data is optical flow. This stage closes the loop of Preview, Evaluate and Bear with out a human operator.
7.3 Developing Getting Up Motions
Research on biped walking robots has focused on the dy namics of walking, treating it as an advanced problem in control [Miura and Shimoyama, 1983]. However, focus ing attention on intelligent reactivity in the real world, it is more important to build, not a robot that won't fall down, but a robot that can get up if it does fall down [Inaba et al., 1995a]. The behaviors of getting up are in dispensable for robots. The sequences shown in Figure 14 are examples.


Those behaviors are developed through Preview with a virtual body and environment simulator. Preview can calculate the balance of the robot body. As shown in the figure, the brain can estimate the contact points between the body and the ground by calculating the center of its body gravity and the configuration of the body lims.
In the case that the effect of the difference between the virtual body and the real body is large, the Bear opera tion through the Observe process is indispensable. The task here is to build a good observer that can recognizes the robot body. This flow through Observe to Bear is a good way to investigate emerging properties for real embodied agents. The remote-brained approach with a multi-degreed body such as an apoid will help with this task.
7.4 Developing Walking Motions
In order to focus attention on flexible mobility in the real world, it is important to build, not only a robot that can run, but also a robot that can stand and walk even in statical way. Figure 15 shows experiments of walking done by the apoids ((a),(b) and (c) respectively). A se quence of walking is generated through Observe and Bear loop done by a human at the first stage. If the mother has only the virtual body simulator inside, it is able to build a teaching loop through the Preview. This makes the Bear operation interactive. The next stages are to make methods for Teach, to make Observe by computer vision and to make Preview by computer pro gram.
In order to think about a loop through the Build op eration, the mother can make use of the apoids explained above. That is, if the mother has computer vision-based Observe, it can change the body structure by select ing one from already developed apoids. As the remote brained system does not place the brain inside the body, the mother system can continue raising the same brain after switching the body. This is another feature of the remote-brained system. In this case, the system struc ture becomes a variation of a system with multiple bod ies. The software of apoid (d) is designed using the soft ware developed for apoids (a),(b),(c).
7.5 Interaction Through Whole-Bodied Tactile Sensor Suit
The remote-brained approach based on the integrated sensor images allows us to do experiment not only in vision-based but also multi-sensor based interactions. If we can get a general methodology to give a large num ber of distributed tactile sensors to any robots, it en courages the remote-brained approach in developing a variety of robot bodies with tactile sensors. Our ap proach is the 'sensor suit' approach which covers the whole body of the robot[Inaba, 1996]. We have devel oped a method to build a sensor suit usin& electrically conductive fabric(ECF)[Inaba et al., 1996aj. The ECF is soft fabric electrically plated with Ni and Cu which is usable as a soft and flexible metalic sheet.
The key issue in building a suit with a large number of sensitive regions is how to wire the cable from the sensing segments to the processing subsystem. Our approach is
(b

based on two methods: sewing with ECS and the video multiplexer to pass on all signals. As wiring is done by electrical conductive string(ECS), it keeps the suit flexible enough to allow movement of the body.
Figure 16 shows an experiment in sensor suit based interactions with humans. (a) is the tactile sensing re gion for the right arm. (b) is the real sensor image of the robot sensor suit when the human holds the right upper arm of the robot. In this experiment, if a human touches a left or a right arm, the robot looks in the corresponding direction to start interactions with the human. The pro gram for this behavior monitors the sensing regions and detects pattern changes in these regions. If the change is larger than a threshold, it decides whether to move its head to look at the human.
8 Concluding Remarks
This paper has presented a new framework, the 'Remote Brained Approach', as a step towards future robotics, and illustrated it with real examples. The wireless con nection between a brain and a body allows us to do a variety of vision-based experiments. Software for vision based behaviors can be trained by playing games with humans; this and other software can be kept in the mother environment and inherited by the next genera tion of robots. Further, this approach allows us to design new types of robot body, such as the apelike body.
Regarding the question of whether the inference and representation techniques of AI are useful for robots in the real world, it seems the answer will depend on how AI is interfaced to the real world. The remote brained approach allows us to tie AI directly to the world, en abling the verification of high-level AI techniques which could previously only be used in simulation. For robot research, this approach opens the way to the use of pow erful parallel computers. For AI, this approach allows experiments with realistic (indeed, real) agents, an es sential step to the application of AI in the real world.
The technology underlying this approach is of course wireless communications. The driving force behind this technology has been the needs of human communications. We are now witnessing the dawn of an age where wireless communications will surround everyone, expanding each individual's scope of activity, and chang ing the way people think. It will have the same effect on robots.
Wireless permits robot bodies free movement, but it also changes the way we conceptualize robotics. In our laboratory it has enabled the development of a new re search environment, better suited to robotics and real world AI.
There has been a missing link in research, between 'AI which couldn't survive if embodied in the real world' and 'robots with feeble intelligence'. Remote-Brained Robotics opens the way for engineering advances which will bridge the gap.
@inproceedings{ijcai1997p232,
title = {Remote-Brained Robots},
author = {Inaba, Masayuki},
booktitle = {Proceedings of the Fifteenth International Joint Conference on
Artificial Intelligence, {IJCAI-97}},
publisher = {International Joint Conferences on Artificial Intelligence},
editor = {Martha E. Pollack},
pages = {1593--1605},
year = {1997},
note = {Invited Speakers},
url = {https://ijcai.org/proceedings/1997/232},
}
TY - CPAPER TI - Remote-Brained Robots AU - Inaba, Masayuki T2 - Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, IJCAI-97 PB - International Joint Conferences on Artificial Intelligence PY - 1997 SP - 1593 EP - 1605 UR - https://ijcai.org/proceedings/1997/232 ER -Copied
Machine-readable versions of this paper, for tools and AI agents: