Action Selection for Single- and Multi-Robot Tasks Using Cooperative Extended Kohonen Maps

1 Introduction

A central issue in the design of behavior-based control archi­ tectures for autonomous agents is the formulation of effective action selection mechanisms (ASMs) to coordinate the be­ haviors. This paper describes a neural network-based ASM for autonomous non-holonomic mobile robots. Our motiva­ tion is to develop a motion control strategy that can perform distributed multi-robot surveillance in unknown, dynamic, and unpredictable environments. By implementing the ASM using an assemblage of self-organizing neural networks, it in­ duces the following key features that significantly enhance the agent's action selection capability: self-organization of continuous state and action spaces to provide smooth, effi­ cient and fine motion control, and action selection via the cooperation and competition of Extended Kohonen Maps to achieve more complex motion tasks.

2 Action Selection Framework

Our proposed ASM, termed Cooperative Extended Kohonen Maps (EKMs), is implemented by connecting an ensemble of EKMs. An EKM extends the Kohonen Self-Organizing Map. Besides encoding a set of input weights that self-organize in the sensory input space, the neurons also produce outputs that vary with the incoming sensed inputs. Our implementation extends the work of [Low et a!., 2002] by connecting several EKMs to form cooperative EKMs. These neural networks co­ operate and compete to produce an appropriate motor action for the robot to approach targets, negotiate unforeseen, possi­ bly concave, obstacles, and keep away from robot kins when it is tracking moving targets (Fig. I).

Our ASM framework consists of four types of EKMs: tar­ get localization, obstacle localization, robot kin localization, and motor control EKMs. In the presence of a target, neurons in the target localization EKM, which encodes target location in the local sensory input space U', are activated (Fig. la). A target field with the shape of an elongated Gaussian is pro­ duced (Fig. 1 b) such that the neurons at and near the target location have the strongest activities. The elongated target field is crucial to the robot's avoidance of concave obstacles.

Similarly, the presence of an obstacle activates neurons in the obstacle localization EKMs. The neurons in these EKMs at and near the obstacle locations will be activated to produce obstacle fields (Fig. 1 c). These obstacle fields are stretched along the obstacle directions such that neurons beyond the obstacle locations are also inhibited to indicate inaccessibil­ ity. Robot kin fields are activated in a similar way in the robot EKMs in the pre sense of robot kins.

In activating the motor control EKM, the obstacle fields are subtracted from the target field (Fig. I d) . If the target lies within the obstacle fields, the activation of the motor con­ trol EKM neurons close to the target location will be sup­ pressed. Consequently, another neuron at a location that is not inhibited by the obstacle fields becomes most highly ac­ tivated (Fig. ld). This neuron produces a control parameter that moves the robot away from the obstacle. While the robot moves around the obstacle, the target and obstacle localiza­ tion EKMs are continuously updated with the current loca­ tions and directions of the target and obstacles. Their interac­ tions with the motor control EKM produce fine and smooth motion control of the robot to negotiate the obstacle and reach the target. In the case of multi-robot tracking of multiple tar­ gets, multiple target fields and robot kins fields are activated. The robots act like highly repulsive obstacles to other robots, thus separating them from each other.

3 Experiments and Discussions

Two qualitative tests were conducted to demonstrate the ca­ pabilities of cooperative EKMs in performing complex mo­ tion tasks. The experiments were performed using Webots, an embodied simulator for Khepera mobile robots, which in­ corporated 10% noise in its sensors and actuators.

The environment for the first test consisted of three rooms connected by two doorways with unforeseen static obsta­ cles (Fig. 2). The robot began in the top corner of the left­ most room and was tasked to move into the narrow corner of the right-most room via checkpoints plotted by a planner. The robot with cooperative EKMs was able to move through the checkpoints to the goal by traversing between narrowly spaced convex obstacles in the first and the last room, and overcoming an concave obstacle in the middle room. A robot with potential fields would be trapped by these local minimas.

The second test (Fig. 3) illustrates how two robots endowed with cooperative EKMs cooperate to track four moving tar­ gets. When the targets were moving out of the robots' sensory range, the robot below chose to track the two targets moving to the bottom left while the robot above responded by track­ ing the two targets moving to the top right. In this manner, all targets could be observed by the robots. This test shows that the two robots can cooperate to track multiple moving targets without communicating with each other.

Two quantitative tests were conducted to determine the overall tracking performance of the robot team based on the following performance index: T

where N is total number of targets, n is number of targets being tracked at time t, and the experiment lasts T amount of time. For both tests, Nand T were fixed respectively as 10 targets and 1000 time steps at intervals of 128 ms.

The first test compared the mean observation coverage of robots adopting four different tracking strategies: coopera­ tive EKMs, potential fields, fixed deployment, and random deployment. The environment or arena was an enclosed obstacle-free region that varied in size. The mobile tar­ gets were forward-moving, obstacle-avoidance vehicles that changed their direction and speed with 5% probability. Five robots, each with target and robot sensing radius of 0.3 m, were deployed in this task. The fixed deployment approach distributed stationary robots uniformly over the arena. The random deployment approach allowed the robots to move randomly in a manner similar to the moving targets. Test re­ sults in Fig. 4(a) reveal that, in very large arenas, tracking strategies that respond dynamically to targets' motion (co­ operative EKMs and potential fields) are significantly better than those that do not (fixed and random). In particular, co­ operative EKMs offered the highest observation coverage as it could overcome local minimas posed by targets and robots.

The second test compared the mean observation coverage of the cooperative-EKM robots with different sensing ranges and number of robots. The size of the arena was 6.4 m 2 , which corresponded to the largest arena used for the first test. Test results in Fig. 4(b) show that observation coverage in­ creases with increasing number of robots and sensing range.