Planning Considerations for a Roving Robot with Arm

1. 0 Introduction

Autonomous goal-directed coordination of locomotion, manipulation, a nd sensation a nd percep­ tion in a s emi-natural environment is the capability being sought by the JPL Robot Res ea r ch Program. The initial goal of the program is to demonstrate the integration of sensory a nd moto r func tions in the autonomous p e rformance of manipulation a nd loco­ motion tasks in respons e to global commands issued by an operator. The long-range goal is to develop, test, and display concepts of robot structure, system integration and operation, and machine intelligence for the design and use of adaptive autonomous rna­ chines for advanced space and planetary exploration. The JPL program utili zes results of progress ob­ tained at other institutions engaged in robotics and artificial intelligence work reviewed in Ref. 1. (Ref. 1 contains an extensive list of related litera­ ture.) The robot breadboard itself is a mobile vehicle (similar to that used by the astronauts on the moon) equipped with a six degree - of -fr eedom manip­ ulator (a modified version of the Stanfo rd Electri c Arm, see R e f. 2), a complement of sensors (TV, laser range finder, navigation and guidance sensors, tactile sensors, and, eventually, proximity sensors), and a local mini-computer in communication with remote computers , graphic displays, and operator consoles.

*This paper presents the results of one phase of re­ search carried out at the Jet Propulsion Laboratory, California Institute of Technology, under Contract No. NAS 7-100, sponsored by the National Aero­ nautics and Space Administration.

independent spring action at each wheel. Inputs to the vehicle navigation, guidance, and control system are furnished by odometers mounted on the front wheels (providing vehicle center line distance travel information). a "ruggedized" directional gyro com­ pass (providing directional reference), and wheel drive moto r tachomet:ers (providing information on vehicle velocity).

The manipulator is a modified version of the Stanford Electric Arm described in detail in Ref. 2. It has six degrees of freedom, allowing any desired hand position and orientation in an open or slightly obscured workspace. The reachable set of points (the workspace ) i s within a radius of 52 inches mea­ sured from the origin of the manipulator base refer­ ence frame . (See Fig. 2.) The s i x joints connecting

For Link-Joint Pairs of Arm

the links from the base to the hand are in the follow­ ing sequence: two rotary joints (providing shoulder azimuth and elevation action}. a linear joint (provi­ ding in and out reach action}. and three rotary joints (providing the wrist action) . . The hand is presently a s imple parallel jaw mechanism. The joints are driven by permanent magnet DC torque motors geared directly to the corresponding links . Depen­ ding on the relative position of the links, the arm can handle loads of up to 5 - 8 pounds Earth weight. The arm servo control utilizes analog position mea­ surements from the joint output s and anal og velocity measurements from the motor shafts. Holding torque at each joint is provided by electromagnetic brakes. The arm's structural stiffness and tight servo control can provide hand positioning accuracy within a few tenths of an inch. A suitable articulated and adaptively controlled hand will be added at a later date.

Environment sensing and perception is mainly obtained from two sources: TV cameras and laser ranging. The laser ranging device is a GaAs pulsed mode laser with fast pulse (-10 ns). The beam is pointed by a gimballed mirror and detected by a photomultiplier. Provision s are made for multipulse averaging using analog integration and variable averaging time . The sensing range is tentatively up to 150 feet. The design is based on previous JPL experiments (Ref. 3). Relat e d data handling prob­ lems are treated in Ref. 4. The vision system con­ sists of two identical and optically parallel vidicon TV cameras which will provide" digitized stereoscopic input to both scene analysis and operator display (Ref. 5) . A 7 29 by 7 29 resolution sequential column digitizer furnishes video data for computer-rate digital picture processing and operator display. The TV cameras and laser are mounted on a pan a nd tilt mechanisms referenced to a common coordinate system and w ill be used as an integrated scene anal­ ysis subsystem. Arm-mounted proximity sensors (descr ibed in Ref. 6) and tactile sensors will at a later date augment the environment sensing and perception subsystem.

The proximity sensor is a small (about 0. 3 cubic inch) elec tro-optical device with a small ellipsoid-shaped sensitive volume permanently fo­ cused at a distance of a few inches in front of the sensor. If this proximity sensor is mounted to an appropriate place on the hand, the sensitive vo lume will move with and ahead of the hand at a known dis­ tance relative to a reference point on the hand. A voltage signal will appear when the sensitive volume "touches" a solid surface as the hand approaches the surface. This voltage signal can be used to guide and control the terminal motion of the hand in direct response to sensed relative hand-object position and orientati on. Of course, several proximity sensors can be mounted on the hand, providing several sensi­ tive volumes in a known pattern around the h and and facilitating the design of a versatile conditiona l ter­ minal guidance and control logic for hand motion.

The computing and data handling subsystem architecture is currently based on a remote PDP-10 in the ARPA net as an off- line computer. This will be connected to a local real time computer perfor­ ming realtime robot control and I/O functions. (See Fig. 3.) The remote computer system will be used

- C<WIJ1II

Fig. 3. Computing and Data Handling Subsystem Architecture

to process TV and laser pictures, to construct the "world model," to operate the different subsystem planning programs, and to execute top-level decision­ making programs of the robot executive (REX). REX is described in detail below.

The realtime computer will interface with the robot through input and output units which contain A/D and D/A converters. A Cable Unit will contain the necessary logic devices allowing the robot to be tethered to the CPU via a 50-100 foot cable . TV data from the robot w ill interface directly (via the Video Converter) to the CPU through a separate cable. A disk storage unit w ill be used for fast, random access, mass storage and will serve to store the operating system routines, robot support programs, TV and laser pictures, and status files for the operator terminal. A magnetic tape unit will be used for performing system diagnostics and for entering oper­ ating system programs and any robot subsystem pro­ grams developed in the off-line computer. The operator console will allow operator interaction with the system and development of subsystem support programs.

The operator -robot interface functions will be performed through an IMLAC terminal connected to the realtime computer. (See Fig. 3.) During pro­ gram development, the IMLAC will also be used for simulation studies. The IMLAC terminal includes a CRT display, a teletype, and a read /write cassette recorder.

2. 2 Breadboard Software

The software system architecture is essentially hierarchical with the robot executive (REX) control ­ ling and monitoring the various software subsystems. REX performs problem solving, interacts with the human input command structure, manages the "world model," and calls the major subsystem software modules (vehicle, arm, etc.). The major subsystems are designed to be largely independent of each other. Necessary data concerning the state of the robot and environment used by the various subsystems are fur­ nished through the "world model."

A Master Control Program (MCP) ties the various subsystem programs together and acts as an operating system for them. Present plans are to design the MCP in the remote computer using the mechanisms in SAIL (Stanford Artificial Intelligence Language , Refs. 7 and 8) for the creation and control of concurrent processes. Operating system modifi ­ cations are also planned based on the use of the TENEX paging system (Ref. 9) which makes available interrupts of various types not incorporated into SAIL. Subsystem programs can be written in SAIL or possibly in other languages. SAIL provides easy linkages to FORTRAN and assembly languages.

The total breadboard system is of experimental nature. Thus, the software system is intended to be expandable and evolutionary.

3. 0 Planning Manipulator Motion

This task involves three separate efforts: sys­ tem architecture effects, trajectory planning, and manipulator dynamics and control. These are, re­ spectively, planning for manipulator motion, planning of motion, and execution of planning. Planning for manipulator motion occurs at the system level irlthe selection, design, and placement of robot hardware. Planning of motion involves the selection and imple­ mentationof methods of specifying particular motions and motion constraints. E xecution planning deals with motion control implementation schemes. We now consider each of these three separately.

3 . 1 System Architecture Effects

Placement of the manipulator along the center line of the vehicle about 8 inches from the front edge of the vehicle platform allows a reasonable workspace for the manipulator on the ground (which is 18 inches below the platform) while still permitting access to tools and sample storage bins near the center of the platform. This placement does, however, give rise to several motion constraints; the manipulator can easily collide with the platform, the front edge of the platform, the wheels, and the wheel drive motors, even though the basic vehicle was modified exten­ sively to minimize this problem.

The selection and placement of· sensors give rise to additional motion constraints for the manip­ ulator as well as allowing the manipulator to know its world. The manipulator makes use of both exter­ nal and internal sensors. In the initial configuration of the robot, the primary external sensors to be used by the manipulator are the dual TV and laser range finder. These are used to determine a priori manip ­ ulator targets and are assumed to have sufficient r esolution in the initial simplified robot environment for effective targe t specification. L ater. operation of the robot in richer and more perceptually complex environments rendering the sole use of these sensors open to question will be accompanied by the use of conditional arm control loops regulated by direct inFuts from tactile and proximity sensors. Use of these latter devices in conjunction with an adaptive, articulated terminal effector will permit the arm to respond directly to relevant aspects of the environ ­ ment.

Fig. 4a shows a proximity sensor mounted on the hand,. while Fig. 4b shows the concept of proxim­ ity sensor application for terminal guidance and control of hand motion in "distance seeking" and "distance keeping" modes of operation . A more de­ tailed treatment of proximity sensor application to manipulator control can be found in Ref. 10.

d = UNIFORM DISTANCE BETWEEN A

FIXED POINT ON THE HAND AND

SOLID OBJECTS DURING MOTION

CONCEPT OF PROXIMITY SENSOR APPLICATION FOR CLOSED-LOOP GUIDANCE AND CONTROL OF HAND MOTION

Fig. 4. Terminal Guidance and Control Of Hand Motion Using Proximity Sensing

Our robot -hand application calls for handling a variety of both regular (man-made ) and irregular (natural) objects of different size and weight in var­ ious manipulative tasks. Therefore, a versatile hand-finger mechanism will ultimately be required. An articulated and adaptively controlled hand (using design and control principles similar to those applied in prosthetic hand research; see Refs. 11 and 12) will not require detailed a priori information on ob ­ jects to be handled and will ease the control of many details of a grasping motion since the hand, while gripping an object and monitoring only one actuator, adapts itself "reflexively" to the shape, size, orien­ tation, and weight of the object.

Placement of tactile and proximity sensors on the manipulator itself creates no new obstacles or additional constraints, but placement of the TV /laser head is a serious problem. The present configuration places these sensors on a mast above the arm support post and represents a tradeoff between good viewing angle and keeping out of the way of the manipulator. Even so, it is possible for the manipulator to collide with the TV/laser head, either in normal motion or in switching from a right arm to left arm configura­ tion. However, since the manipulator and TV /laser systems are not run simultaneously in this bread­ board and the TV/laser head is mounted on a pan /tilt mechanism, these sensors can be moved out of the way.

The physical dimensions of the manipulator have been modified from the original Stanford design to permit a greater workspace on the ground. Spe­ cifically, the arm support post has been reduced b y two inches and the extendable boom lengthened con­ siderably .

The initial configuration of the robot thus pro­ vides the tools for deterministic planning of arm motion based on a priori TV /laser data with motion implementation based on feedback from. internal position (pot) and rate (tach) sensors. Only primitive tactile feedback in the initial configuration allows for some conditional modification of plan. In subsequent configurations, proximity sensing and an adaptive terminal effector will permit more flexible and environment-responsive manipulator motion planning. In all cases, rover hardware design constrains ma­ nipulator motion by presenting a series of permanent obstacles to the arm.

3. 2 Trajectory Planning

The term "trajectory" is here meant to refer to some description, partial or complete, of the path that the arm follows.

Trajectory "planning" is the activity preceding arm motion (that is, trajectory execution) , the pur­ pose of which is to constrain or otherwise define that motion. Target and environmental obstacle in­ formation are here assumed to be provided by REX and the "world model. "

The degree to which the trajectory is to respond to external sensing during trajectory execution de­ fines a continuum of trajectory planning. At one extreme is purely deterministic trajectory planning. Any external sensing to be done is performed during the planning stage; only a catastrophe halts execu - tion of the planned path. Internal sensing is used throughout execution to maintain the adherence of actual motion to plan . Deterministic planning as­ sumes a static world during arm motion as well as sufficient a priori knowledge and execution a c curacy capabilities, and limits adaptive control. Towards the other extreme is conditional planning, the nature of which is highly dependent on the specific external sensors used. The benefits of conditional planning are fle xibility of response and possible reduction of planning time at the possible cost of increased r eal ­ time computation requirements.

In the initial configuration, deterministic plan­ ning similar to that used in the Stanford hand-eye project (Refs. 13 and 14) is to be employed. Reasons include the initial lack of proximity sensing, the adequacy of TV /laser a priori information in the initial simplified environment, and the fact that most of the obstacles to arm motion are (in the initial configuration) permanent obstacles known a priori and resulting from the placeme nt of vehicle hardware. Conditional planning is to be implemented and inter­ faced with existing deterministic planning at a later time.

A related distinction concerns the manner in which the trajectory plan is specified. Either a sequence of a few points or the complete time history of the arm (that is, the path) can be planned. Arm motion in a constrained workspace involves path planning.

Deterministic path planning can be performed in joint - variable space or in 3-space. In the former case, the time his tory of each joint is planned; it is the combination of the time histories of the joint vari­ ables that describes the motion of the arm. In the latter case, it is the motion of a particular point on the manipulator (commonly, a point on the hand) that is planned; the required joint variable time histories are derived from the plan. The advantage of planning in joint-variable space is that the plan is formulated more directly in terms of the variables to be con­ trolled during motion. The associated disadvantage is the difficulty in determining where the various links will be during motion, a task required to guar ­ antee avoidance of collisions with the other parts of the robot, the natural environment, or even with the arm itself.

Constraining the fingertips to describe an elliptical ar c is an example of planning in 3 - space. In addition to the difficulties in finding the path de­ scribed by other points on other links of the arm, there is also the problem of determining the kine­ matic sequence of joint variable values required to implement the plan.

In the Stanford Hand-Eye Project (Refs. 13 and 14) , the focus to date has been on deterministic path planning in joint-variable space, with some condi ­ tional planning. Specifically, the time histories of the joint variables have been specified in terms of sequences of polynomials with continuity of joint variable value and its first two derivatives guaran­ teed at the boundary points of polynomials in the polynomial sequence. The number of polynomial segments and specified motion contraints d e termine the total number of coefficients required for a com­ plete quantitative specification of joint trajectory . The JPL robot research program uses a modified version of the Stanford planning algorithm.

Fig. 5a shows the time history of a joint vari­ able described by a cubic, a quintic, and another cubic . It has been found that trajectories using polynomials of degree five or higher typically wan­ der, as shown. This behavior appears in observa­ tion as gross extraneous motion of the arm.

Use of a quartic-cubic-quarti c trajectory re­ veals a somewhat different problem. As shown in Fig. 5b, the desire to assure an appropriate direc­ tion of departure and approach of the terminal effec ­ t o r can be thwarted by the tendency of the quartic ­ cubic -quartic trajectory to over shoot or under shoot its endpoint values. The Stanford Hand-Eye project used both of these polynomial sequences, eliminating overshoot by special code.

A third polynomial sequence, five cubics, is being implemented for the JPL arm. This trajectory appears to minimize the "wander" and "overshoot" problems. Typically, as in Fig. 5c, there is no over shoot; wander, when it occurs, is small.

Obstacle avoidance has been implemented for the JPL arm in two ways . The first method consists of the specification of an additional safe intermediate position for the joint(s) most critically affecting arm motion in the sensitive direction. An additional cubic is required for each additional intermediate position specified. The effect of implementing this type of obstacle avoidance is shown in the trajectory illustrate d in Figs. 6 and 7. Fig. 6 describes joint variable motion and Fig. 7 illustrates the projection of the same planned trajectories in the base coordi ­ nate system, as well as the associated total veloci­ ties.

As seen in Fig. 7a, the task is to lift an object from the ground (from x 0 = 20", y 0 = 30", z 0 = -17" in the base reference frame) and deposit it on the vehicle platform at x 0 = 10", y 0 = -25", z 0 = 2". The hand orientation is also specified at both end points of the trajectory. Further, to aid the deter­ mination of the lift-off and terminal approach phases of the joint trajectories, two intermittent hand posi­ tions are also specified. As seen in Fig. 6, joint trajectories #2 and #5 contain, respectively, 9 and 8 segments due to obstacle avoidance.

A second method of obstacle avoidance is called the "freeway method." Precomputed safe trajecto­ ries ( called "freeways" ) relating commonly accessed points are utilized in conjunction with entrance and exit "ramps" relating planned arm configurations to existing fre eways. This freeway method is poten­ tially useful in avoiding obstacles permanently affixed to the vehicle, such as the TV/laser head and support, the wheels and wheel motors, and the vehicle plat­ form. Presently, there are 14 such permanent obstacles on the JPL robot. The freeway method can be used more frequently if the vehicle is positioned in a predetermined standard manner with respect to objects of intended manipulation.

Obstacle detection is performed by the rela­ tively cumbersome method of checking for collision with all possible objects at various points along the trajectory. Of course, for many obstacles, the safety of several links must be examined. In the case of permanent obstacles , however, the invariant property of the relationship has been exploited to produce a series of increasingly complex tests. Thus, in many cases, simple checks of joint variable values can assure safe motion.

3. 3 Manipulator Dynamics and Control

Execution planning deals with the specification of control laws and the design of control schemes whose implementation will assure that the physical motion of the arm will follow the desired motion.

Arm motion between distant points without prescribed continuous trajectories between the points can simply be controlled by driving each motor at some preset rate and terminating the motor drive at each joint when an appropriate signal (potentiome­ ter or some external sensor ) indicated that the joint position had reached the preset or desired terminal value. Arm trajectories planned in terms of contin ­ uous space -time coordination of joint motions, how­ ever, require that the joints be driven to comply strictly with the planned time histories of joint positions.

Several techniques are available to build a suitable position servo for each joint drive. (See Ref. 15.) An appealing computer-oriented servo technique (used in the Stanford Hand-Eye project) i s to compute the required torque or force as a func­ tion of time for each joint drive, accounting also for gear ratio, efficiency, and possible nonlinearities, and construct the joint position servo loops around the computed non>inal torque or force inputs . (See Fig. 8. )

The dynamics of motion at the six joints of the arm is described by a coupled set of six second order nonlinear differential equations with time - varying (in fact, with state-varying) coefficients. There is no simple proportionality between torque (or force) acting at one joint and the acceleration of the same joint when several joints are in motion simulta ­ neously. Even if only one joint moves at a given time, the proportionality between torque and accel ­ eration is a complex function of the actual configura­ tion of all links ahead of the moving joint and any load in the hand. The total variations in link iner­ tias as seen at the joint drives due to changes in arm link configuration or load in the hand have been calculated for the JPL arm (Ref. 16) and are shown in Fig. 9.

In the case of simultaneous motion of several arm joints, the effective torque (or force) acting at each joint is the sum of a number of dynamic com­ ponents: inertial acceleration of the joint; reaction torques or forces due to acceleration and velocity at other joints; gravity terms. The relative impor­ tance of the various dynamic components related to the planned motion displayed in ·Figs. 6 and 7 and computed for the actual kinematic and inertial param­ eters of the JPL robot research arm is illustrated in Fig. 10. To simplify the displayed variations, only motions and inertias of the first three jotnts are accounted for in the diagrams of Fig. 10. If the speed of motion (or rather, the total time of motion) is changed by a factor n, then the acceleration and velocity dependent torque and force-components of Fig. 10 can simply be scaled by a factor n2.

Two questions are currently investigated (Ref. 16): to what extent should the reaction components be accounted for, and in what form should the state­ varying dynamic coefficients be specified to ease control scheme implementation.

4. 0 Summary

The integration of several robot subsystems into a functioning autonomous adaptive machine re ­ quires significant planning considerations on all system leve ls. The JPL robot research program is currently focused on subsystem planning and design and is in the process of implementing results in hardware and software. The manipulator itself has been manufactured and is undergoing final testing. Vehicle modifications are partly completed. Con­ struction of the TV and laser systems are in pro ­ gress. Selection of complementary sensors and other hardware and acquisition of a realtime control ­ ler (the local computer) are underway. The design and building of the A / D and D/A interface units are also underway. Development of the executive and subsystem software and the m a ster control program have begun. The future concerns are mainly related to the implementation of subsystem execution pro ­ grams, planning for subsys tern interaction, and the integration of subsystems into a unified robot bread ­ board.

The outlined considerations and results related to the design of a self-contained planning algorithm for manipulator contr'ol, when the manipulator is part of a total robot system, suggest conducting fur ­ ther work in two directions: direct path planning in the object space; and truly adaptive manipulator con­ trol through unification of deterministic and condi­ tional elements in the planning algorithm.

--- ~~ ---

Acknowledgment

The preparation of the first part of the paper benefited from discussions with several members of the JPL AI working group.