Workpiece Orientation Correction with a Robot Arm Using Visual Information

WORKPIECE ORIENTATION CORRECTION WITH A

ROBOT ARM USING VISUAL INFORMATION

John R. Birk, Robert B. Kelley, Vivek V. Badami

Department of Electrical Engineering University of Rhode Island Kingston, RI 02881

If robots had the capability of supplying ma­ chines with workpieces which are not preoriented, there would be a substa.ntial increase in the num­ ber of robots used in industry. A significant contribution to this development would be a method which yields a trajectory whereby such a robot held workpiece can be transported from its recep­ tacle to a machine fixture without a collision, An algorithm is described to find such a trajec­ tory. This algorithm employs refinement of an in­ itial estimate of the position and orientation of a robot held workpiece. The refinement procedure uses visual information. With the refined esti­ mate of the hand-to-workpiece transformation, the workpiece can be moved through a safe trajectory, which was established during a prior instruction phase.

The algorithm is based on linear perturbation theory. This means that the change in a particu­ lar visual feature on the workpiece can be ap­ proximately expressed as a linear function of the perturbations in workpiece position and orienta­ tion (pose) relative to a nominal workpiece pose. Thus [Birk, ~ ~. 1976], ~f = P( 6p) where 6f = change in visual feature values, 6p = perturba­

tions in workpiece pose, and P = Jacobian matrix,

During the instruction phase of the algo­ rithm when safe workpiece trajectories are de­ fined, the Jacobian matrix for each nominal pose is determined. First, the nominal pose is assumed by the robot held workpiece as it is presented to two TV cameras. The robot arm joint values, fea­ ture values, and nominal pose are recorded for fu­ ture use. Useful visual features include the cen­ troid, the area and the eigenvectors of the binary workpiece image. The workpiece coordinate system is then subjected to a sequence of known perturba­ tions, which may be accomplished by incrementing robot arm joints. For each movement, the corre­ sponding changes in feature values are measured. This permits the Jacobian matrix for that nominal pose to be determined. This procedure is repeated for each specific nominal pose along the trajec­ tory,

The execution phase algorithm is used to re­ fine the estimate of the hand-to-workpiece trans­ formation (hTw)· After a workpiece has been ac­ quired from a receptacle, hTw is assumed to be known approximately. To verify this, the differ­ ences in image feature values from those extracted during instruction are computed. If these differ­ ences are within the bounds of the linear region which were found during instruction, the refine­ ment algorithm can be applied.

The perturbations in workpiece pose c'an be calculated from 6p = pt( 6f), where pt = (pseudo) inverse of P. 6p defines a correction transforma­ tion matrix, Tc, and therefore can be used to cor­ rect the hand-to-workpiece transformation (hTw)c = (hTw)Tc-1· To ensure smooth convergence, a new hand-to-workpiece transformation can be obtained by (hTw>new = cx(hTw)c + (1-cx)(hTw) where 0 :s: ex :S: 1. If the robot arm joint values were adjusted t o re­ flect this updated estimate of hTw, then the work­ piece pose should be closer to that specifie~ dur­ ing instruction.

After moving the workpiece to a new pose based on (hTw>new· visual information is again ex­ tracted. The correction procedure i s repeated, testing for convergence, until the deviations which are used to update the hand-to-workpiece transformation have been reduced to all f all with­

in a tolerance zone.

To servo the workpiece through a safe trajec­ tory which was established during instruction, the deviation of (hTw>new from hjTw, the hand-to­ workpiece transformation for the jth legal way in which the workpiece was held during instruction, is found. If this deviation is small, it is as­ sumed that the arm structures will not collide with the workstation. Hence, the following equa­ tion can be used,

where OThek is the robot base-to-hand transforma­ tion for the kth trajectory point during execu­ tion. The robot arm joint values may be obtained from OThek· The arm joint solution procedure i s specific for different kinematic configurations.

The selection of visual features and proce­ dures to increase the rate of convergence of the estimate of hTw are issues still to be resolved, The status of experimental results will be report­ ed at the conference.

Acknowleds_ement

This material is based upon research partially supported by the National Science Foundation under Grant No. APR74-13935. Any publications generated as a result of the activities supported by this grant may not be copyrighted without the specific written approval of the Grants Officer and the authors. Any opinions, findings, and conclusions or recommendations in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation,

Reference

J. Birk, R. Kelley, V. Bada~i, R. Brand, N. Chen, D. Seres, R. Silva, and L. Wilson, "Robot Computa­ tions for Orienting Workpieces - Second Report," Grant APR74-13935, Department of Electrical Engi­ neering, University of Rhode Island, Aug. 1976,