--- title: "Grasp Planning from Human Prehension" authors: - "Thea Iberall" doi: "" url: "https://ijcai.org/proceedings/1987/233" pdf: "https://ijcai.org/proceedings/1987/0233.pdf" venue: "Proceedings of the Tenth International Joint Conference on Artificial Intelligence" year: 1987 pages: "1153-1155" track: "Robotics" keywords: --- # Grasp Planning from Human Prehension Thea Iberall *Proceedings of the Tenth International Joint Conference on Artificial Intelligence*, pages 1153-1155. ## Abstract The study of the human hand in prehensile movements can offer insights into th e d es ign of robot grasp planners . A d es ign for a. two-level hi erarchical grasp planner is presented. The first phase maps an object and task representation into a grasporient ed description of the task. The second phase maps this description into an opposition space, which captures the available forces and degrees of fr ee dom of the hand. As th ese mappings a.renotu niqu e, constraints acting on the hand/object/task interaction are discussed. ## Full text ## I. Introduction Current research into the nature of robot task specifica­ tions has demonstrated the need for hierarchical systems [Lozano-Perez and Brooks 1985, Vijaykumar and Arbib 1986]. In Vijaykumar and Arbib [1986], four different types of planners were identified. A strategic planner deals with high-level; more global, decisions, breaking down high-level task specifications into more primitive commands. A tac­ tical planner then deals with lower-level, more local, deci­ sions, performing a subset of primitive commands. Other primitive commands go to the path planner, necessary for moving the manipulator through a cluttered space. Finally, a grasp planner determines a proper grasp, based on the primitive 'grasp' command coming from the strategic plan­ ner. A task plan is produced by first searching for rela­ tionships that constrain the solution. In a similar planning system, Lozano-Perez and Brooks [1985] produce a skeletal plan, and then look towards refining it through application of constraint knowledge. By studying the interaction between the central nervous system (CNS) and the human hand within the context of prehension (i.e., grasping), w.e hope to isolate issues for grasp planners for dextrous robot hands. The shape that the hand takes on during prehension depends on a vari­ ety of complex issues, involving object and task proper­ ties, hand and arm capabilities, and even CNS features. When observing human reaching and grasping behaviors, one notices both variability and invariances. Examples of invariances include the typical S-shaped trajectory of the arm [e.g., Jeannerod 1981], the two-phase velocity profiles noted by [Jeannerod 1981] and modelled by [Arbib et al 1985], and the opening of the hand larger than the obj­ ject to be grasped [Jeannerod 1981]. However, human arm and ha.nd movements also contain a great deal of variabil­ ity [e.g., MacKenzie and Marteniuk 1985]. Variability is also observed at a higher level, in the multiple strategies people use to grasp objects. Yet, even at this high level, stereotypical hand postures have been noted in the litera­ ture [Schlesinger 1919, Napier 1956]. Our approach, therefore, to designing a grasp planner for human prehension is to account for invariances and vari­ aiblity, identifying and resolving the constraints acting on these behaviors. Both the path planner and the grasp plan­ ner are limited both by internal constraints as well as con­ straints that arise from their interaction. The arm, while following some internal kinematic and dynamic constraints [Hollerbach and Flash 1982, MacKenzie and Marteniuk 1985, Marteniuk et al1987], cannot be directed anywhere; instead, it must be directed towards a location that makes sense for the prehensile posture. Such hand constraints, as finger pad locations, lengths of fingers, flexibility of liga­ mentous and tendonous structures, etc., all contribute to determine what that planned location should be. In the next section, a human hand-oriented grasp planner is discussed, in isolation for now from path planning issues involving arm and body effects. In the third section, some of the constraints acting at the hand vs object level are analyzed. ## II. Grasp Planner In order to develop a grasp planner, both an object rep­ resentation and a task representation are needed. Further­ more, hand variables that will be modified during a partic­ ular motion must be described. Between the input and the output, the mapping from the object/task representation to the hand also must be described. In Figure 1, a block diagram of a grasp planner is shown. Figure 1: A Two-Level Grasp Planner ![Image](https://ijcai.org/proceedings/1987/fulltext/233/image_000000_7dfe0e49bf3c9563b25ec7ea60ae8821d76be151f78cfba33bab6d814a81d203.png) The grasp command is sent from the strategic planner, along with a pointer to the object to be grasped and the context within which it will be grasped. Information about the object is available in an object database. For exam­ ple, if the object to be grasped is a hammer, then the com­ mand GRASP(HAMMER, SWING\_UP \_DOWN) would tell the grasp planner that the object is the one in the database identified as HAMMER. The task for which it will be used involves the SWING\_UP \_DOWN degree of freedom. (See Lyons [1985] and Vijaykumar and Arbib [1986] for other context information). For the object representation, sys­ tems that select the correct object features that can then be mapped into a robot hand posture have previously been discussed [Lozano-Perez and Winston 1977, Lyons 1985]. While Lozano-Perez and Winston [1977] define objects in terms of solid geometries (e.g., cylinders, holes, etc), Vi­ j aykumar and Arbib [1986] define objject features in terms of surfaces (e.g., cylindrical surfaces, rectangular faces, etc). We find this latter approach to be more useful for a grasp planner, because the hand must grasp surfaces. An obj­ ject representation for a large steel hammer, based on the specifications of Vijaykumar and Arbib [1986], would be: | (hammer-! | (hammer-! | (hammer-! | |------------------------------|------------------------------|-------------------------| | 276mm) (length | | | | (width | 83mm) | | | (height | 29nun) | | | (weight | .45kg) | | | (material | 'steel) | | | (features (f1 | f2 f3 f4 f5)) | f2 | | (fl | (fl | (fl | | (circular-face | (circular-face | (circular-face | | ( adj-features (£2)) | ( adj-features (£2)) | | | (diameter | (diameter | 31mm) f1 | | (normal-axis | (normal-axis | '-1 axis) | | (center | (center | (-261.5mm 0 0)))) | | (cylindrical-surface | (cylindrical-surface | (cylindrical-surface | | ( adj-features | ( adj-features | (fl f4)) | | (height | (height | 24 7nun) | | (diameter | (diameter | | | ( smf-normal-axis '-w | ( smf-normal-axis '-w | 31nun) | | | | axis) | | (center (-138nun | (center (-138nun | | | | (circular-face | 0 0)))) | | | adj-features (diameter (f4)) | 29mm) | | (normal-axis | (normal-axis | '-w axis) | | (center (0 | (center (0 | | | -41.5mm 0)))) | -41.5mm 0)))) | -41.5mm 0)))) | | (cylindrical-surface | (cylindrical-surface | (cylindrical-surface | | ( adj-features (f2 f3 | ( adj-features (f2 f3 | f5)) | | (height (diameter | (height (diameter | 83mm) 29mm) | | (surf-normal-axis (center | (surf-normal-axis (center | '1 axis) | | (circular-face | (circular-face | (14.5mm 0 0)))) | | (not·mal-axis 'w | (not·mal-axis 'w | 29mm) axis))) (0 41.5mm | | (center (antic-force-axis 'w | (center (antic-force-axis 'w | 0)) axis)))) | Once a command is sent, the grasp planner maps the grasp command into hand variables. A two-step process is visualized to accomplish this task (see Figure 1). In the first phase, the grasp command parameters are mapped into a more useful, grasp-oriented, set of task requirements. Task requirements consist of both functional constraints and physical constraints [Iberall et al 1986, Iberall 1986]. Functional constraints are less a property of the object to be grasped, and more a property of the manner in which the object will be used within the task. A fundamental func­ tional constraint on most prehensile tasks is that the object not be dropped. The posture used by the hand during the task must be able to overcome the anticipated forces acting on the object. Initially, during the lifting phase of a task, the initial force is likely to be just the weight of the object. Later, during the actual task, other forces such as addi­ tional inertial forces come into play. Underlying these func­ tional constraints are physical constraints, that are based both on properties of the object as well as on properties of the hand. In Iberall [1986], we showed how task require­ ments could be described in terms of one or more oppo$i­ tion vector$. Opposition vectors exist between opposable surfaces, and task-related DOFs, forces, and torques all are described relative to an opposition vector. For example, the center of mass of a hammer is somewhere in or very near the head of the hammer. When one picks up a ham­ mer by its handle, a torque therefore arises and it can be measured relative to the opposition vector. The opposition vector extends through the handle between hand surfaces. The swinging of the hammer through the air will cause the opposition vector to both rotate and translate. This swing­ ing will also, of course, generate other torques acting upon the opposition vector. The internal task requirements for lifting the hammer, lying cin a support that allows access to the handle, would be as follows: ![Image](https://ijcai.org/proceedings/1987/fulltext/233/image_000001_9c8ee577c567e7a0794b706b9b38000c061c71b3ebb565c99918457180b53ca7.png) In the second phase of the grasp planner (Figure 1 ), these internal task requirements are mapped into specific hand variables. In lberall [1987a, 1987b], it was shown that hand postures could be described as combinations of three basic oppo$ition$ (pad, palm, and side), with an opposition be­ ing defined by the axis along which the force was being ap­ plied relative to the palm. The opposition occurs between the surfaces of virtual finger$ [Arbib et a/1985], which are groups of real fingers acting together in applying a force against other fingers or against task torques. The particu­ lar shape chosen by the hand is called an oppo$ition $pace ~berall et al1986], which defines the functional capabili­ tites of that hand posture for executing a stable grasp and manipulating the object. As seen in Figure 2, a hand is shown grasping two differ­ ent size hammers, with a simple vector notation showing the virtual fingers being used in the oppositions. In Figure 2a, two opposition vectors are shown for a medium-sized hammer. The vectors go through the same opposable sur­ faces but are positioned differently along the handle. As the hand preshapes (seen in the left side of Figure 2a), the opposition space thus formed is the one where. side op­ position (between the thumb as VF1 and index finger as VF2) occurs with palm opposition (between the palm as VF1 and all four fingers as VF2). In Figure 2b, we see the mapping for a larger and heavier hammer. One opposition vector is used, with palm opposition occurring between the palm (VF1) and the four fingers (VF2) . In this latter case, the thumb is pressing against VF1, helping VF2 to create more force to counteract the anticipated large task-related torques. Figure 2: Preshaping and Grasping Two Different Sized Hammers. ![Image](https://ijcai.org/proceedings/1987/fulltext/233/image_000002_21665107e09374aa9d9939a926d5fc50e98aef98b3228d499c79c1e73824ce22.png) Objects of course have an infinite number of opposition vectors. However, only some of these locations are useful for the task for which the object will be used. The point of the internal task requirements is to discover those useful lo­ cations in a way that will capture the person's intent for the object as it relates to the object's task-significant proper­ ties. However, neither the object-to-task-requirement map­ ping nor the task-requirement-to-opposition-space mapping (which is basically an inverse kinematic mapping) are one­ to-one mappings. In order to approach a functional speci­ fication of these mappings, we look towards capturing the constraints acting on both phases. In the next section, the sources for some of these constraints are analyzed. ## III. Constraints on Grasping Constraints on prehensile behaviors arise from a variety of sources. The hand and wrist bones have been shaped through millions of years of evolution, leaving modern man with a skeletal support and articulation system useful for enhanced object manipulation. Extrinsic muscles in the forearm send tendons into the hand, thus allowing the hand to be relatively small [Tubiana 1981]. Many of these ex­ trinsic muscles have common origins in the forearm, thus causing constraints on fractionated finger movements. Lig­ aments as well cause constraints on finger movements. For example, abduction of the fingers at the metacarpopha­ langeal joint is only possible when those ligaments are re­ laxed. Thus, anatomical con~traint~ arise from the size and shape (length, width, etc) of the bones, the shape of the articular surfaces between bones, the location of tendon in­ sertions and origins, the location of passive ligaments, the location and nature of skin specialized for prehension (with glands and epidermal ridges), and the size of the hand's surface areas (of the palm, o{ the finger pads, etc). Biome­ chanical con~traint~ arise from the stresses placed on these same bones, tendons, muscles, and ligaments. These in­ clude relative mechanical advantages of the various ten­ dons, directions of movement of the fingers, the amount of force that can be applied by different hand configurations, and the way that these forces can be brought to bear. Other constraints arise from the object (object property con~traint~), and its anticipated interaction with the hand (interaction con~traint1). Object property constraints arise because objects have dimensions, such as length, width, and height. The size along these dimensions produces con­ straints on the location where an object can be grasped. If an object is larger than the hand along one of these di­ mensions, then any opposition vector along that dimension is not useful. If the object is very much smaller than the hand surfaces (as is a thread, for example), then the hand is limited by perceptual con~trainu that require additional information for discriminating the tactile sensations. Ob­ jects also have shape. A complex object can be divided into object features, each of which then have dimensions (as was done in [Vijaykumar and Arbib 1986]). Small features on large objects do not generally make useful locations for grasping [Popielarczyk 1987]. If an opposition vector is cho­ sen through a small feature on a heavy object (for example, through the knob on a radio), the arising torque produced by lifting will most likely exceed the available force at the fingers. Of course, with large objects that specifically have small features carefully placed for grasping (e.g., handles placed above the center of mass on heavy boxes), this is not the case. In addition to the weight consideration, the size and placement of features puts limits on the type of op­ position possible (e.g., palm opposition could not be used in grasping the knob). During the task (object lifting and manipulation), forces and torques act on the object. The weight and distribu­ tion of mass create constraints on the prehensile activity. A torque will arise if the opposition vector chosen fails to pass through a vertical line through or above the center of mass of the object. Moments that tend to twist the op~ position vector can be cancelled by increasing the friction between the fingers and object, either by applying more active force or else increasing the amount of skin surface against the object. If the weight of the object is not offset by (at least) an equal and opposite force, the object will not be lifted. This force can be supplied in a direction normal to the weight vector, with its magnitude affected partially by the coefficient of static friction. This latter value de­ pends on the material and texture of the object surfaces, as well as on the hand surface's frictional responses [Jo­ hanssen and Westling 1982]. Frictional characteristics can enhance interactions, thus reducing the amount of active force necessary in the prehensile configuration (e.g., fewer real fingers in a vi.rtual finger). If the surface texture is smooth, then any opposition vector between the surfaces will have a reduced frictional component. If the pads (with their epidermal ridges and greasy lubricant) are not used (with an appropriately larger applied force), the object will likely slip out of the grasp. A surface also has a measure of compliance, affecting the amount of force that can be applied against the surface. Another surface feature is the temperature of the object, which provides constraints in terms of the acceptable range for contact with human skin. Objects and their features have acce~1ibility con~trainu. Some object features are unavailable for grasping because of other objects. An example is the bottom of an object not being accessible because of the table or the surface sup­ porting it. In this short analysis, major constraints arising from the hand (anatomical, biomechanical, and perceptual), the ob­ ect (dimensions, shape, feature accessibility), and their in­ teraction in the task (forces and torques, skin and object surface characteristics, object and hand size relationships) have been delineated. Further analysis is needed, and em­ pirical results for quantifing these variables. Some of the more general issues have been ignored, such as constraints related to the arm and body, constraints imposed by the laws of physics, and constraints related to the CNS in gen­ eral. ## IV. Conclusion The study of the human hand in prehensile movements can offer insights into the design of robot grasp planners. A design for a hierarchical grasp planner is presented. The first phase takes an object and task description and breaks them down into internal task requirements. These task re­ quirements include functional and physical constraints, and are designed to capture the int.ended use of the object in prehensile-specific terms. The task requirements are then mapped into an opposition space, which captures the avail­ able forces and DOFs of the hand, using vectors to repre­ sent virtual finger configurations. As the mappings in these phases are not unique, knowledge about constraints acting on the hand/ object /task interaction first must be identified and then applied. Empirical verification and quantification of the effects of these constraints and other constraints re­ main as future tasks. ## Acknowledgements I would like to thank my students at The Hartford Grad­ uate Center, including Ted Popielarczyk, Ralph Zampano, and Daniel Eldridge. I would also like to thank R. Vijayku­ mar, M. Arbib, and C. MacKenzie; also E. I. Robbins for editing an earlier draft of this paper. ## References 1. Arbib, M. A., Iberall, T. and Lyons, D. (1985): Coordi- nated Control Programs for Movements of the Hand. In: Hand Function and the Neocortex, A. W. Goodwin and I. Darian-Smith (Eds), Berlin: Springer Verlag, 111-129. 2. Hollerbach, J. R. and Flash, T. (1982): Dynamic Interac- tions Between Limb Segments during Planar Arm Move- ments, Bioi. Cyber, 44: 67-77. 3. Iberall, T. (1986): The Representation of Objects for Grasping, Proc Eighth Cog. Sci. Soc Conf, Amherst, Mass, Aug 15-17, 547-561. 4. Iberall, T. (1987a): A Neural Model of Human Prehension, COINS TR 87-01, Dept Comp Sci, Univ Mass, Amh, Mass. 5. Iberall, T. (1987b): The Nature of Human Prehension: Three Dextrous Hands in One . IEEE Inter Conf Robotics 1156 ROBOTICS and Automation, Raleigh, NC, March 30-April 3, 396-401. 6. Iberall, T., Bingham, G. and Arbib, M. A. (1986): Op- position Space as a Structuring Concept for the Analysis of Skilled Hand Movements. In: Generation and Modula- tion of Action PatternJ 1 H. Heuer and C. Fromm, Berlin: Springer-Verlag, 158-173. 7. Jeannerod, M. (1981): Intersegmental coordination Dur- ing Reaching at Natural Visual Objects. In: Attention and Performance IX, J. Long and A. Baddeley (Eds), Hillsdale: Erlbaum, 153-168. 8. Johansson, R.S. and Westling, G. (1984): Influences of Cu- taneous Sensory Input on Motor Coordination During Pre- cision Manipulation. In: SomatoJenJory MechaniJmJ, C Von Fuler & 0 Franzen (Eds), Lond: MacMillian, 249-260 9. Lozano-Perez, T . and Brooks, R. (1985): An Approach to Automatic Robot Programming, AI Memo 842, MIT AI Lab, Cambridge, Mass. 10. Lozano-Perez, T. and Winston, P. H. (1977): LAMA: A Language for Automatic Mechanical Assembly, Proc 5th Inti J Conf Artif Intel, Cambridge, Mass, Aug, 710-716. 11. Lyons, D. (1985): A Simple Set of Grasps for a Dextrous Hand, COINS TR 85-37, Dept Comp and Info Sci, Univ Mass, Amherst, Mass. 12. MacKenzie, C. L. and Marteniuk, R. G. (1985): Motor Skill: Feedback, Knowledge, and Structural Issues, Can J PJych, 39(2): 313-337. 13. Marteniuk, R.G., MacKenzie, C.L., Jeannerod, M., Athenes, S. and Dugas, C. (1987): Constraints on Human Arm Movement Trajectories, Can J PJych, in press. 14. Napier, J. R. (1956): Prehensile Movements of the Human Hand, J Bone Jt Surg, 38(4): 902-913. 15. Popielarczyk, T. (1987): A Study of Object Constraints for Dextrous Hand Prehension, Proc 3rd Hartford Grad Center Computer Science Con/, Hartford, CT, Apr 25, 227-231. 16. Schlesinger, G. (1919): Der Mechanische Aufbau der Kun- stlichen Glider. In: ErJatzglider und ArbeitJhilfen, M. Bor- chardt et al (Eds), Berlin: Springer. 17. Thbiana, R. (1981): The Architecture and Functions of the Hand. In: The Hand, vol1, R. Tubiana (Ed), Phil: W .B. Saunders, 19-93. 18. Vijaykumar, R. and Arbib, M. A. (1986): A Task-level Robot Planner for Assembly Operations, COINS TR 86- 31, Dept Comp Sci, Univ Mass, Amherst, Mass.