--- title: "Modelling and Monitoring Manipulation Environment" authors: - "T. Hasegava" - "H. Inoue" doi: "" url: "https://ijcai.org/proceedings/1979/80" pdf: "https://ijcai.org/proceedings/1979/0080.pdf" venue: "Proceedings of the Sixth International Joint Conference on Artificial Intelligence" year: 1979 pages: "" track: "" keywords: --- # Modelling and Monitoring Manipulation Environment T. Hasegava, H. Inoue *Proceedings of the Sixth International Joint Conference on Artificial Intelligence*. ## Abstract The development of high-level manipulator language will cause an explosion of declarations about the real world environment. Procedures may be written inn a few statements of higher level language, while world descriptions will still require hundreds of statements. Since the world model is a complex data base describing three-dimensional objects, it is not easy to construct it. A system is proposed which enables a user to construct objects model interactively. The system provides a laser spot as a measuring tool and a display monitor as a visualizing device of the internal data base and the real world. Thus the user can supervise the process and the result of model construction. ## Full text ## 1. INTRODUCTION Efforts are being made at some laboratories to deve lop a high level language for manipulation. But those experi ences reveal the problem of an explosion of declarations about the real world environment. That is, manipulation procedures may be wri tten in a few statements of the higher leve l language, . while the world description will still require hundreds of statements. Grossman and Taylor (1978) proposed an interactive system for model generation using the manipulator itself as a measuring tool. But their system forces us to step into the working space of the manipulator and requires the labor of guiding it. Also, lack of the means for visualizing the constructed model would increase the possibility of errors in the process of modelling. We proposP nn alternative approach for mode 11 ·j np; Hlld monitoring a manipulation environment. Our system uses a laser spot to spe~ify and measure a feature point, and a special dispJay to visualize generated model and r eal environment simultaneously. The combi 11u t. ion of the laser pointer and the <.li s play promises natural man-machine interaction . ## 2. CONCEPT Qf ~ARCHITECTURE The schematic diagram of the system is shown in Fig. 1. ## 2 .1 Major Hardware Units Hardware units are connected to PDP-11/34 mini­ computer on its UNIBUS. ## a. ~ Spot Pointer A laser spot is used for measuring and pointing. Deflection angles of a laser beam and the two dimensional position of the reflected spot on the imaging plane are taken into the computer. A standard 525-line TV camera and an optical scanner assembly are used for this purpose. The spot position is detected from TV video signal as a center of the brightest area. ## b. Superimposable Display ~ This unit is composed of a graphic refresh memory, a TV camera and a monitoring display. The display screen is divided into 640x4Ro dots which correspond to 64ox480 bits of the refresh memory. Reading of the refresh memory is synchronized with the TV camera scanning. Thus the bit pattern written in the memory is trans­ formed into the video signal and superimposed onto the video image of the real world from the TV camera. A zoom lens and a pan-tilt head of the camera help the accurate pointing and measuring. ## 2.2 Pointer Operation A user manipulates a joystick to locate the spot at the place wherever he wants. Then the spot position in the work space coordinates system is calculated based on the triangulation and is stored in the data area with a name for the later reference. The spot is also controllable by the computer. ## 2.3 Geometry Editor Using obtained points from the real world, this module makes basic geometric elements. They are lines, planes, vectors, rotation matrices and so on. Generation of basic bodies and synthesis to complicated objects are also performed. Furthermore non-geometric informa­ tion about objects is added to construct better model. ## 2.4 Graphics ~Scope Accomodation Calculating the direction and the range of view of the TV camera, the graphics generates the projected skelton image of objects from the stored data. With this generated image, the user can easily grasp three dimensional shapes and spatial relations of objects. ## 3. EXAMPLES QE ~ SYSTEM PERFORMANCE The hardware of this system is completed, and the software implementation is now in progress. Fig. 2 through 4 illustrate how the system will work. A vise and its handle are placed in the work space and are monitored. (Fig. 2 ) The generated model of the handle and handle­ shaft are displayed on CRT. (Fig. 3) The real scene and the generated image are superimposed on the monitoring display. This provides us convenient ways for verification of the generated model. (Fig. 4) ![Image](https://ijcai.org/proceedings/1979/fulltext/80/image_000000_95f2600f8c6f30c5d4bc98cba619b457bc78ad34f44de47019f6dfcbc6d7bbbe.png) ## 4. CONCLUSION 'l'he superimpose display allows us to monitor the real world and the internal data simultane­ ously . Judging whether the generated image and the real scene are well overlapped or not, we can supervise the process and the result of model construction. Thus, combination of the laser pointer and the superjmpose display promises natural m:m-machi ne interact ion. f>ince active scanning of the Jaser spot by computer is possible, the laser pointer will be developed to a semi-automated local vision. Fig. 2 Real Scene ![Image](https://ijcai.org/proceedings/1979/fulltext/80/image_000001_141b3b99097f279f95829a77899e237eb08242e6668997af1cbb567f1c04bc2a.png) ## References 1. Grossman, D. D. and Taylor, R. H. (1978) "Interactive Generation of Objects Models with a Manipulator." IEEE Trans. Systems, Man, and Cybernetics., Vol. SMC-8, pp667-679 2. Ishii, M. and Nagata, T. (1976), "Feature Extraction of Three-Dimensional Objects and Visual Processing in a Hand-Eye System Using Laser Tracker." Pattern Recognition, Vol. 8, pp229-237