Abstract: The video image of the end effector (26) and work surface (32) in the vicinity of the end effector is converted into a series of continually generated digitized images (132), which are further processed to detect and track the relative motion between the end effector and the work surface. After calibration (126), whereby an index of coordinates and arm orientation is established for a known positioning of the end effector relative to a known reference feature (28a) on the work surface, the tracking (168) of the motion of the end effector is utilized to continually update the index (104) to indicate the actual coordinate and orientation of the end effector relative to the work surface. The updated index is displayed on a live video monitor (44), and can be used as input to the feedback circuit (94) of the manipulator control system.