Abstract: A nonholonomic camera space manipulation system which allows a mobile manipulator which can have both holonomic and nonholonomic movement to autonomously use computer vision to move with respect to a goal position without any prior knowledge or calibration between the two or more video cameras used with the computer vision and the manipulator base or arm, the cameras and the goal, or the base and arm and the goal position. Cues are associated with the manipulator arm and the goal position(s) or target bodies; the cues being distinguishable in the two dimensional focal plane camera spaces of the cameras from the surrounding environment. A processing unit, identifying the position of the cues in each camera space, compares those positions and instructs movements of the manipulator base and/or manipulator arm to achieve movement of the visual cue on the arm with respect to the goal position or target body visual cue in camera space.

Abstract: A means and method for camera space manipulation includes a manipulator arm extending from a base to an outward end. The arm is movable through a work space to accomplish various tasks. One or more cameras are movably oriented towards the arm and work space to capture the arm and work space in what will be called camera space or camera vision. A visual cue is associated with the outward end of the manipulator arm. Additionally, a visual cue is associated with an object which is desired to be engaged by the manipulator arm or by what is held by the manipulator arm. A control device is connected to the camera or cameras and to the manipulator. According to identification and tracking of the visual cues in the camera space, the control device instructs appropriate motors to move the manipulator arm according to estimations for engagement calculated by the control device.