Abstract: Systems, methods, and apparatuses for controlling a robot including a manipulator, including: determining three-dimensional (3D) geometry information about a target object based on an image of the target object; determining 3D geometry information about a scene in which the target object is to be placed based on at least one image of the scene; obtaining affordance information by providing the 3D geometry information about the target object and the 3D geometry information about the scene to at least one neural network model; commanding the robot to grasp the target object using the manipulator according to a grasp orientation corresponding to the affordance information; and commanding the robot to position the manipulator according to a placement direction corresponding to the affordance information in order to place the target object at a location in the scene.

Abstract: A depth image is used to obtain a three dimensional (3D) geometry of an object as an object mesh. The object mesh is obtained using an object shell representation. The object shell representation is based on a series of depth images denoting the entry and exit points on the object surface that camera rays would pass through. Given a set of entry points in the form of a masked depth image of an object, an object shell (an entry image and an exit image) is generated. Since entry and exit images contain neighborhood information given by pixel adjacency, the entry and exit images provide partial meshes of the object which are stitched together in linear time using the contours of the entry and exit images. A complete object mesh is provided in the camera coordinate frame.

Abstract: A depth image is used to obtain a three dimensional (3D) geometry of an object as an object mesh. The object mesh is obtained using an object shell representation. The object shell representation is based on a series of depth images denoting the entry and exit points on the object surface that camera rays would pass through. Given a set of entry points in the form of a masked depth image of an object, an object shell (an entry image and an exit image) is generated. Since entry and exit images contain neighborhood information given by pixel adjacency, the entry and exit images provide partial meshes of the object which are stitched together in linear time using the contours of the entry and exit images. A complete object mesh is provided in the camera coordinate frame.

Abstract: Two-phase gripper. The gripper reorients and grasps an object while being picked up. The gripper includes a parallel jaw gripper including a pair of opposed, two-phase fingers, each finger including a cavity covered by an elastic strip wherein the elastic strip includes a point contact. Closure of the jaws of the gripper on an object at a first relatively lower force results in contact with lower friction between the point contact on the elastic strip on the fingers and the object allowing the object to rotate under gravity as the gripper is raised. Thereafter, closure of the jaws of the gripper on the object at a second relatively higher force causes the elastic strip to receded into the cavity resulting in multi-point contact with higher friction between the fingers and the object to securely grasp the object. In a preferred embodiment, the cavity is a Y-shaped groove and the object is cylindrical or prismatic.

Abstract: Two-phase gripper. The gripper reorients and grasps an object while being picked up, The gripper Includes a parallel jaw gripper including a pair of opposed, two-phase fingers, each finger including a cavity covered by an elastic strip wherein the elastic strip includes a point contact. Closure of the jaws of the gripper on as object at a first relatively lower force results in contact with lower friction between the point contact on the elastic strip on the fingers and the object allowing the object to rotate under gravity as the gripper is raised. Thereafter, closure of the jaws of the gripper on the object at a second relatively higher force causes the elastic strip to receded into the cavity resulting in multi-point contact with higher friction between the fingers and the object to securely grasp the object. In a preferred embodiment, the cavity is a Y-shaped groove and the object is cylindrical or prismatic.