Abstract: A method of generating a composite map from image data and spatial map data may include: acquiring a spatial map of an environment; acquiring a plurality of images of a portion of the environment; generating a three-dimensional (3D) image of the portion of the environment using the plurality of images; identifying a floor in the 3D image; generating a synthetic spatial map of the portion of the environment based on the floor in the 3D image; determining a location of the portion of the environment within the spatial map by identifying a region of the spatial map that corresponds to the synthetic spatial map; and generating a composite map by associating the 3D image with the location within the spatial map of the portion of the environment.

Abstract: Methods and devices for processing image data for scene completion, including receiving an original image from an original viewpoint corresponding to a first direction, wherein the original image includes an object; obtaining a first image from a new viewpoint corresponding to a second direction different from the first direction by rotating the original image based on 3-dimensional (3D) information generated from 2-dimensional (2D) information which is obtained from the original image; determining an area within the first image for generating a second surface of the object based on depth information about a depth between the object and the background of the original image, wherein the determined area is expected to include an object area; and obtaining a second image by inputting the first image and the determined area to an artificial intelligence (AI) inpainting model, wherein the AI inpainting model generates the second surface of the object which occupies a portion of the determined area in the second im

Abstract: Systems, methods, and apparatuses for controlling a robot including a manipulator, including: determining three-dimensional (3D) geometry information about a target object based on an image of the target object; determining 3D geometry information about a scene in which the target object is to be placed based on at least one image of the scene; obtaining affordance information by providing the 3D geometry information about the target object and the 3D geometry information about the scene to at least one neural network model; commanding the robot to grasp the target object using the manipulator according to a grasp orientation corresponding to the affordance information; and commanding the robot to position the manipulator according to a placement direction corresponding to the affordance information in order to place the target object at a location in the scene.

Abstract: A depth image is used to obtain a three dimensional (3D) geometry of an object as an object mesh. The object mesh is obtained using an object shell representation. The object shell representation is based on a series of depth images denoting the entry and exit points on the object surface that camera rays would pass through. Given a set of entry points in the form of a masked depth image of an object, an object shell (an entry image and an exit image) is generated. Since entry and exit images contain neighborhood information given by pixel adjacency, the entry and exit images provide partial meshes of the object which are stitched together in linear time using the contours of the entry and exit images. A complete object mesh is provided in the camera coordinate frame.

Abstract: A depth image is used to obtain a three dimensional (3D) geometry of an object as an object mesh. The object mesh is obtained using an object shell representation. The object shell representation is based on a series of depth images denoting the entry and exit points on the object surface that camera rays would pass through. Given a set of entry points in the form of a masked depth image of an object, an object shell (an entry image and an exit image) is generated. Since entry and exit images contain neighborhood information given by pixel adjacency, the entry and exit images provide partial meshes of the object which are stitched together in linear time using the contours of the entry and exit images. A complete object mesh is provided in the camera coordinate frame.

Abstract: Two-phase gripper. The gripper reorients and grasps an object while being picked up. The gripper includes a parallel jaw gripper including a pair of opposed, two-phase fingers, each finger including a cavity covered by an elastic strip wherein the elastic strip includes a point contact. Closure of the jaws of the gripper on an object at a first relatively lower force results in contact with lower friction between the point contact on the elastic strip on the fingers and the object allowing the object to rotate under gravity as the gripper is raised. Thereafter, closure of the jaws of the gripper on the object at a second relatively higher force causes the elastic strip to receded into the cavity resulting in multi-point contact with higher friction between the fingers and the object to securely grasp the object. In a preferred embodiment, the cavity is a Y-shaped groove and the object is cylindrical or prismatic.

Abstract: Two-phase gripper. The gripper reorients and grasps an object while being picked up, The gripper Includes a parallel jaw gripper including a pair of opposed, two-phase fingers, each finger including a cavity covered by an elastic strip wherein the elastic strip includes a point contact. Closure of the jaws of the gripper on as object at a first relatively lower force results in contact with lower friction between the point contact on the elastic strip on the fingers and the object allowing the object to rotate under gravity as the gripper is raised. Thereafter, closure of the jaws of the gripper on the object at a second relatively higher force causes the elastic strip to receded into the cavity resulting in multi-point contact with higher friction between the fingers and the object to securely grasp the object. In a preferred embodiment, the cavity is a Y-shaped groove and the object is cylindrical or prismatic.