Abstract: Systems and methods for visuo-tactile object pose estimation are provided. In one embodiment, a method includes receiving image data about an object and receiving depth data about the object. The method also includes generating a visual estimate of the object based on the image data and the depth data. The method further includes receiving tactile data about the object. The method yet further includes generating a tactile estimate of the object based on the tactile data. The method includes estimating a pose of the object based on the visual estimate and the tactile estimate.

Abstract: A system and method for multimodal object-centric representation learning that include receiving data associated with an image and a depth map of an object. The system and method also include determining an object-surface point cloud based on the image and the depth map. The system and method additionally include determining multi-resolution receptive fields based on the object-surface point cloud. The system and method further include passing the multi-resolution receptive fields through convolutional encoders to learn an object centric representation of the object.

Abstract: Pose fusion estimation may be achieved via a first and second set of sensors receiving a first and second set of data, passing the first and second set of data through a graph-based neural network to generate a set of geometric features to be passed through a pose fusion network to generate a first and second pose estimate. A second portion of the pose fusion network may receive the set of geometric features and generate a second set of geometric features and the second pose estimate based on the set of geometric features. A first portion of the pose fusion network may receive the first set of data and the second set of geometric features and generate the first pose estimate based on a fusion of the first set of data and the second set of geometric features.

Abstract: Systems and methods for visuo-tactile object pose estimation are provided. In one embodiment, a computer implemented method includes receiving image data, depth data, and tactile data about the object in the environment. The computer implemented method also includes generating a visual estimate of the object that includes an object point cloud. The computer implemented method further includes generating a tactile estimate of the object that includes a surface point cloud based on the tactile data. The computer implemented method yet further includes estimating a pose of the object based on the visual estimate and the tactile estimate by fusing the object point cloud and the surface point cloud in a 3D space. The pose is a six-dimensional pose.

Abstract: Systems and methods for visuo-tactile object pose estimation are provided. In one embodiment, a method includes receiving image data about an object and receiving depth data about the object. The method also includes generating a visual estimate of the object based on the image data and the depth data. The method further includes receiving tactile data about the object. The method yet further includes generating a tactile estimate of the object based on the tactile data. The method includes estimating a pose of the object based on the visual estimate and the tactile estimate.

Abstract: Methods, grasping systems, and computer-readable mediums storing computer executable code for grasping an object are provided. In an example, a depth image of the object may be obtained by a grasping system. A potential grasp point of the object may be determined by the grasping system based on the depth image. A tactile output corresponding to the potential grasp point may be estimated by the grasping system based on data from the depth image. The grasping system may be controlled to grasp the object at the potential grasp point based on the estimated tactile output.

Abstract: Systems and methods for tactile output estimation are provided. In one embodiment, the system includes a depth map module, an estimation module, and a surface module. The depth map module is configured to identify a region of interest (RoI) of an object. The area of the RoI corresponds to a tactile sensor size of a tactile sensor. The depth module is further configured to receive depth data for the RoI from a depth sensor and generate a depth map for the RoI based on a volume of the depth data relative to a frame of reference of the RoI. The estimation module is configured to estimate a tactile sensor output based on the depth map. The surface module configured to determine surface properties based on the estimated tactile sensor output.

Abstract: Aspects of the present disclosure include methods, apparatuses, and computer readable media of traversing an obstacle including receiving optical data associated with an area near the robot, identifying the obstacle in the area based on the optical data, deflating, in response to identifying the obstacle, a ball of the robot, and applying a downward force through the deflated ball to propel the robot over the obstacle.

Abstract: Aspects of the present disclosure include methods, apparatuses, and computer readable media of traversing an obstacle including receiving optical data associated with an area near the robot, identifying the obstacle in the area based on the optical data, deflating, in response to identifying the obstacle, a ball of the robot, and applying a downward force through the deflated ball to propel the robot over the obstacle.

Abstract: Aspects of the present disclosure include methods, apparatuses, and computer readable media for performing an undesirable task including receiving an indication identifying a person performing a task at a first time, receiving a plurality of input data associated with the person while performing the task, determining whether the task is undesirable based on the plurality of input data, and causing, in response to determining that the task is undesirable, the robot to perform the task at a second time after the first time.

Abstract: Embodiments, systems, and methods for navigational planning of a mobile programmable agent are provided. In some embodiments, the navigational planning may include identifying a plurality of dynamic objects in a physical environment having an origin and a destination. The physical environment is divided into a plurality of plane figures. The location of a centroid of each plane figure can then be calculated. A network of segments is formed from the origin to the destination intersecting the centroids. At least one channel is determined from the origin to the destination using a set of segments. A set of gates is identified along the at least one channel. The state of the gates is selectively determined based on movement of the dynamic objects. A pathway can then be identified within the channel for the mobile programmable agent to traverse from the origin to the destination based on the state of the gates.

Abstract: Methods, grasping systems, and computer-readable mediums storing computer executable code for grasping an object are provided. In an example, a depth image of the object may be obtained by a grasping system. A potential grasp point of the object may be determined by the grasping system based on the depth image. A tactile output corresponding to the potential grasp point may be estimated by the grasping system based on data from the depth image. The grasping system may be controlled to grasp the object at the potential grasp point based on the estimated tactile output.

Abstract: Embodiments, systems, and methods for navigational planning of a mobile programmable agent are provided. In some embodiments, the navigational planning may include identifying a plurality of dynamic objects in a physical environment having an origin and a destination. The physical environment is divided into a plurality of plane figures. The location of a centroid of each plane figure can then be calculated. A network of segments is formed from the origin to the destination intersecting the centroids. At least one channel is determined from the origin to the destination using a set of segments. A set of gates is identified along the at least one channel. The state of the gates is selectively determined based on movement of the dynamic objects. A pathway can then be identified within the channel for the mobile programmable agent to traverse from the origin to the destination based on the state of the gates.

Abstract: An operation management system includes: a plurality of robots; and an operation management unit server configured to manage an operation route in a predetermined region. Each robot has a sensor that recognizes a surrounding environment. When the sensor of a first robot recognizes a predetermined surrounding environment, the server refers to a position of the surrounding environment and sets an operation route of a second robot.

Abstract: A ceiling map building method includes estimating a scale of each ceiling image based on information related to the ceiling image and information related to another ceiling image including a same object as included in the ceiling image, the scale being represented as a ratio of an amount of movement of the object between the two ceiling images to an amount of movement of the camera (6) between the positions thereof when the two ceiling images were respectively captured (ST16), and building a ceiling map (2) (ST2) by converting the ceiling images in accordance with the respective scales so as to have sizes suitable for the ceiling map and combining the converted ceiling images (ST84).

Abstract: A ceiling map building method includes estimating a scale of each ceiling image based on information related to the ceiling image and information related to another ceiling image including a same object as included in the ceiling image, the scale being represented as a ratio of an amount of movement of the object between the two ceiling images to an amount of movement of the camera (6) between the positions thereof when the two ceiling images were respectively captured (ST16), and building a ceiling map (2) (ST2) by converting the ceiling images in accordance with the respective scales so as to have sizes suitable for the ceiling map and combining the converted ceiling images (ST84).

Abstract: To provide a system that can enhance stability of a result of predicting the state of an object. At least one candidate trajectory having a degree of approximation to a reference trajectory generated based on a current state of the object being in a specified rank or higher over a first specification period is specified as “a first candidate trajectory”. At least one candidate trajectory, extending from a last time point of being the first candidate trajectory to before elapse of a second specification period, is specified as “a second candidate trajectory”. Accordingly, it becomes possible to enhance stability of the specification result of the candidate trajectory as a result of predicting the state of the object.

Abstract: An operation management system includes: a plurality of robots; and an operation management unit server configured to manage an operation route in a predetermined region. Each robot has a sensor that recognizes a surrounding environment. When the sensor of a first robot recognizes a predetermined surrounding environment, the server refers to a position of the surrounding environment and sets an operation route of a second robot.

Abstract: To provide a system that can enhance stability of a result of predicting the state of an object. At least one candidate trajectory having a degree of approximation to a reference trajectory generated based on a current state of the object being in a specified rank or higher over a first specification period is specified as “a first candidate trajectory”. At least one candidate trajectory, extending from a last time point of being the first candidate trajectory to before elapse of a second specification period, is specified as “a second candidate trajectory”. Accordingly, it becomes possible to enhance stability of the specification result of the candidate trajectory as a result of predicting the state of the object.

Abstract: A system capable of causing an agent to continuously execute a plurality of different subtasks while securing the continuity of behavior of the agent is provided. A plurality of state variable trajectories representing the time series of a state variable of an object are generated according to a stochastic transition model in which the state variable of the object is represented as a random variable. The stochastic transition model is defined so that the transition mode of the state variable is determined according to an execution probability of each subtask in which a probability distribution is represented by a Dirichlet distribution. An operation of the agent is controlled so that the state of the object transits according to one state variable trajectory (desired state variable trajectory) maximizing or optimizing the joint probability of a whole of the stochastic transition model among the plurality of state variable trajectories.