Abstract: A computer system automatically selects robot end-effectors for pick-and-place applications using a model-predictive control algorithm. The system may select an end-effector to replace an existing end-effector in order to optimize (or at least increase) throughput. The system uses a predictive model of reward, where reward of each potential grasp for each end tool is parameterized by a deep neural network. The system may also use a variety of metrics to evaluate the performance of the tool-selection algorithm, and thereby improve performance of the system.

Abstract: A computer-implemented system for fully automating the training, auto-tuning, and deployment of machine learning models at customer sites, especially models for robotic grasping tasks. The system automatically: (1) detects/predicts performance degradation of one or more machine learning models; (2) triggers a new model training/fine-tuning job in response to such detection/prediction; and (3) deploys the new model upon training completion, without stopping or pausing the production line.

Abstract: A computer system trains a neural network to predict, for each pixel in an input image, the position that a robot's end effector would reach if a grasp (“poke”) were attempted at that position. Training data consists of images and end effector positions recorded while a robot attempts grasps in a pick-and-place environment. For an automated grasping policy, the approach is self-supervised, as end effector position labels may be recovered through forward kinematics, without human annotation. Although gathering such physical interaction data is expensive, it is necessary for training and routine operation of state of the art manipulation systems. Therefore, the system comes “for free” while collecting data for other tasks (e.g., grasping, pushing, placing). The system achieves significantly lower root mean squared error than traditional structured light sensors and other self-supervised deep learning methods on difficult, industry-scale jumbled bin datasets.

Abstract: A computer system trains a neural network on an instance segmentation task by casting the problem as one of mapping each pixel to a probability distribution over arbitrary instance labels. This simplifies both the training and inference problems, because the formulation is end-to-end trainable and requires no post-processing to extract maximum a posteriori estimates of the instance labels.

Abstract: A computer system uses Learning from Demonstration (LfD) techniques in which a multitude of tasks are demonstrated without requiring careful task set up, labeling, and engineering, and learns multiple modes of behavior from visual data, rather than averaging the multiple modes. As a result, the computer system may be used to control a robot or other system to exhibit the multiple modes of behavior in appropriate circumstances.