Abstract: A system for trajectories imitation for robotic manipulators is provided. The system includes an interface configured to receive a plurality of task descriptions, wherein the interface is configured to communicate with a real-world robot, a memory to store computer-executable programs including a robot simulator, a training module and a transfer module, and a processor, in connection with the memory. The processor is configured to perform training using the training module, for the task descriptions on the robot simulator, to produce a plurality of source policy with subgoals for the task descriptions. The processor performs training using the training module, for the task descriptions on the real-world robot, to produce a plurality of target policy with subgoals for the task descriptions, and update the parameters of the transfer module from corresponding trajectories with the subgoals for the robot simulator and real-world robot.

Abstract: A controller is provided for interactive classification and recognition of an object in a scene using tactile feedback. The controller includes an interface configured to transmit and receive the control, sensor signals from a robot arm, gripper signals from a gripper attached to the robot arm, tactile signals from sensors attached to the gripper and at least one vision sensor, a memory module to store robot control programs, and a classifier and recognition model, and a processor to generate control signals based on the control program and a grasp pose on the object, configured to control the robot arm to grasp the object with the gripper.

Abstract: A controller for controlling a system having uncertainties in its dynamics subject to constraints on an operation of the system is provided. The controller is configured to acquire historical data of the operation of the system, and determine, for the system in a current state, a current control action transitioning a state of the system from the current state to a next state. The current control action is determined according to a robust and constraint Markov decision process (RCMDP) that uses the historical data to optimize a performance cost of the operation of the system subject to an optimization of a safety cost enforcing the constraints on the operation, wherein a state transition for each of state and action pairs in the performance cost and the safety cost is represented by a plurality of state transitions capturing the uncertainties of the dynamics of the system.

Abstract: A system for trajectories imitation for robotic manipulators is provided. The system includes an interface configured to receive a plurality of task descriptions, wherein the interface is configured to communicate with a real-world robot, a memory to store computer-executable programs including a robot simulator, a training module and a transfer module, and a processor, in connection with the memory. The processor is configured to perform training using the training module, for the task descriptions on the robot simulator, to produce a plurality of source policy with subgoals for the task descriptions. The processor performs training using the training module, for the task descriptions on the real-world robot, to produce a plurality of target policy with subgoals for the task descriptions, and update the parameters of the transfer module from corresponding trajectories with the subgoals for the robot simulator and real-world robot.

Abstract: A system for parameter tuning for robotic manipulators is provided. The system includes an interface configured to receive a task specification, a plurality of physical parameters, and a plurality of control parameters, wherein the interface is configured to communicate with a real-world robot via a robot controller. The system further includes a memory to store computer-executable programs including a robot simulation module, a robot controller, and an auto-tuning module a processor, in connection with the memory. In this case, the processor is configured to acquire, in communication with the real-world robot, state values of the real-world robot, state values of the robot simulation module, simultaneously update, by use of a predetermined optimization algorithm with the auto-tuning module, an estimate of one or more of the physical, and said control parameters, and store the updated parameters.

Abstract: A controller is provided for interactive classification and recognition of an object in a scene using tactile feedback. The controller includes an interface configured to transmit and receive the control, sensor signals from a robot arm, gripper signals from a gripper attached to the robot arm, tactile signals from sensors attached to the gripper and at least one vision sensor, a memory module to store robot control programs, and a classifier and recognition model, and a processor to generate control signals based on the control program and a grasp pose on the object, configured to control the robot arm to grasp the object with the gripper.

Abstract: A controller for controlling a system having uncertainties in its dynamics subject to constraints on an operation of the system is provided. The controller is configured to acquire historical data of the operation of the system, and determine, for the system in a current state, a current control action transitioning a state of the system from the current state to a next state. The current control action is determined according to a robust and constraint Markov decision process (RCMDP) that uses the historical data to optimize a performance cost of the operation of the system subject to an optimization of a safety cost enforcing the constraints on the operation, wherein a state transition for each of state and action pairs in the performance cost and the safety cost is represented by a plurality of state transitions capturing the uncertainties of the dynamics of the system.

Abstract: A multi-modal dense correspondence image processing system submit the multi-modal images to a neural network to produce multi-modal features for each pixel of each of the multi-modal image. Each multi-modal image includes an image of a first modality and a corresponding image of a second modality different from the first modality. The neural network includes a first subnetwork trained to extract first features from pixels of the first modality, a second subnetwork trained to extract second features from pixels of the second modality, and a combiner configured to combine the first features and the second features to produce multi-modal features of a multi-modal image. The system compares the multi-modal features of a pair of multi-modal images to estimate a dense correspondence between pixels of the multi-modal images of the pair and outputs the dense correspondence between pixels of the multi-modal images in the pair.

Abstract: A multi-modal dense correspondence image processing system submit the multi-modal images to a neural network to produce multi-modal features for each pixel of each of the multi-modal image. Each multi-modal image includes an image of a first modality and a corresponding image of a second modality different from the first modality. The neural network includes a first subnetwork trained to extract first features from pixels of the first modality, a second subnetwork trained to extract second features from pixels of the second modality, and a combiner configured to combine the first features and the second features to produce multi-modal features of a multi-modal image. The system compares the multi-modal features of a pair of multi-modal images to estimate a dense correspondence between pixels of the multi-modal images of the pair and outputs the dense correspondence between pixels of the multi-modal images in the pair.

Abstract: A radar imaging system to reconstruct a radar reflectivity image of a scene including an object moving with the scene, includes an optical sensor to track the object over a period of time including multiple time steps to produce, for each of the multiple time steps, a deformation of a nominal shape of the object, and an electromagnetic sensor to acquire snapshots of the scene over the multiple time steps to produce a set of radar reflectivity image of the object with deformed shapes defined by the corresponding deformations of the nominal shape of the object.

Abstract: A radar imaging system to reconstruct a radar reflectivity image of a scene including an object moving with the scene, includes an optical sensor to track the object over a period of time including multiple time steps to produce, for each of the multiple time steps, a deformation of a nominal shape of the object, and an electromagnetic sensor to acquire snapshots of the scene over the multiple time steps to produce a set of radar reflectivity image of the object with deformed shapes defined by the corresponding deformations of the nominal shape of the object.

Abstract: A real-time treatment system for tracking of an abnormal growth of tissue in a body of a patient. An offline stage includes storing historical image data of internal structures of bodies of different patients caused by a respiration of patients. A processor obtains for each patient image data, a sequence of dense motion vector fields. An online stage includes accepting an initial pair of temporally separated images of internal anatomical structures of the body of the patient during respiration in real-time. An imaging processor tracks changes of positions of landmark points over the initial pair of temporally separated images, to produce a first local motion representation of the body. An interpolation processor interpolates the first local motion representation to produce a first dense motion vector field for a controller to track a first location of the abnormal growth of tissue of the patient, to produce a first location.

Abstract: A method registers a deformable image to a fixed image by first acquiring the deformable image and the fixed image. Multi-resolution multi-level image pyramids are generated for the deformable and fixed images. Levels of the multi-resolution multi-level image pyramids are processed in a coarse to fine order, and for each deformable and fixed image at each level partitioning the deformable and fixed image into overlapping blocks, and labeling each block with a label by solving, iteratively and in parallel, until a termination condition is reached, a binary maximum flow problem, within ?-expansion, for each block and performing boundary fusion for the blocks, and then registering each block according to the labels to form the fixed image.

Abstract: A method registers a source image with a target image, wherein the images are deformable, by first measuring dissimilarity between the source image and the target image. The dissimilarity minimized using a discrete energy function. At multiple scales, multi-scale Markov random field registration is applied to the source and target images to determine a deformation vector field. Then, the target image is warped according to the deformation field vector to obtain a warped target mage registered to the source image.

Abstract: A method denoises a noisy image by, for each pixel in the noisy image, first constructing a key from a patch, wherein the patch includes locally neighboring pixels around the pixel. A function is selected from a function library using the key. Then, the function is applied to the patch to generate a corresponding noise free pixel for the pixel.

Abstract: A method denoises a noisy image by, for each pixel in the noisy image, first constructing a key from a patch, wherein the patch includes locally neighboring pixels around the pixel. A function is selected from a function library using the key. Then, the function is applied to the patch to generate a corresponding noise free pixel for the pixel.

Abstract: A method registers a source image with a target image, wherein the images are deformable, by first measuring dissimilarity between the source image and the target image. The dissimilarity minimized using a discrete energy function. At multiple scales, multi-scale Markov random field registration is applied to the source and target images to determine a deformation vector field. Then, the target image is warped according to the deformation field vector to obtain a warped target mage registered to the source image.

Abstract: To generate a pixel-accurate depth map, data from a range-estimation sensor (e.g., a time-of flight sensor) is combined with data from multiple cameras to produce a high-quality depth measurement for pixels in an image. To do so, a depth measurement system may use a plurality of cameras mounted on a support structure to perform a depth hypothesis technique to generate a first depth-support value. Furthermore, the apparatus may include a range-estimation sensor which generates a second depth-support value. In addition, the system may project a 3D point onto the auxiliary cameras and compare the color of the associated pixel in the auxiliary camera with the color of the pixel in reference camera to generate a third depth-support value. The system may combine these support values for each pixel in an image to determine respective depth values. Using these values, the system may generate a depth map for the image.

Abstract: Where images are displayed such that unintended light is also included and that light cannot be fully subtracted from the displayed image, an image processor compensates by compensating for the remaining light using perceptual models. In some cases, the perceptual model includes a plurality of user sensitivities and computation is performed, at least in part, based on expected user sensitivity to light pollution in portions of the images. The perceptual model might include a contrast sensitivity function, threshold-vs-intensity, saliency prediction and visual masking. The model might include artist indications of view attention. A light pollution likelihood map might be used to reduce the amount of computation needed, using a likelihood measure to determine an amount of computation to perform.

Abstract: Where images are displayed such that unintended light is also included and that light cannot be fully subtracted from the displayed image, an image processor compensates by compensating for the remaining light using perceptual models. In some cases, the perceptual model includes a plurality of user sensitivities and computation is performed, at least in part, based on expected user sensitivity to light pollution in portions of the images. The perceptual model might include a contrast sensitivity function, threshold-vs-intensity, saliency prediction and visual masking. The model might include artist indications of view attention. A light pollution likelihood map might be used to reduce the amount of computation needed, using a likelihood measure to determine an amount of computation to perform.