Patents by Inventor Jeroen van Baar

Jeroen van Baar has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20220126453
    Abstract: A controller is provided for interactive classification and recognition of an object in a scene using tactile feedback. The controller includes an interface configured to transmit and receive the control, sensor signals from a robot arm, gripper signals from a gripper attached to the robot arm, tactile signals from sensors attached to the gripper and at least one vision sensor, a memory module to store robot control programs, and a classifier and recognition model, and a processor to generate control signals based on the control program and a grasp pose on the object, configured to control the robot arm to grasp the object with the gripper.
    Type: Application
    Filed: October 22, 2020
    Publication date: April 28, 2022
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Radu Ioan Corcodel, Siddarth Jain, Jeroen van Baar
  • Publication number: 20220121188
    Abstract: A controller for controlling a system having uncertainties in its dynamics subject to constraints on an operation of the system is provided. The controller is configured to acquire historical data of the operation of the system, and determine, for the system in a current state, a current control action transitioning a state of the system from the current state to a next state. The current control action is determined according to a robust and constraint Markov decision process (RCMDP) that uses the historical data to optimize a performance cost of the operation of the system subject to an optimization of a safety cost enforcing the constraints on the operation, wherein a state transition for each of state and action pairs in the performance cost and the safety cost is represented by a plurality of state transitions capturing the uncertainties of the dynamics of the system.
    Type: Application
    Filed: December 16, 2020
    Publication date: April 21, 2022
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Mouhacine Benosman, Reazul Russel, Jeroen van Baar
  • Patent number: 11210560
    Abstract: A multi-modal dense correspondence image processing system submit the multi-modal images to a neural network to produce multi-modal features for each pixel of each of the multi-modal image. Each multi-modal image includes an image of a first modality and a corresponding image of a second modality different from the first modality. The neural network includes a first subnetwork trained to extract first features from pixels of the first modality, a second subnetwork trained to extract second features from pixels of the second modality, and a combiner configured to combine the first features and the second features to produce multi-modal features of a multi-modal image. The system compares the multi-modal features of a pair of multi-modal images to estimate a dense correspondence between pixels of the multi-modal images of the pair and outputs the dense correspondence between pixels of the multi-modal images in the pair.
    Type: Grant
    Filed: October 2, 2019
    Date of Patent: December 28, 2021
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventor: Jeroen van Baar
  • Publication number: 20210103770
    Abstract: A multi-modal dense correspondence image processing system submit the multi-modal images to a neural network to produce multi-modal features for each pixel of each of the multi-modal image. Each multi-modal image includes an image of a first modality and a corresponding image of a second modality different from the first modality. The neural network includes a first subnetwork trained to extract first features from pixels of the first modality, a second subnetwork trained to extract second features from pixels of the second modality, and a combiner configured to combine the first features and the second features to produce multi-modal features of a multi-modal image. The system compares the multi-modal features of a pair of multi-modal images to estimate a dense correspondence between pixels of the multi-modal images of the pair and outputs the dense correspondence between pixels of the multi-modal images in the pair.
    Type: Application
    Filed: October 2, 2019
    Publication date: April 8, 2021
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventor: Jeroen van Baar
  • Patent number: 10768294
    Abstract: A radar imaging system to reconstruct a radar reflectivity image of a scene including an object moving with the scene, includes an optical sensor to track the object over a period of time including multiple time steps to produce, for each of the multiple time steps, a deformation of a nominal shape of the object, and an electromagnetic sensor to acquire snapshots of the scene over the multiple time steps to produce a set of radar reflectivity image of the object with deformed shapes defined by the corresponding deformations of the nominal shape of the object.
    Type: Grant
    Filed: March 19, 2018
    Date of Patent: September 8, 2020
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Petros T Boufounos, Hassan Mansour, Rakshith Sharma Srinivasa, Jeroen van Baar
  • Publication number: 20190285740
    Abstract: A radar imaging system to reconstruct a radar reflectivity image of a scene including an object moving with the scene, includes an optical sensor to track the object over a period of time including multiple time steps to produce, for each of the multiple time steps, a deformation of a nominal shape of the object, and an electromagnetic sensor to acquire snapshots of the scene over the multiple time steps to produce a set of radar reflectivity image of the object with deformed shapes defined by the corresponding deformations of the nominal shape of the object.
    Type: Application
    Filed: March 19, 2018
    Publication date: September 19, 2019
    Inventors: Petros T. Boufounos, Hassan Mansour, Rakshith Sharma Srinivasa, Jeroen van Baar
  • Publication number: 20190134425
    Abstract: A real-time treatment system for tracking of an abnormal growth of tissue in a body of a patient. An offline stage includes storing historical image data of internal structures of bodies of different patients caused by a respiration of patients. A processor obtains for each patient image data, a sequence of dense motion vector fields. An online stage includes accepting an initial pair of temporally separated images of internal anatomical structures of the body of the patient during respiration in real-time. An imaging processor tracks changes of positions of landmark points over the initial pair of temporally separated images, to produce a first local motion representation of the body. An interpolation processor interpolates the first local motion representation to produce a first dense motion vector field for a controller to track a first location of the abnormal growth of tissue of the patient, to produce a first location.
    Type: Application
    Filed: November 8, 2017
    Publication date: May 9, 2019
    Inventor: Jeroen van Baar
  • Patent number: 9449384
    Abstract: A method registers a source image with a target image, wherein the images are deformable, by first measuring dissimilarity between the source image and the target image. The dissimilarity minimized using a discrete energy function. At multiple scales, multi-scale Markov random field registration is applied to the source and target images to determine a deformation vector field. Then, the target image is warped according to the deformation field vector to obtain a warped target mage registered to the source image.
    Type: Grant
    Filed: April 30, 2014
    Date of Patent: September 20, 2016
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Jeroen van Baar, Dohyung Seo
  • Patent number: 9449246
    Abstract: A method registers a deformable image to a fixed image by first acquiring the deformable image and the fixed image. Multi-resolution multi-level image pyramids are generated for the deformable and fixed images. Levels of the multi-resolution multi-level image pyramids are processed in a coarse to fine order, and for each deformable and fixed image at each level partitioning the deformable and fixed image into overlapping blocks, and labeling each block with a label by solving, iteratively and in parallel, until a termination condition is reached, a binary maximum flow problem, within ?-expansion, for each block and performing boundary fusion for the blocks, and then registering each block according to the labels to form the fixed image.
    Type: Grant
    Filed: May 14, 2015
    Date of Patent: September 20, 2016
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Jeroen van Baar, Dohyung Seo
  • Publication number: 20160063685
    Abstract: A method denoises a noisy image by, for each pixel in the noisy image, first constructing a key from a patch, wherein the patch includes locally neighboring pixels around the pixel. A function is selected from a function library using the key. Then, the function is applied to the patch to generate a corresponding noise free pixel for the pixel.
    Type: Application
    Filed: September 3, 2014
    Publication date: March 3, 2016
    Inventors: Oncel Tuzel, Jay Thornton, Jeroen van Baar
  • Patent number: 9262810
    Abstract: A method denoises a noisy image by, for each pixel in the noisy image, first constructing a key from a patch, wherein the patch includes locally neighboring pixels around the pixel. A function is selected from a function library using the key. Then, the function is applied to the patch to generate a corresponding noise free pixel for the pixel.
    Type: Grant
    Filed: September 3, 2014
    Date of Patent: February 16, 2016
    Assignee: Mitsubishi Electric Research Laboratories, inc.
    Inventors: Oncel Tuzel, Jay Thornton, Jeroen van Baar
  • Publication number: 20150317788
    Abstract: A method registers a source image with a target image, wherein the images are deformable, by first measuring dissimilarity between the source image and the target image. The dissimilarity minimized using a discrete energy function. At multiple scales, multi-scale Markov random field registration is applied to the source and target images to determine a deformation vector field. Then, the target image is warped according to the deformation field vector to obtain a warped target mage registered to the source image.
    Type: Application
    Filed: April 30, 2014
    Publication date: November 5, 2015
    Applicant: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Jeroen van Baar, Dohyung Seo
  • Publication number: 20140368615
    Abstract: To generate a pixel-accurate depth map, data from a range-estimation sensor (e.g., a time-of flight sensor) is combined with data from multiple cameras to produce a high-quality depth measurement for pixels in an image. To do so, a depth measurement system may use a plurality of cameras mounted on a support structure to perform a depth hypothesis technique to generate a first depth-support value. Furthermore, the apparatus may include a range-estimation sensor which generates a second depth-support value. In addition, the system may project a 3D point onto the auxiliary cameras and compare the color of the associated pixel in the auxiliary camera with the color of the pixel in reference camera to generate a third depth-support value. The system may combine these support values for each pixel in an image to determine respective depth values. Using these values, the system may generate a depth map for the image.
    Type: Application
    Filed: June 12, 2013
    Publication date: December 18, 2014
    Inventors: Jeroen van Baar, Paul A. Beardsley, Marc Pollefeys, Markus Gross
  • Patent number: 8570319
    Abstract: Where images are displayed such that unintended light is also included and that light cannot be fully subtracted from the displayed image, an image processor compensates by compensating for the remaining light using perceptual models. In some cases, the perceptual model includes a plurality of user sensitivities and computation is performed, at least in part, based on expected user sensitivity to light pollution in portions of the images. The perceptual model might include a contrast sensitivity function, threshold-vs-intensity, saliency prediction and visual masking. The model might include artist indications of view attention. A light pollution likelihood map might be used to reduce the amount of computation needed, using a likelihood measure to determine an amount of computation to perform.
    Type: Grant
    Filed: June 30, 2010
    Date of Patent: October 29, 2013
    Assignee: Disney Enterprises, Inc.
    Inventors: Jeroen van Baar, Steven Poulakos, Rasmus Tamstorf, Markus Gross
  • Publication number: 20110175904
    Abstract: Where images are displayed such that unintended light is also included and that light cannot be fully subtracted from the displayed image, an image processor compensates by compensating for the remaining light using perceptual models. In some cases, the perceptual model includes a plurality of user sensitivities and computation is performed, at least in part, based on expected user sensitivity to light pollution in portions of the images. The perceptual model might include a contrast sensitivity function, threshold-vs-intensity, saliency prediction and visual masking. The model might include artist indications of view attention. A light pollution likelihood map might be used to reduce the amount of computation needed, using a likelihood measure to determine an amount of computation to perform.
    Type: Application
    Filed: June 30, 2010
    Publication date: July 21, 2011
    Applicant: Disney Enterprises, Inc.
    Inventors: Jeroen van Baar, Steven Poulakos, Rasmus Tamstorf, Markus Gross
  • Publication number: 20090122195
    Abstract: A system and method combines videos for display in real-time. A set of narrow-angle videos and a wide-angle video are acquired of the scene, in which a field of view in the wide-angle video substantially overlaps the fields of view in the narrow-angle videos. Homographies are determined among the narrow-angle videos using the wide-angle video. Temporally corresponding selected images of the narrow-angle videos are transformed and combined into a transformed image. Geometry of an output video is determined according to the transformed image and geometry of a display screen of an output device. The homographies and the geometry of the display screen are stored in a graphic processor unit, and subsequent images in the set of narrow-angle videos are transformed and combined by the graphic processor unit to produce an output video in real-time.
    Type: Application
    Filed: November 9, 2007
    Publication date: May 14, 2009
    Inventors: Jeroen van Baar, Wojciech Matusik
  • Patent number: 7355583
    Abstract: A method generates input text with a pointing device at a distance from a writing surface. Motion information of a mobile coordinate frame of the pointing device is sensed relative to a stationary coordinate frame of the writing surface using a motion sensor configured to track motion of the pointing device. The input text is then determined from the motion information. A projection of a track of the motion of the pointing device on the writing surface can also be determined from the motion information, and the track can be displayed on the writing surface as the input text.
    Type: Grant
    Filed: August 10, 2004
    Date of Patent: April 8, 2008
    Assignee: Mitsubishi Electric Research Laboretories, Inc.
    Inventors: Paul A. Beardsley, Jeroen van Baar
  • Patent number: 7252387
    Abstract: A method adjusts a pose of a projector with respect to a display surface. A homography HD,P is determined between a display surface and a projector. The homography HD,P is decomposed into rotation and translation parameters expressing a pose MP of the projector. An ideal pose M0 of the projector corresponding to an ideal homography H0 is determined. A pose adjustment MA is determined according to MA=M0(MP)?1. The pose adjustment MA is decomposed into rotation and translation adjustment parameters. Then, the projector is adjusted mechanically to the ideal pose M0 according to the rotation and translation adjustment parameters.
    Type: Grant
    Filed: March 21, 2005
    Date of Patent: August 7, 2007
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Ramesh Raskar, Jeroen van Baar
  • Patent number: 7154395
    Abstract: A wireless location and identification system includes a controller and a tag. The controller has a projector configured to generate a unique temporal sequence of light intensities for each pixel in an output image of the projector and a wireless transceiver configured to emit an output signal and receive an input signal. The tag includes a photo-sensor configured to sense the unique temporal sequence of light intensities of a particular pixel, a transducer configured to receive the output signal from the transceiver, and a means for modulating, according to the unique temporal sequence of light intensities, the input signal for the transceiver in response to receiving the output signal and sensing the unique temporal sequence of light intensities of the particular pixel to indicate a location of the tag to the controller.
    Type: Grant
    Filed: July 1, 2004
    Date of Patent: December 26, 2006
    Assignee: Mitsubishi Electric Research Laboratories, Inc.
    Inventors: Ramesh Raskar, Paul A. Beardsley, Jeroen van Baar, Paul H. Dietz
  • Patent number: 7137707
    Abstract: A rigidly-coupled projector-camera system is augmented with laser pointers. The projector-camera system and lasers can be used to determine homography between the projector and a display surface, so that a projection of any desired geometry can be made on the display surface.
    Type: Grant
    Filed: July 1, 2004
    Date of Patent: November 21, 2006
    Assignee: Mitsubishi Electric Research Laboratories, Inc
    Inventors: Paul A. Beardsley, Jeroen van Baar, Ramesh Raskar