Patents by Inventor Andrew Rabinovich

Andrew Rabinovich has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20190147341
    Abstract: Systems, devices, and methods for training a neural network and performing image interest point detection and description using the neural network. The neural network may include an interest point detector subnetwork and a descriptor subnetwork. An optical device may include at least one camera for capturing a first image and a second image. A first set of interest points and a first descriptor may be calculated using the neural network based on the first image, and a second set of interest points and a second descriptor may be calculated using the neural network based on the second image. A homography between the first image and the second image may be determined based on the first and second sets of interest points and the first and second descriptors. The optical device may adjust virtual image light being projected onto an eyepiece based on the homography.
    Type: Application
    Filed: November 14, 2018
    Publication date: May 16, 2019
    Applicant: Magic Leap, Inc.
    Inventors: Andrew Rabinovich, Daniel DeTone, Tomasz Jan Malisiewicz
  • Publication number: 20190147298
    Abstract: Methods and systems for meta-learning are described for automating learning of child tasks with a single neural network. The order in which tasks are learned by the neural network can affect performance of the network, and the meta-learning approach can use a task-level curriculum for multi-task training. The task-level curriculum can be learned by monitoring a trajectory of loss functions during training. The meta-learning approach can learn to adapt task loss balancing weights in the course of training to get improved performance on multiple tasks on real world datasets. Advantageously, learning to dynamically balance weights among different task losses can lead to superior performance over the use of static weights determined by expensive random searches or heuristics. Embodiments of the meta-learning approach can be used for computer vision tasks or natural language processing tasks, and the trained neural networks can be used by augmented or virtual reality devices.
    Type: Application
    Filed: November 9, 2018
    Publication date: May 16, 2019
    Inventors: Andrew RABINOVICH, Vijay BADRINARAYANAN, Srivignesh RAJENDRAN, Chen-Yu LEE
  • Publication number: 20190130275
    Abstract: Systems and methods for training a multitask network is disclosed. In one aspect, training the multitask network includes determining a gradient norm of a single-task loss adjusted by a task weight for each task, with respect to network weights of the multitask network, and a relative training rate for the task based on the single-task loss for the task. Subsequently, a gradient loss function, comprising a difference between (1) the determined gradient norm for each task and (2) a corresponding target gradient norm, can be determined. An updated task weight for the task can be determined and used in the next iteration of training the multitask network, using a gradient of the gradient loss function with respect to the task weight for the task.
    Type: Application
    Filed: October 24, 2018
    Publication date: May 2, 2019
    Inventors: Zhao Chen, Vijay Badrinarayanan, Andrew Rabinovich
  • Patent number: 10255529
    Abstract: The present disclosure provides an improved approach to implement structure learning of neural networks by exploiting correlations in the data/problem the networks aim to solve. A greedy approach is described that finds bottlenecks of information gain from the bottom convolutional layers all the way to the fully connected layers. Rather than simply making the architecture deeper, additional computation and capacitance is only added where it is required.
    Type: Grant
    Filed: March 13, 2017
    Date of Patent: April 9, 2019
    Assignee: Magic Leap, Inc.
    Inventors: Andrew Rabinovich, Vijay Badrinarayanan, Daniel DeTone, Srivignesh Rajendran, Douglas Bertram Lee, Tomasz Malisiewicz
  • Publication number: 20190005670
    Abstract: Augmented reality devices and methods for computing a homography based on two images. One method may include receiving a first image based on a first camera pose and a second image based on a second camera pose, generating a first point cloud based on the first image and a second point cloud based on the second image, providing the first point cloud and the second point cloud to a neural network, and generating, by the neural network, the homography based on the first point cloud and the second point cloud. The neural network may be trained by generating a plurality of points, determining a 3D trajectory, sampling the 3D trajectory to obtain camera poses viewing the points, projecting the points onto 2D planes, comparing a generated homography using the projected points to the ground-truth homography and modifying the neural network based on the comparison.
    Type: Application
    Filed: June 27, 2018
    Publication date: January 3, 2019
    Applicant: Magic Leap, Inc.
    Inventors: Daniel DeTone, Tomasz Jan Malisiewicz, Andrew Rabinovich
  • Publication number: 20180322147
    Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics.
    Type: Application
    Filed: July 9, 2018
    Publication date: November 8, 2018
    Inventors: David Petrou, Andrew Rabinovich, Hartwig Adam
  • Publication number: 20180300897
    Abstract: Systems and methods for reducing error from noisy data received from a high frequency sensor by fusing received input with data received from a low frequency sensor by collecting a first set of dynamic inputs from the high frequency sensor, collecting a correction input point from the low frequency sensor, and adjusting a propagation path of a second set of dynamic inputs from the high frequency sensor based on the correction input point either by full translation to the correction input point or dampened approach towards the correction input point.
    Type: Application
    Filed: December 29, 2017
    Publication date: October 18, 2018
    Applicant: Magic Leap, Inc.
    Inventors: Michael Janusz Woods, Andrew Rabinovich
  • Publication number: 20180268220
    Abstract: Systems and methods for estimating a layout of a room are disclosed. The room layout can comprise the location of a floor, one or more walls, and a ceiling. In one aspect, a neural network can analyze an image of a portion of a room to determine the room layout. The neural network can comprise a convolutional neural network having an encoder sub-network, a decoder sub-network, and a side sub-network. The neural network can determine a three-dimensional room layout using two-dimensional ordered keypoints associated with a room type. The room layout can be used in applications such as augmented or mixed reality, robotics, autonomous indoor navigation, etc.
    Type: Application
    Filed: March 16, 2018
    Publication date: September 20, 2018
    Inventors: Chen-Yu Lee, Vijay Badrinarayanan, Tomasz Malisiewicz, Andrew Rabinovich
  • Publication number: 20180239144
    Abstract: Methods and systems for triggering presentation of virtual content based on sensor information. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergences. The system may monitor information detected via the sensors, and based on the monitored information, trigger access to virtual content identified in the sensor information. Virtual content can be obtained, and presented as augmented reality content via the display system. The system may monitor information detected via the sensors to identify a QR code, or a presence of a wireless beacon. The QR code or wireless beacon can trigger the display system to obtain virtual content for presentation.
    Type: Application
    Filed: February 15, 2018
    Publication date: August 23, 2018
    Inventors: Michael Janusz Woods, Andrew Rabinovich, Richard Leslie Taylor
  • Patent number: 10031927
    Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics.
    Type: Grant
    Filed: November 2, 2015
    Date of Patent: July 24, 2018
    Assignee: Google LLC
    Inventors: David Petrou, Andrew Rabinovich, Hartwig Adam
  • Publication number: 20180137642
    Abstract: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.
    Type: Application
    Filed: November 14, 2017
    Publication date: May 17, 2018
    Inventors: Tomasz Malisiewicz, Andrew Rabinovich, Vijay Badrinarayanan, Debidatta Dwibedi
  • Publication number: 20180075659
    Abstract: A sensory eyewear system for a mixed reality device can facilitate user's interactions with the other people or with the environment. As one example, the sensory eyewear system can recognize and interpret a sign language, and present the translated information to a user of the mixed reality device. The wearable system can also recognize text in the user's environment, modify the text (e.g., by changing the content or display characteristics of the text), and render the modified text to occlude the original text.
    Type: Application
    Filed: September 12, 2017
    Publication date: March 15, 2018
    Inventors: Eric Browy, Michael Janusz Woods, Andrew Rabinovich
  • Publication number: 20180053056
    Abstract: A head-mounted augmented reality (AR) device can include a hardware processor programmed to receive different types of sensor data from a plurality of sensors (e.g., an inertial measurement unit, an outward-facing camera, a depth sensing camera, an eye imaging camera, or a microphone); and determining an event of a plurality of events using the different types of sensor data and a hydra neural network (e.g., face recognition, visual search, gesture identification, semantic segmentation, object detection, lighting detection, simultaneous localization and mapping, relocalization).
    Type: Application
    Filed: August 22, 2017
    Publication date: February 22, 2018
    Inventors: Andrew Rabinovich, Tomasz Jan Malisiewicz, Daniel DeTone
  • Publication number: 20170337470
    Abstract: A method for generating inputs for a neural network based on an image includes receiving the image, identifying a position within the image, and identifying a subset of the image at the position. The subset of the image is defined by a first set of corners. The method also includes perturbing at least one of the first set of corners to form a second set of corners. The second set of corners defines a modified subset of the image. The method further includes determining a homography based on a comparison between the subset of the image and the modified subset of the image, generating a transformed image by applying the homography to the image, and identifying a subset of the transformed image at the position.
    Type: Application
    Filed: May 19, 2017
    Publication date: November 23, 2017
    Applicant: Magic Leap, Inc.
    Inventors: Daniel DeTone, Tomasz Malisiewicz, Andrew Rabinovich
  • Publication number: 20170262737
    Abstract: The present disclosure provides an improved approach to implement structure learning of neural networks by exploiting correlations in the data/problem the networks aim to solve. A greedy approach is described that finds bottlenecks of information gain from the bottom convolutional layers all the way to the fully connected layers. Rather than simply making the architecture deeper, additional computation and capacitance is only added where it is required.
    Type: Application
    Filed: March 13, 2017
    Publication date: September 14, 2017
    Applicant: Magic Leap, Inc.
    Inventors: Andrew Rabinovich, Vijay Badrinarayanan, Daniel DeTone, Srivignesh Rajendran, Douglas Bertram Lee, Tomasz Malisiewicz
  • Publication number: 20170161919
    Abstract: A method of determining a pose of an image capture device includes capturing an image using an image capture device. The method also includes generating a data structure corresponding to the captured image. The method further includes comparing the data structure with a plurality of known data structures to identify a most similar known data structure. Moreover, the method includes reading metadata corresponding to the most similar known data structure to determine a pose of the image capture device.
    Type: Application
    Filed: December 5, 2016
    Publication date: June 8, 2017
    Applicant: Magic Leap, Inc.
    Inventors: Brigit Schroeder, Tomasz J. Malisiewicz, Andrew Rabinovich
  • Patent number: 9367756
    Abstract: Methods and systems for selecting a representative image of an entity are disclosed. According to one embodiment, a computer-implemented method for selecting a representative image of an entity is disclosed. The method includes: accessing a collection of images of the entity; clustering, based on similarity of one or more similarity features, images from the collection to form a plurality of similarity clusters; and selecting the representative image from one of said similarity clusters. Further, based on cluster size of said similarity clusters popular clusters can be determined, and the selection of the representative image can be from the popular clusters. In addition, the method can further include assigning a headshot score based upon a portion of the respective image covered by the entity to respective images in said popular clusters, and further selecting the representative image based upon the headshot score.
    Type: Grant
    Filed: April 16, 2014
    Date of Patent: June 14, 2016
    Assignee: Google Inc.
    Inventors: Anand Pillai, Andrew Rabinovich
  • Publication number: 20160055182
    Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics.
    Type: Application
    Filed: November 2, 2015
    Publication date: February 25, 2016
    Inventors: David Petrou, Andrew Rabinovich, Hartwig Adam
  • Patent number: 9230194
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting training images. One of the methods includes determining, for each of a plurality of labels that each designate a respective food class of a plurality of food classes, a respective measure of importance. A respective sample size is determined for the label based on the respective measure of importance of the label. A number of training images are selected for each respective label according to the determined sample size for the label. A predictive model is trained using the selected training images as training data.
    Type: Grant
    Filed: September 16, 2013
    Date of Patent: January 5, 2016
    Assignee: Google Inc.
    Inventors: Andrew Rabinovich, Hartwig Adam
  • Patent number: 9208177
    Abstract: A facial recognition search system identifies one or more likely names (or other personal identifiers) corresponding to the facial image(s) in a query as follows. After receiving the visual query with one or more facial images, the system identifies images that potentially match the respective facial image in accordance with visual similarity criteria. Then one or more persons associated with the potential images are identified. For each identified person, person-specific data comprising metrics of social connectivity to the requester are retrieved from a plurality of applications such as communications applications, social networking applications, calendar applications, and collaborative applications. An ordered list of persons is then generated by ranking the identified persons in accordance with at least metrics of visual similarity between the respective facial image and the potential image matches and with the social connection metrics.
    Type: Grant
    Filed: February 20, 2014
    Date of Patent: December 8, 2015
    Assignee: Google Inc.
    Inventors: David Petrou, Andrew Rabinovich, Hartwig Adam