Patents by Inventor Pushmeet Kohli

Pushmeet Kohli has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 9911032
    Abstract: Tracking hand or body pose from image data is described, for example, to control a game system, natural user interface or for augmented reality. In various examples a prediction engine takes a single frame of image data and predicts a distribution over a pose of a hand or body depicted in the image data. In examples, a stochastic optimizer has a pool of candidate poses of the hand or body which it iteratively refines, and samples from the predicted distribution are used to replace some candidate poses in the pool. In some examples a best candidate pose from the pool is selected as the current tracked pose and the selection processes uses a 3D model of the hand or body.
    Type: Grant
    Filed: January 4, 2017
    Date of Patent: March 6, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Cem Keskin, Jonathan Taylor, Toby Sharp, Shahram Izadi, Andrew William Fitzgibbon, Pushmeet Kohli, Duncan Paul Robertson
  • Patent number: 9886652
    Abstract: Correspondences in content items may be determined using a trained decision tree to detect distinctive matches between portions of content items. The techniques described include determining a first group of patches associated with a first content item and processing a first patch based at least partly on causing the first patch to move through a decision tree, and determining a second group of patches associated with a second content item and processing a second patch based at least partly on causing the second patch to move through the decision tree. The techniques described include determining that the first patch and the second patch are associated with a same leaf node of the decision tree and determining that the first patch and the second patch are corresponding patches based at least partly on determining that the first patch and the second patch are associated with the same leaf node.
    Type: Grant
    Filed: March 15, 2016
    Date of Patent: February 6, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Sean Ryan Francesco Fanello, Shahram Izadi, Pushmeet Kohli, Christoph Rhemann, Shenlong Wang
  • Publication number: 20170372126
    Abstract: Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.
    Type: Application
    Filed: September 11, 2017
    Publication date: December 28, 2017
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph SHOTTON, Cem KESKIN, Christoph RHEMANN, Toby SHARP, Duncan Paul ROBERTSON, Pushmeet KOHLI, Andrew William FITZGIBBON, Shahram IZADI
  • Publication number: 20170316347
    Abstract: Crowdsourcing systems with machine learning are described. Specifically, item-label inference methods and systems are presented, for example, to provide aggregated answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples, an item-label inference system infers variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, an item-label inference system provides aggregated labels while considering the inferred worker characteristics and the inferred characteristics of the worker communities. In examples the item-label inference system provides uncertainty information associated with the inference results for selecting workers and generating future tasks.
    Type: Application
    Filed: July 17, 2017
    Publication date: November 2, 2017
    Inventors: Matteo Venanzi, John Philip Guiver, Gabriella Kazai, Pushmeet Kohli, Milad Shokouhi
  • Patent number: 9773155
    Abstract: Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.
    Type: Grant
    Filed: October 14, 2014
    Date of Patent: September 26, 2017
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Cem Keskin, Christoph Rhemann, Toby Sharp, Duncan Paul Robertson, Pushmeet Kohli, Andrew William Fitzgibbon, Shahram Izadi
  • Publication number: 20170270390
    Abstract: Correspondences in content items may be determined using a trained decision tree to detect distinctive matches between portions of content items. The techniques described include determining a first group of patches associated with a first content item and processing a first patch based at least partly on causing the first patch to move through a decision tree, and determining a second group of patches associated with a second content item and processing a second patch based at least partly on causing the second patch to move through the decision tree. The techniques described include determining that the first patch and the second patch are associated with a same leaf node of the decision tree and determining that the first patch and the second patch are corresponding patches based at least partly on determining that the first patch and the second patch are associated with the same leaf node.
    Type: Application
    Filed: March 15, 2016
    Publication date: September 21, 2017
    Inventors: Sean Ryan Francesco Fanello, Shahram Izadi, Pushmeet Kohli, Christoph Rhemann, Shenlong Wang
  • Patent number: 9767419
    Abstract: Crowdsourcing systems with machine learning are described, for example, to aggregate answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples a machine learning system jointly learns variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, the machine learning system learns aggregated labels. In examples learnt variables describing characteristics of an individual crowd worker are related, by addition of noise, to learnt variables describing characteristics of a community of which the individual is a member. In examples the crowdsourcing system uses the learnt variables describing characteristics of individual workers and of communities of workers for any one or more of: active learning, targeted training of workers, targeted issuance of tasks, calculating and issuing rewards.
    Type: Grant
    Filed: January 24, 2014
    Date of Patent: September 19, 2017
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Matteo Venanzi, John Philip Guiver, Gabriella Kazai, Pushmeet Kohli, Milad Shokouhi
  • Patent number: 9734424
    Abstract: Filtering sensor data is described, for example, where filters conditioned on a local appearance of the signal are predicted by a machine learning system, and used to filter the sensor data. In various examples the sensor data is a stream of noisy video image data and the filtering process denoises the video stream. In various examples the sensor data is a depth image and the filtering process refines the depth image which may then be used for gesture recognition or other purposes. In various examples the sensor data is one dimensional measurement data from an electric motor and the filtering process denoises the measurements. In examples the machine learning system comprises a random decision forest where trees of the forest store filters at their leaves. In examples, the random decision forest is trained using a training objective with a data dependent regularization term.
    Type: Grant
    Filed: April 14, 2014
    Date of Patent: August 15, 2017
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Sean Ryan Francesco Fanello, Cem Keskin, Pushmeet Kohli, Shahram Izadi, Jamie Daniel Joseph Shotton, Antonio Criminisi
  • Publication number: 20170147947
    Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.
    Type: Application
    Filed: October 28, 2016
    Publication date: May 25, 2017
    Inventors: Jamie Daniel Joseph Shotton, Toby Sharp, Pushmeet Kohli, Reinhard Sebastian Bernhard Nowozin, John Michael Winn, Antonio Criminisi
  • Publication number: 20170116471
    Abstract: Tracking hand or body pose from image data is described, for example, to control a game system, natural user interface or for augmented reality. In various examples a prediction engine takes a single frame of image data and predicts a distribution over a pose of a hand or body depicted in the image data. In examples, a stochastic optimizer has a pool of candidate poses of the hand or body which it iteratively refines, and samples from the predicted distribution are used to replace some candidate poses in the pool. In some examples a best candidate pose from the pool is selected as the current tracked pose and the selection processes uses a 3D model of the hand or body.
    Type: Application
    Filed: January 4, 2017
    Publication date: April 27, 2017
    Inventors: Jamie Daniel Joseph Shotton, Cem Keskin, Jonathan Taylor, Toby Sharp, Shahram Izadi, Andrew William Fitzgibbon, Pushmeet Kohli, Duncan Paul Robertson
  • Patent number: 9626766
    Abstract: A method of sensing depth using an RGB camera. In an example method, a color image of a scene is received from an RGB camera. The color image is applied to a trained machine learning component which uses features of the image elements to assign all or some of the image elements a depth value which represents the distance between the surface depicted by the image element and the RGB camera. In various examples, the machine learning component comprises one or more entangled geodesic random decision forests.
    Type: Grant
    Filed: February 28, 2014
    Date of Patent: April 18, 2017
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Antonio Criminisi, Duncan Paul Robertson, Peter Kontschieder, Pushmeet Kohli, Henrik Turbell, Adriana Dumitras, Indeera Munasinghe, Jamie Daniel Joseph Shotton
  • Patent number: 9619035
    Abstract: A gesture detection and recognition technique is described. In one example, a sequence of data items relating to the motion of a gesturing user is received. A selected set of data items from the sequence are tested against pre-learned threshold values, to determine a probability of the sequence representing a certain gesture. If the probability is greater than a predetermined value, then the gesture is detected, and an action taken. In examples, the tests are performed by a trained decision tree classifier. In another example, the sequence of data items can be compared to pre-learned templates, and the similarity between them determined. If the similarity for a template exceeds a threshold, a likelihood value associated with a future time for a gesture associated with that template is updated. Then, when the future time is reached, the gesture is detected if the likelihood value is greater than a predefined value.
    Type: Grant
    Filed: March 4, 2011
    Date of Patent: April 11, 2017
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Sebastian Nowozin, Pushmeet Kohli, Jamie Daniel Joseph Shotton
  • Patent number: 9613298
    Abstract: Tracking using sensor data is described, for example, where a plurality of machine learning predictors are used to predict a plurality of complementary, or diverse, parameter values of a process describing how the sensor data arises. In various examples a selector selects which of the predicted values are to be used, for example, to control a computing device. In some examples the tracked parameter values are pose of a moving camera or pose of an object moving in the field of view of a static camera; in some examples the tracked parameter values are of a 3D model of a hand or other articulated or deformable entity. The machine learning predictors have been trained in series, with training examples being reweighted after training an individual predictor, to favor training examples on which the set of predictors already trained performs poorly.
    Type: Grant
    Filed: June 2, 2014
    Date of Patent: April 4, 2017
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Abner Guzmán-Rivera, Pushmeet Kohli, Benjamin Michael Glocker, Jamie Daniel Joseph Shotton, Shahram Izadi, Toby Sharp, Andrew William Fitzgibbon
  • Patent number: 9552070
    Abstract: Tracking hand or body pose from image data is described, for example, to control a game system, natural user interface or for augmented reality. In various examples a prediction engine takes a single frame of image data and predicts a distribution over a pose of a hand or body depicted in the image data. In examples, a stochastic optimizer has a pool of candidate poses of the hand or body which it iteratively refines, and samples from the predicted distribution are used to replace some candidate poses in the pool. In some examples a best candidate pose from the pool is selected as the current tracked pose and the selection processes uses a 3D model of the hand or body.
    Type: Grant
    Filed: September 23, 2014
    Date of Patent: January 24, 2017
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Cem Keskin, Jonathan James Taylor, Toby Sharp, Shahram Izadi, Andrew William Fitzgibbon, Pushmeet Kohli, Duncan Paul Robertson
  • Publication number: 20160370867
    Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.
    Type: Application
    Filed: May 24, 2016
    Publication date: December 22, 2016
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
  • Patent number: 9489639
    Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.
    Type: Grant
    Filed: November 13, 2013
    Date of Patent: November 8, 2016
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Toby Sharp, Pushmeet Kohli, Reinhard Sebastian Bernhard Nowozin, John Michael Winn, Antonio Criminisi
  • Publication number: 20160205109
    Abstract: A method of controlling access to content such as web sites on the intranet or interne is described. For example, the method comprises receiving an address of content to be accessed and obtaining similarity of the address to previously labeled addresses of other content items. The similarity is based on co-occurrence of addresses of content items in records of browsing sessions from many consenting users. For example, a browsing session record comprises addresses of content items accessed by a user in a time period during which the user is actively accessing content. A co-occurrence of addresses of content items is the existence of the addresses in the same browsing session record. Access to the content is then controlled on the basis of the similarity.
    Type: Application
    Filed: January 13, 2015
    Publication date: July 14, 2016
    Inventors: Pushmeet Kohli, Yoram Bachrach, Filip Radlinski, Ulrich Paquet, Li Quan Khoo
  • Patent number: 9380224
    Abstract: A method of sensing depth using an infrared camera. In an example method, an infrared image of a scene is received from an infrared camera. The infrared image is applied to a trained machine learning component which uses the intensity of image elements to assign all or some of the image elements a depth value which represents the distance between the surface depicted by the image element and the infrared camera. In various examples, the machine line component comprises one or more random decision forests.
    Type: Grant
    Filed: February 28, 2014
    Date of Patent: June 28, 2016
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Cem Keskin, Sean Ryan Francesco Fanello, Shahram Izadi, Pushmeet Kohli, David Kim, David Sweeney, Jamie Daniel Joseph Shotton, Duncan Paul Robertson, Sing Bing Kang
  • Patent number: 9372544
    Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.
    Type: Grant
    Filed: May 16, 2014
    Date of Patent: June 21, 2016
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
  • Publication number: 20160163054
    Abstract: Systems and methods for reducing interference between multiple infra-red depth cameras are described. In an embodiment, the system comprises multiple infra-red sources, each of which projects a structured light pattern into the environment. A controller is used to control the sources in order to reduce the interference caused by overlapping light patterns. Various methods are described including: cycling between the different sources, where the cycle used may be fixed or may change dynamically based on the scene detected using the cameras; setting the wavelength of each source so that overlapping patterns are at different wavelengths; moving source-camera pairs in independent motion patterns; and adjusting the shape of the projected light patterns to minimize overlap. These methods may also be combined in any way. In another embodiment, the system comprises a single source and a mirror system is used to cast the projected structured light pattern around the environment.
    Type: Application
    Filed: January 20, 2016
    Publication date: June 9, 2016
    Inventors: Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Stephen Edward Hodges, David Alexander Butler, Andrew Fitzgibbon, Pushmeet Kohli