Patents by Inventor Pushmeet Kohli

Pushmeet Kohli has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20130244782
    Abstract: Real-time camera tracking using depth maps is described. In an embodiment depth map frames are captured by a mobile depth camera at over 20 frames per second and used to dynamically update in real-time a set of registration parameters which specify how the mobile depth camera has moved. In examples the real-time camera tracking output is used for computer game applications and robotics. In an example, an iterative closest point process is used with projective data association and a point-to-plane error metric in order to compute the updated registration parameters. In an example, a graphics processing unit (GPU) implementation is used to optimize the error metric in real-time. In some embodiments, a dense 3D model of the mobile camera environment is used.
    Type: Application
    Filed: February 23, 2013
    Publication date: September 19, 2013
    Applicant: MICROSOFT CORPORATION
    Inventors: Richard Newcombe, Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler
  • Publication number: 20130166481
    Abstract: A tractable model solves certain labeling problems by providing potential functions having arbitrary dependencies upon an observed dataset (e.g., image data). The model uses decision trees corresponding to various factors to map dataset content to a set of parameters used to define the potential functions in the model. Some factors define relationships among multiple variable nodes. When making label predictions on a new dataset, the leaf nodes of the decision tree determine the effective weightings for such potential functions. In this manner, decision trees define non-parametric dependencies and can represent rich, arbitrary functional relationships if sufficient training data is available. Decision trees training is scalable, both in the training set size and by parallelization. Maximum pseudolikelihood learning can provide for joint training of aspects of the model, including feature test selection and ordering, factor weights, and the scope of the interacting variable nodes used in the graph.
    Type: Application
    Filed: December 27, 2011
    Publication date: June 27, 2013
    Applicant: Microsoft Corporation
    Inventors: Reinhard Sebastian Bernhard Nowozin, Carsten Curt Eckard Rother, Bangpeng Yao, Toby Leonard Sharp, Pushmeet Kohli
  • Publication number: 20130156298
    Abstract: Using high-level attributes to guide image processing is described. In an embodiment high-level attributes of images of people such as height, torso orientation, body shape, gender are used to guide processing of the images for various tasks including but not limited to joint position detection, body part classification, medical image analysis and others. In various embodiments one or more random decision forests are trained using images where global variable values such as player height are known in addition to ground-truth data appropriate for the image processing task concerned. In some examples sequences of images are used where global variables are static or vary smoothly over the sequence. In some examples one or more trained random decision forests are used to find global variable values as well as output values for the task concerned such as joint positions or body part classes.
    Type: Application
    Filed: December 15, 2011
    Publication date: June 20, 2013
    Applicant: MICROSOFT CORPORATION
    Inventors: Pushmeet KOHLI, Jamie Daniel Joseph SHOTTON, Min SUN
  • Publication number: 20130156297
    Abstract: Learning image processing tasks from scene reconstructions is described where the tasks may include but are not limited to: image de-noising, image in-painting, optical flow detection, interest point detection. In various embodiments training data is generated from a 2 or higher dimensional reconstruction of a scene and from empirical images of the same scene. In an example a machine learning system learns at least one parameter of a function for performing the image processing task by using the training data. In an example, the machine learning system comprises a random decision forest. In an example, the scene reconstruction is obtained by moving an image capture apparatus in an environment where the image capture apparatus has an associated dense reconstruction and camera tracking system.
    Type: Application
    Filed: December 15, 2011
    Publication date: June 20, 2013
    Applicant: MICROSOFT CORPORATION
    Inventors: Jamie Daniel Joseph SHOTTON, Pushmeet KOHLI, Stefan Johannes Josef HOLZER, Shahram IZADI, Carsten Curt Eckard ROTHER, Sebastian NOWOZIN, David KIM, David MOLYNEAUX, Otmar HILLIGES
  • Publication number: 20130107010
    Abstract: Surface segmentation from RGB and depth images is described. In one example, a computer receives an image of a scene. The image has pixels which each have an associated color value and an associated depth value representing a distance between from an image sensor to a surface in the scene. The computer uses the depth values to derive a set of three-dimensional planes present within the scene. A cost function is used to determine whether each pixel belongs to one of the planes, and the image elements are labeled accordingly. The cost function has terms dependent on the depth value of a pixel, and the color values of the pixels and at least one neighboring pixel. In various examples, the planes can be extended until they intersect to determine the extent of the scene, and pixels not belonging to a plane can be labeled as objects on the surfaces.
    Type: Application
    Filed: November 2, 2011
    Publication date: May 2, 2013
    Applicant: MICROSOFT CORPORATION
    Inventors: Derek HOIEM, Pushmeet KOHLI
  • Patent number: 8401225
    Abstract: Moving object segmentation using depth images is described. In an example, a moving object is segmented from the background of a depth image of a scene received from a mobile depth camera. A previous depth image of the scene is retrieved, and compared to the current depth image using an iterative closest point algorithm. The iterative closest point algorithm includes a determination of a set of points that correspond between the current depth image and the previous depth image. During the determination of the set of points, one or more outlying points are detected that do not correspond between the two depth images, and the image elements at these outlying points are labeled as belonging to the moving object. In examples, the iterative closest point algorithm is executed as part of an algorithm for tracking the mobile depth camera, and hence the segmentation does not add substantial additional computational complexity.
    Type: Grant
    Filed: January 31, 2011
    Date of Patent: March 19, 2013
    Assignee: Microsoft Corporation
    Inventors: Richard Newcombe, Shahram Izadi, Otmar Hilliges, David Kim, David Molyneaux, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler
  • Patent number: 8401242
    Abstract: Real-time camera tracking using depth maps is described. In an embodiment depth map frames are captured by a mobile depth camera at over 20 frames per second and used to dynamically update in real-time a set of registration parameters which specify how the mobile depth camera has moved. In examples the real-time camera tracking output is used for computer game applications and robotics. In an example, an iterative closest point process is used with projective data association and a point-to-plane error metric in order to compute the updated registration parameters. In an example, a graphics processing unit (GPU) implementation is used to optimize the error metric in real-time. In some embodiments, a dense 3D model of the mobile camera environment is used.
    Type: Grant
    Filed: January 31, 2011
    Date of Patent: March 19, 2013
    Assignee: Microsoft Corporation
    Inventors: Richard Newcombe, Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler
  • Publication number: 20120306734
    Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.
    Type: Application
    Filed: May 31, 2011
    Publication date: December 6, 2012
    Applicant: MICROSOFT CORPORATION
    Inventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
  • Publication number: 20120306876
    Abstract: Generating computer models of 3D objects is described. In one example, depth images of an object captured by a substantially static depth camera are used to generate the model, which is stored in a memory device in a three-dimensional volume. Portions of the depth image determined to relate to the background are removed to leave a foreground depth image. The position and orientation of the object in the foreground depth image is tracked by comparison to a preceding depth image, and the foreground depth image is integrated into the volume by using the position and orientation to determine where to add data derived from the foreground depth image into the volume. In examples, the object is hand-rotated by a user before the depth camera. Hands that occlude the object are integrated out of the model as they do not move in sync with the object due to re-gripping.
    Type: Application
    Filed: June 6, 2011
    Publication date: December 6, 2012
    Applicant: MICROSOFT CORPORATION
    Inventors: Jamie Daniel Joseph SHOTTON, Shahram IZADI, Otmar HILLIGES, David KIM, David MOLYNEAUX, Pushmeet KOHLI, Andrew FITZGIBBON, Stephen Edward HODGES
  • Publication number: 20120288186
    Abstract: An enhanced training sample set containing new synthesized training images that are artificially generated from an original training sample set is provided to satisfactorily increase the accuracy of an object recognition system. The original sample set is artificially augmented by introducing one or more variations to the original images with little to no human input. There are a large number of possible variations that can be introduced to the original images, such as varying the image's position, orientation, and/or appearance and varying an object's context, scale, and/or rotation. Because there are computational constraints on the amount of training samples that can be processed by object recognition systems, one or more variations that will lead to a satisfactory increase in the accuracy of the object recognition performance are identified and introduced to the original images.
    Type: Application
    Filed: May 12, 2011
    Publication date: November 15, 2012
    Applicant: Microsoft Corporation
    Inventors: Pushmeet Kohli, Jamie Shotton, Motaz el Saban
  • Publication number: 20120257814
    Abstract: Image completion using scene geometry is described, for example, to remove marks from digital photographs or complete regions which are blank due to editing. In an embodiment an image depicting, from a viewpoint, a scene of textured objects has regions to be completed. In an example, geometry of the scene is estimated from a depth map and the geometry used to warp the image so that at least some surfaces depicted in the image are fronto-parallel to the viewpoint. An image completion process is guided using distortion applied during the warping. For example, patches used to fill the regions are selected on the basis of distortion introduced by the warping. In examples where the scene comprises regions having only planar surfaces the warping process comprises rotating the image. Where the scene comprises non-planar surfaces, geodesic distances between image elements may be scaled to flatten the non-planar surfaces.
    Type: Application
    Filed: April 8, 2011
    Publication date: October 11, 2012
    Applicant: Microsoft Corporation
    Inventors: Pushmeet KOHLI, Toby SHARP, Carsten Curt Eckard ROTHER
  • Publication number: 20120251008
    Abstract: Classification algorithm optimization is described. In an example, a classification algorithm is optimized by calculating an evaluation sequence for a set of weighted feature functions that orders the feature functions in accordance with a measure of influence on the classification algorithm. Classification thresholds are determined for each step of the evaluation sequence, which indicate whether a classification decision can be made early and the classification algorithm terminated without evaluating further feature functions. In another example, a classifier applies the weighted feature functions to previously unseen data in the order of the evaluation sequence and determines a cumulative value at each step. The cumulative value is compared to the classification thresholds at each step to determine whether a classification decision can be made early without evaluating further feature functions.
    Type: Application
    Filed: March 31, 2011
    Publication date: October 4, 2012
    Applicant: MICROSOFT CORPORATION
    Inventors: Sebastian Nowozin, Pushmeet Kohli
  • Publication number: 20120237127
    Abstract: This application describes grouping variables together to minimize cost or time of performing computer vision analysis techniques on images. In one instance, the pixels of an image are represented by a lattice structure of nodes that are connected to each other by edges. The nodes are grouped or merged together based in part on the energy function associated with each edge that connects the nodes together. The energy function of the edge is based in part on the energy functions associated with each node. The energy functions of the node are based on the possible states in which the node may exist. The states of the node are representative of an object, image, or any other feature or classification that may be associated with the pixels in the image.
    Type: Application
    Filed: March 14, 2011
    Publication date: September 20, 2012
    Applicant: Microsoft Corporation
    Inventors: Pushmeet Kohli, Sebastian Reinhard Bernhard Nowozin
  • Publication number: 20120239174
    Abstract: Predicting joint positions is described, for example, to find joint positions of humans or animals (or parts thereof) in an image to control a computer game or for other applications. In an embodiment image elements of a depth image make joint position votes so that for example, an image element depicting part of a torso may vote for a position of a neck joint, a left knee joint and a right knee joint. A random decision forest may be trained to enable image elements to vote for the positions of one or more joints and the training process may use training images of bodies with specified joint positions. In an example a joint position vote is expressed as a vector representing a distance and a direction of a joint position from an image element making the vote. The random decision forest may be trained using a mixture of objectives.
    Type: Application
    Filed: March 17, 2011
    Publication date: September 20, 2012
    Applicant: Microsoft Corporation
    Inventors: Jamie Daniel Joseph Shotton, Pushmeet Kohli, Ross Brook Girshick, Andrew Fitzgibbon, Antonio Criminisi
  • Publication number: 20120225719
    Abstract: A gesture detection and recognition technique is described. In one example, a sequence of data items relating to the motion of a gesturing user is received. A selected set of data items from the sequence are tested against pre-learned threshold values, to determine a probability of the sequence representing a certain gesture. If the probability is greater than a predetermined value, then the gesture is detected, and an action taken. In examples, the tests are performed by a trained decision tree classifier. In another example, the sequence of data items can be compared to pre-learned templates, and the similarity between them determined. If the similarity for a template exceeds a threshold, a likelihood value associated with a future time for a gesture associated with that template is updated. Then, when the future time is reached, the gesture is detected if the likelihood value is greater than a predefined value.
    Type: Application
    Filed: March 4, 2011
    Publication date: September 6, 2012
    Applicant: Mirosoft Corporation
    Inventors: Sebastian Nowozin, Pushmeet Kohli, Jamie Daniel Joseph Shotton
  • Publication number: 20120219209
    Abstract: Image labeling with global parameters is described. In an embodiment a pose estimation system executes automatic body part labeling. For example, the system may compute joint recognition or body part segmentation for a gaming application. In another example, the system may compute organ labels for a medical imaging application. In an example, at least one global parameter, for example body height is computed for each of the images to be labeled. In an example, the global parameter is used to modify an image labeling process. For example the global parameter may be used to modify the input image to a canonical scale. In another example, the global parameter may be used to adaptively modify previously stored parameters of the image labeling process. In an example, the previously stored parameters may be computed from a reduced set of training data.
    Type: Application
    Filed: February 25, 2011
    Publication date: August 30, 2012
    Applicant: Microsoft Corporation
    Inventors: Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Blake, Inmar-Ella Givoni
  • Publication number: 20120207346
    Abstract: An object detection system is disclosed herein. The object detection system allows detection of one or more objects of interest using a probabilistic model. The probabilistic model may include voting elements usable to determine which hypotheses for locations of objects are probabilistically valid. The object detection system may apply an optimization algorithm such as a simple greedy algorithm to find hypotheses that optimize or maximize a posterior probability or log-posterior of the probabilistic model or a hypothesis receiving a maximal probabilistic vote from the voting elements in a respective iteration of the algorithm. Locations of detected objects may then be ascertained based on the found hypotheses.
    Type: Application
    Filed: February 10, 2011
    Publication date: August 16, 2012
    Applicant: Microsoft Corporation
    Inventors: Pushmeet Kohli, Victor Lempitsky, Olga Barinova
  • Publication number: 20120196679
    Abstract: Real-time camera tracking using depth maps is described. In an embodiment depth map frames are captured by a mobile depth camera at over 20 frames per second and used to dynamically update in real-time a set of registration parameters which specify how the mobile depth camera has moved. In examples the real-time camera tracking output is used for computer game applications and robotics. In an example, an iterative closest point process is used with projective data association and a point-to-plane error metric in order to compute the updated registration parameters. In an example, a graphics processing unit (GPU) implementation is used to optimize the error metric in real-time. In some embodiments, a dense 3D model of the mobile camera environment is used.
    Type: Application
    Filed: January 31, 2011
    Publication date: August 2, 2012
    Applicant: Microsoft Corporation
    Inventors: Richard Newcombe, Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Pushmeet Kohli, Andrew Fitzgibbon, Stephen Edward Hodges, David Alexander Butler
  • Publication number: 20120195471
    Abstract: Moving object segmentation using depth images is described. In an example, a moving object is segmented from the background of a depth image of a scene received from a mobile depth camera. A previous depth image of the scene is retrieved, and compared to the current depth image using an iterative closest point algorithm. The iterative closest point algorithm includes a determination of a set of points that correspond between the current depth image and the previous depth image. During the determination of the set of points, one or more outlying points are detected that do not correspond between the two depth images, and the image elements at these outlying points are labeled as belonging to the moving object. In examples, the iterative closest point algorithm is executed as part of an algorithm for tracking the mobile depth camera, and hence the segmentation does not add substantial additional computational complexity.
    Type: Application
    Filed: January 31, 2011
    Publication date: August 2, 2012
    Applicant: Microsoft Corporation
    Inventors: Richard NEWCOMBE, Shahram IZADI, Otmar HILLIGES, David KIM, David MOLYNEAUX, Jamie Daniel Joseph SHOTTON, Pushmeet KOHLI, Andrew FITZGIBBON, Stephen Edward HODGES, David Alexander BUTLER
  • Publication number: 20120194650
    Abstract: Systems and methods for reducing interference between multiple infra-red depth cameras are described. In an embodiment, the system comprises multiple infra-red sources, each of which projects a structured light pattern into the environment. A controller is used to control the sources in order to reduce the interference caused by overlapping light patterns. Various methods are described including: cycling between the different sources, where the cycle used may be fixed or may change dynamically based on the scene detected using the cameras; setting the wavelength of each source so that overlapping patterns are at different wavelengths; moving source-camera pairs in independent motion patterns; and adjusting the shape of the projected light patterns to minimize overlap. These methods may also be combined in any way. In another embodiment, the system comprises a single source and a mirror system is used to cast the projected structured light pattern around the environment.
    Type: Application
    Filed: January 31, 2011
    Publication date: August 2, 2012
    Applicant: Microsoft Corporation
    Inventors: Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Stephen Edward Hodges, David Alexander Butler, Andrew Fitzgibbon, Pushmeet Kohli