Patents by Inventor Pushmeet Kohli

Pushmeet Kohli has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20200372654
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a plurality of possible segmentations of an image. In one aspect, a method comprises: receiving a request to generate a plurality of possible segmentations of an image; sampling a plurality of latent variables from a latent space, wherein each latent variable is sampled from the latent space in accordance with a respective probability distribution over the latent space that is determined based on the image; generating a plurality of possible segmentations of the image, comprising, for each latent variable, processing the image and the latent variable using a segmentation neural network having a plurality of segmentation neural network parameters to generate the possible segmentation of the image; and providing the plurality of possible segmentations of the image in response to the request.
    Type: Application
    Filed: May 22, 2020
    Publication date: November 26, 2020
    Inventors: Simon Kohl, Bernardino Romera-Paredes, Danilo Jimenez Rezende, Seyed Mohammadali Eslami, Pushmeet Kohli, Andrew Zisserman, Olaf Ronneberger
  • Patent number: 10832163
    Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.
    Type: Grant
    Filed: October 28, 2016
    Date of Patent: November 10, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Toby Sharp, Pushmeet Kohli, Reinhard Sebastian Bernhard Nowozin, John Michael Winn, Antonio Criminisi
  • Patent number: 10817552
    Abstract: Generally discussed herein are devices, systems, and methods for encoding input-output examples. A method of generating a program using an encoding of input-output examples, may include processing an input example of the input-output examples, using a first long short term memory (LSTM) neural network, one character at a time to produce an input feature vector, processing an output example associated with the input example in the input-output examples, using the LSTM neural network, one character at a time to produce an output feature vector, determining (a) a cross-correlation between the input feature vector and the output feature vector or (b) previously computed feature vectors for a different input-output example that are sufficiently close to the input feature vector and the output feature vector, respectively, and using the determined cross-correlation or previously computed vector, generating a program consistent with the input example and the output example.
    Type: Grant
    Filed: March 27, 2017
    Date of Patent: October 27, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Abdelrahman S. A. Mohamed, Pushmeet Kohli, Rishabh Singh, Emilio Parisotto
  • Patent number: 10795645
    Abstract: Described are systems, methods, and computer-readable media for program generation in a domain-specific language based on input-output examples. In accordance with various embodiments, a neural-network-based program generation model conditioned on an encoded set of input-output examples is used to generate a program tree by iteratively expanding a partial program tree, beginning with a root node and ending when all leaf nodes are terminal.
    Type: Grant
    Filed: March 27, 2017
    Date of Patent: October 6, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Abdelrahman S. A. Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, Pushmeet Kohli, Emilio Parisotto
  • Patent number: 10762443
    Abstract: Crowdsourcing systems with machine learning are described. Specifically, item-label inference methods and systems are presented, for example, to provide aggregated answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples, an item-label inference system infers variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, an item-label inference system provides aggregated labels while considering the inferred worker characteristics and the inferred characteristics of the worker communities. In examples the item-label inference system provides uncertainty information associated with the inference results for selecting workers and generating future tasks.
    Type: Grant
    Filed: July 17, 2017
    Date of Patent: September 1, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Matteo Venanzi, John Philip Guiver, Gabriella Kazai, Pushmeet Kohli, Milad Shokouhi
  • Patent number: 10761612
    Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.
    Type: Grant
    Filed: May 15, 2019
    Date of Patent: September 1, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
  • Publication number: 20200234145
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent interacting with an environment. In one aspect, a method comprises: obtaining a graph of nodes and edges that represents an interaction history of the agent with the environment; generating an encoded representation of the graph representing the interaction history of the agent with the environment; processing an input based on the encoded representation of the graph using an action selection neural network, in accordance with current values of action selection neural network parameters, to generate an action selection output; and selecting an action from a plurality of possible actions to be performed by the agent using the action selection output generated by the action selection neural network.
    Type: Application
    Filed: January 22, 2020
    Publication date: July 23, 2020
    Inventors: Hanjun Dai, Yujia Li, Chenglong Wang, Rishabh Singh, Po-Sen Huang, Pushmeet Kohli
  • Publication number: 20190354689
    Abstract: There is described a neural network system implemented by one or more computers for determining graph similarity. The neural network system comprises one or more neural networks configured to process an input graph to generate a node state representation vector for each node of the input graph and an edge representation vector for each edge of the input graph; and process the node state representation vectors and the edge representation vectors to generate a vector representation of the input graph. The neural network system further comprises one or more processors configured to: receive a first graph; receive a second graph; generate a vector representation of the first graph; generate a vector representation of the second graph; determine a similarity score for the first graph and the second graph based upon the vector representations of the first graph and the second graph.
    Type: Application
    Filed: May 17, 2019
    Publication date: November 21, 2019
    Inventors: Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, Pushmeet Kohli
  • Publication number: 20190278380
    Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.
    Type: Application
    Filed: May 15, 2019
    Publication date: September 12, 2019
    Applicant: Microsoft Technology Licensing, LLC
    Inventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
  • Patent number: 10346856
    Abstract: Personality aggregation and web browsing is described, for example, to find personality profiles of website audiences for use in recommendation systems, advertising systems, or other web services. In an embodiment natural browsing sequences of users who have given their consent are submitted to a pattern matching process to identify personality trait scores serendipitously occurring in the sequences. In an embodiment, an aggregator combines the personality trait scores by website to obtain audience personality profiles. In an example, a machine learning process carries out the aggregation and enables audience personality profiles of other websites to be predicted. For example, a random decision forest is trained using the natural browsing sequences having identified personality trait scores and once trained, is used to predict personality trait scores of other websites.
    Type: Grant
    Filed: August 9, 2012
    Date of Patent: July 9, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Pushmeet Kohli, Filip Radlinski, Michael Stanislaw Kosinski
  • Patent number: 10331222
    Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.
    Type: Grant
    Filed: May 24, 2016
    Date of Patent: June 25, 2019
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
  • Patent number: 10311282
    Abstract: Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.
    Type: Grant
    Filed: September 11, 2017
    Date of Patent: June 4, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Cem Keskin, Christoph Rhemann, Toby Sharp, Duncan Paul Robertson, Pushmeet Kohli, Andrew William Fitzgibbon, Shahram Izadi
  • Patent number: 10235605
    Abstract: Image labeling is described, for example, to recognize body organs in a medical image, to label body parts in a depth image of a game player, to label objects in a video of a scene. In various embodiments an automated classifier uses geodesic features of an image, and optionally other types of features, to semantically segment an image. For example, the geodesic features relate to a distance between image elements, the distance taking into account information about image content between the image elements. In some examples the automated classifier is an entangled random decision forest in which data accumulated at earlier tree levels is used to make decisions at later tree levels. In some examples the automated classifier has auto-context by comprising two or more random decision forests. In various examples parallel processing and look up procedures are used.
    Type: Grant
    Filed: April 10, 2013
    Date of Patent: March 19, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Antonio Criminisi, Peter Kontschieder, Pushmeet Kohli, Jamie Daniel Joseph Shotton
  • Patent number: 10154041
    Abstract: A method of controlling access to content such as web sites on the intranet or interne is described. For example, the method comprises receiving an address of content to be accessed and obtaining similarity of the address to previously labeled addresses of other content items. The similarity is based on co-occurrence of addresses of content items in records of browsing sessions from many consenting users. For example, a browsing session record comprises addresses of content items accessed by a user in a time period during which the user is actively accessing content. A co-occurrence of addresses of content items is the existence of the addresses in the same browsing session record. Access to the content is then controlled on the basis of the similarity.
    Type: Grant
    Filed: January 13, 2015
    Date of Patent: December 11, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Pushmeet Kohli, Yoram Bachrach, Filip Radlinski, Ulrich Paquet, Li Quan Khoo
  • Patent number: 10127497
    Abstract: An inference engine is described for efficient machine learning. For example, an inference engine executes a plurality of ordered steps to carry out inference on the basis of observed data. For each step, a plurality of inputs to the step are received. A predictor predicts an output of the step and computes uncertainty of the prediction. Either the predicted output or a known output is selected on the basis of the uncertainty. If the known output is selected, the known output is computed, (for example, using a resource intensive, accurate process). The predictor is retrained using the known output and the plurality of inputs of the step as training data. For example, computing the prediction is fast and efficient as compared with computing the known output.
    Type: Grant
    Filed: October 14, 2014
    Date of Patent: November 13, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Seyed Mohammadali Eslami, Daniel Stefan Tarlow, Pushmeet Kohli, John Winn
  • Patent number: 10110881
    Abstract: Model fitting from raw time of flight image data is described, for example, to track position and orientation of a human hand or other entity. In various examples, raw image data depicting the entity is received from a time of flight camera. A 3D model of the entity is accessed and used to render, from the 3D model, simulations of raw time of flight image data depicting the entity in a specified pose/shape. The simulated raw image data and at least part of the received raw image data are compared and on the basis of the comparison, parameters of the entity are computed.
    Type: Grant
    Filed: October 30, 2014
    Date of Patent: October 23, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Toby Sharp, Jonathan James Taylor, Pushmeet Kohli, Shahram Izadi, Andrew William Fitzgibbon, Reinhard Sebastian Bernhard Nowozin
  • Publication number: 20180276535
    Abstract: Generally discussed herein are devices, systems, and methods for encoding input-output examples. A method of generating a program using an encoding of input-output examples, may include processing an input example of the input-output examples, using a first long short term memory (LSTM) neural network, one character at a time to produce an input feature vector, processing an output example associated with the input example in the input-output examples, using the LSTM neural network, one character at a time to produce an output feature vector, determining (a) a cross-correlation between the input feature vector and the output feature vector or (b) previously computed feature vectors for a different input-output example that are sufficiently close to the input feature vector and the output feature vector, respectively, and using the determined cross-correlation or previously computed vector, generating a program consistent with the input example and the output example.
    Type: Application
    Filed: March 27, 2017
    Publication date: September 27, 2018
    Inventors: Abdelrahman S.A. Mohamed, Pushmeet Kohli, Rishabh Singh, Emilio Parisotto
  • Publication number: 20180275967
    Abstract: Described are systems, methods, and computer-readable media for program generation in a domain-specific language based on input-output examples. In accordance with various embodiments, a neural-network-based program generation model conditioned on an encoded set of input-output examples is used to generate a program tree by iteratively expanding a partial program tree, beginning with a root node and ending when all leaf nodes are terminal.
    Type: Application
    Filed: March 27, 2017
    Publication date: September 27, 2018
    Inventors: Abdelrahman S.A. Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, Pushmeet Kohli, Emilio Parisotto
  • Patent number: 10049458
    Abstract: Systems and methods for reducing interference between multiple infra-red depth cameras are described. In an embodiment, the system comprises multiple infra-red sources, each of which projects a structured light pattern into the environment. A controller is used to control the sources in order to reduce the interference caused by overlapping light patterns. Various methods are described including: cycling between the different sources, where the cycle used may be fixed or may change dynamically based on the scene detected using the cameras; setting the wavelength of each source so that overlapping patterns are at different wavelengths; moving source-camera pairs in independent motion patterns; and adjusting the shape of the projected light patterns to minimize overlap. These methods may also be combined in any way. In another embodiment, the system comprises a single source and a mirror system is used to cast the projected structured light pattern around the environment.
    Type: Grant
    Filed: January 20, 2016
    Date of Patent: August 14, 2018
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Stephen Edward Hodges, David Alexander Butler, Andrew Fitzgibbon, Pushmeet Kohli
  • Patent number: 9959675
    Abstract: A “Layout Optimizer” provides various real-time iterative constraint-satisfaction methodologies that use constraint-based frameworks to generate optimized layouts that map or embed virtual objects into environments. The term environment refers to combinations of environmental characteristics, including, but not limited to, 2D or 3D scene geometry or layout, scene colors, patterns, and/or textures, scene illumination, scene heat sources, fixed or moving people, objects or fluids, etc., any of which may evolve or change over time. A set of parameters are specified or selected for each object. Further, the environmental characteristics are determined automatically or specified by users. Relationships between objects and/or the environment derived from constraints associated with objects and the environment are then used to iteratively determine optimized self-consistent and scene-consistent object layouts.
    Type: Grant
    Filed: June 9, 2014
    Date of Patent: May 1, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ran Gal, Pushmeet Kohli, Eyal Ofek, Lior Shapira