Patents by Inventor Shahram Izadi

Shahram Izadi has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10331222
    Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.
    Type: Grant
    Filed: May 24, 2016
    Date of Patent: June 25, 2019
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
  • Patent number: 10311282
    Abstract: Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.
    Type: Grant
    Filed: September 11, 2017
    Date of Patent: June 4, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Cem Keskin, Christoph Rhemann, Toby Sharp, Duncan Paul Robertson, Pushmeet Kohli, Andrew William Fitzgibbon, Shahram Izadi
  • Patent number: 10234941
    Abstract: A wearable sensor for tracking articulated body parts is described such as a wrist-worn device which enables 3D tracking of fingers and optionally also the arm and hand without the need to wear a glove or markers on the hand. In an embodiment a camera captures images of an articulated part of a body of a wearer of the device and an articulated model of the body part is tracked in real time to enable gesture-based control of a separate computing device such as a smart phone, laptop computer or other computing device. In examples the device has a structured illumination source and a diffuse illumination source for illuminating the articulated body part.
    Type: Grant
    Filed: October 4, 2012
    Date of Patent: March 19, 2019
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: David Kim, Shahram Izadi, Otmar Hilliges, David Alexander Butler, Stephen Hodges, Patrick Luke Olivier, Jiawen Chen, Iason Oikonomidis
  • Publication number: 20180356883
    Abstract: An electronic device estimates a pose of a face by fitting a generative face model mesh to a depth map based on vertices of the face model mesh that are estimated to be visible from the point of view of a depth camera. A face tracking module of the electronic device receives a depth image of a face from a depth camera and generates a depth map of the face based on the depth image. The face tracking module identifies a pose of the face by fitting a face model mesh to the pixels of a depth map that correspond to the vertices of the face model mesh that are estimated to be visible from the point of view of the depth camera.
    Type: Application
    Filed: June 7, 2018
    Publication date: December 13, 2018
    Inventors: Julien Pascal Christophe VALENTIN, Jonathan James TAYLOR, Shahram IZADI
  • Publication number: 20180350105
    Abstract: An electronic device estimates a pose of a hand by volumetrically deforming a signed distance field using a skinned tetrahedral mesh to locate a local minimum of an energy function, wherein the local minimum corresponds to the hand pose. The electronic device identifies a pose of the hand by fitting an implicit surface model of a hand to the pixels of a depth image that correspond to the hand. The electronic device uses a skinned tetrahedral mesh to warp space from a base pose to a deformed pose to define an articulated signed distance field from which the hand tracking module derives candidate poses of the hand. The electronic device then minimizes an energy function based on the distance of each corresponding pixel to identify the candidate pose that most closely approximates the pose of the hand.
    Type: Application
    Filed: May 31, 2018
    Publication date: December 6, 2018
    Inventors: Jonathan James TAYLOR, Vladimir TANKOVICH, Danhang TANG, Cem KESKIN, Adarsh Prakash Murthy KOWDLE, Philip L. DAVIDSON, Shahram IZADI, David KIM
  • Publication number: 20180350088
    Abstract: An electronic device estimates a pose of one or more subjects in an environment based on estimating a correspondence between a data volume containing a data mesh based on a current frame captured by a depth camera and a reference volume containing a plurality of fused prior data frames based on spectral embedding and performing bidirectional non-rigid matching between the reference volume and the current data frame to refine the correspondence so as to support location-based functionality. The electronic device predicts correspondences between the data volume and the reference volume based on spectral embedding. The correspondences provide constraints that accelerate the convergence between the data volume and the reference volume. By tracking changes between the current data mesh frame and the reference volume, the electronic device avoids tracking failures that can occur when relying solely on a previous data mesh frame.
    Type: Application
    Filed: May 31, 2018
    Publication date: December 6, 2018
    Inventors: Mingsong DOU, Sean Ryan FANELLO, Adarsh Prakash Murthy KOWDLE, Christoph RHEMANN, Sameh KHAMIS, Philip L. DAVIDSON, Shahram IZADI, Vladimir Tankovich
  • Publication number: 20180350087
    Abstract: An electronic device estimates a depth map of an environment based on stereo depth images captured by depth cameras having exposure times that are offset from each other in conjunction with illuminators pulsing illumination patterns into the environment. A processor of the electronic device matches small sections of the depth images from the cameras to each other and to corresponding patches of immediately preceding depth images (e.g., a spatio-temporal image patch “cube”). The processor computes a matching cost for each spatio-temporal image patch cube by converting each spatio-temporal image patch into binary codes and defining a cost function between two stereo image patches as the difference between the binary codes. The processor minimizes the matching cost to generate a disparity map, and optimizes the disparity map by rejecting outliers using a decision tree with learned pixel offsets and refining subpixels to generate a depth map of the environment.
    Type: Application
    Filed: May 31, 2018
    Publication date: December 6, 2018
    Inventors: Adarsh Prakash Murthy KOWDLE, Vladimir TANKOVICH, Danhang TANG, Cem KESKIN, Jonathan James Taylor, Philip L. DAVIDSON, Shahram IZADI, Sean Ryan FANELLO, Julien Pascal Christophe VALENTIN, Christoph RHEMANN, Mingsong DOU, Sameh KHAMIS, David KIM
  • Publication number: 20180352213
    Abstract: A first and second image of a scene are captured. Each of a plurality of pixels in the first image is associated with a disparity value. An image patch associated with each of the plurality of pixels of the first image and the second image is mapped into a binary vector. Thus, values of pixels in an image are mapped to a binary space using a function that preserves characteristics of values of the pixels. The difference between the binary vector associated with each of the plurality of pixels of the first image and its corresponding binary vector in the second image designated by the disparity value associated with each of the plurality of pixels of the first image is determined. Based on the determined difference between binary vectors, correspondence between the plurality of pixels of the first image and the second image is established.
    Type: Application
    Filed: June 4, 2018
    Publication date: December 6, 2018
    Inventors: Julien Pascal Christophe Valentin, Sean Ryan Fanello, Adarsh Prakash Murthy Kowdle, Christoph Rhemann, Vladimir Tankovich, Philip L. Davidson, Shahram Izadi
  • Patent number: 10148784
    Abstract: A dual-mode, dual-display shared resource computing (SRC) device is usable to stream SRC content from a host SRC device while in an on-line mode and maintain functionality with the content during an off-line mode. Such remote SRC devices can be used to maintain multiple user-specific caches and to back-up cached content for multi-device systems.
    Type: Grant
    Filed: December 14, 2016
    Date of Patent: December 4, 2018
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Shahram Izadi, Behrooz Chitsaz
  • Patent number: 10110881
    Abstract: Model fitting from raw time of flight image data is described, for example, to track position and orientation of a human hand or other entity. In various examples, raw image data depicting the entity is received from a time of flight camera. A 3D model of the entity is accessed and used to render, from the 3D model, simulations of raw time of flight image data depicting the entity in a specified pose/shape. The simulated raw image data and at least part of the received raw image data are compared and on the basis of the comparison, parameters of the entity are computed.
    Type: Grant
    Filed: October 30, 2014
    Date of Patent: October 23, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Toby Sharp, Jonathan James Taylor, Pushmeet Kohli, Shahram Izadi, Andrew William Fitzgibbon, Reinhard Sebastian Bernhard Nowozin
  • Publication number: 20180300588
    Abstract: Values of pixels in an image are mapped to a binary space using a first function that preserves characteristics of values of the pixels. Labels are iteratively assigned to the pixels in the image in parallel based on a second function. The label assigned to each pixel is determined based on values of a set of nearest-neighbor pixels. The first function is trained to map values of pixels in a set of training images to the binary space and the second function is trained to assign labels to the pixels in the set of training images. Considering only the nearest neighbors in the inference scheme results in a computational complexity that is independent of the size of the solution space and produces sufficient approximations of the true distribution when the solution for each pixel is most likely found in a small subset of the set of potential solutions.
    Type: Application
    Filed: March 19, 2018
    Publication date: October 18, 2018
    Inventors: Sean Ryan FANELLO, Julien Pascal Christophe VALENTIN, Adarsh Prakash Murthy KOWDLE, Christoph RHEMANN, Vladimir TANKOVICH, Philip L. DAVIDSON, Shahram IZADI
  • Publication number: 20180285697
    Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.
    Type: Application
    Filed: February 13, 2018
    Publication date: October 4, 2018
    Inventors: Jamie Daniel Joseph Shotton, Benjamin Michael Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, Andrew William Fitzgibbon
  • Patent number: 10049458
    Abstract: Systems and methods for reducing interference between multiple infra-red depth cameras are described. In an embodiment, the system comprises multiple infra-red sources, each of which projects a structured light pattern into the environment. A controller is used to control the sources in order to reduce the interference caused by overlapping light patterns. Various methods are described including: cycling between the different sources, where the cycle used may be fixed or may change dynamically based on the scene detected using the cameras; setting the wavelength of each source so that overlapping patterns are at different wavelengths; moving source-camera pairs in independent motion patterns; and adjusting the shape of the projected light patterns to minimize overlap. These methods may also be combined in any way. In another embodiment, the system comprises a single source and a mirror system is used to cast the projected structured light pattern around the environment.
    Type: Grant
    Filed: January 20, 2016
    Date of Patent: August 14, 2018
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Shahram Izadi, David Molyneaux, Otmar Hilliges, David Kim, Jamie Daniel Joseph Shotton, Stephen Edward Hodges, David Alexander Butler, Andrew Fitzgibbon, Pushmeet Kohli
  • Publication number: 20180173947
    Abstract: The subject disclosure is directed towards active depth sensing based upon moving a projector or projector component to project a moving light pattern into a scene. Via the moving light pattern captured over a set of frames, e.g., by a stereo camera system, and estimating light intensity at sub-pixel locations in each stereo frame, higher resolution depth information at a sub-pixel level may be computed than is captured by the native camera resolution.
    Type: Application
    Filed: February 5, 2018
    Publication date: June 21, 2018
    Inventors: Sing Bing KANG, Shahram IZADI
  • Patent number: 10001845
    Abstract: A 3D silhouette sensing system is described which comprises a stereo camera and a light source. In an embodiment, a 3D sensing module triggers the capture of pairs of images by the stereo camera at the same time that the light source illuminates the scene. A series of pairs of images may be captured at a predefined frame rate. Each pair of images is then analyzed to track both a retroreflector in the scene, which can be moved relative to the stereo camera, and an object which is between the retroreflector and the stereo camera and therefore partially occludes the retroreflector. In processing the image pairs, silhouettes are extracted for each of the retroreflector and the object and these are used to generate a 3D contour for each of the retroreflector and object.
    Type: Grant
    Filed: June 14, 2017
    Date of Patent: June 19, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: David Kim, Shahram Izadi, Vivek Pradeep, Steven Bathiche, Timothy Andrew Large, Karlton David Powell
  • Patent number: 9955140
    Abstract: An initial candidate foreground region is identified within an infrared image that includes pixels exhibiting infrared intensity values within a pre-defined range. A depth of surfaces within the initial candidate foreground region is estimated based on infrared intensity values the pixels of the initial candidate foreground region. The initial candidate foreground region is expanded to an expanded candidate foreground region based on a body-model estimate. The body model estimate is seeded with one or more of the initial candidate foreground region, the depth of surfaces, and/or a face of a human subject identified by facial recognition. Each pixel of the infrared image is identified as either a foreground pixel or a background pixel based on a distance of that pixel relative to the expanded candidate foreground region. Pixels identified as background pixels may be modified within a corresponding visible light image.
    Type: Grant
    Filed: March 11, 2015
    Date of Patent: April 24, 2018
    Assignee: MICROSOFT TECHNOLOGY LICENSING, LLC
    Inventors: Christoph Rhemann, Emad Barsoum, Yao Shen, Simon P. Stachniak, Shahram Izadi
  • Publication number: 20180106905
    Abstract: Detecting material properties such reflectivity, true color and other properties of surfaces in a real world environment is described in various examples using a single hand-held device. For example, the detected material properties are calculated using a photometric stereo system which exploits known relationships between lighting conditions, surface normals, true color and image intensity. In examples, a user moves around in an environment capturing color images of surfaces in the scene from different orientations under known lighting conditions. In various examples, surfaces normals of patches of surfaces are calculated using the captured data to enable fine detail such as human hair, netting, textured surfaces to be modeled. In examples, the modeled data is used to render images depicting the scene with realism or to superimpose virtual graphics on the real world in a realistic manner.
    Type: Application
    Filed: December 18, 2017
    Publication date: April 19, 2018
    Inventors: Otmar HILLIGES, Malte Hanno WEISS, Shahram IZADI, David KIM, Carsten Curt Eckard ROTHER
  • Patent number: 9940553
    Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.
    Type: Grant
    Filed: February 22, 2013
    Date of Patent: April 10, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Jamie Daniel Joseph Shotton, Benjamin Michael Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, Andrew William Fitzgibbon
  • Patent number: 9928420
    Abstract: The subject disclosure is directed towards a high resolution, high frame rate, robust stereo depth system. The system provides depth data in varying conditions based upon stereo matching of images, including actively illuminated IR images in some implementations. A clean IR or RGB image may be captured and used with any other captured images in some implementations. Clean IR images may be obtained by using a notch filter to filter out the active illumination pattern. IR stereo cameras, a projector, broad spectrum IR LEDs and one or more other cameras may be incorporated into a single device, which may also include image processing components to internally compute depth data in the device for subsequent output.
    Type: Grant
    Filed: June 24, 2013
    Date of Patent: March 27, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Adam G. Kirk, Oliver A. Whyte, Sing Bing Kang, Charles Lawrence Zitnick, III, Richard S. Szeliski, Shahram Izadi, Christoph Rhemann, Andreas Georgiou, Avronil Bhattacharjee
  • Patent number: 9922249
    Abstract: The subject disclosure is directed towards active depth sensing based upon moving a projector or projector component to project a moving light pattern into a scene. Via the moving light pattern captured over a set of frames, e.g., by a stereo camera system, and estimating light intensity at sub-pixel locations in each stereo frame, higher resolution depth information at a sub-pixel level may be computed than is captured by the native camera resolution.
    Type: Grant
    Filed: June 21, 2013
    Date of Patent: March 20, 2018
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Sing Bing Kang, Shahram Izadi