Patents by Inventor Jamie Daniel Joseph Shotton
Jamie Daniel Joseph Shotton has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11710309Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.Type: GrantFiled: February 13, 2018Date of Patent: July 25, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Jamie Daniel Joseph Shotton, Benjamin Michael Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, Andrew William Fitzgibbon
-
Publication number: 20230116250Abstract: Computing an output image of a dynamic scene. A value of E is selected which is a parameter describing desired dynamic content of the scene in the output image. Using selected intrinsic camera parameters and a selected viewpoint, for individual pixels of the output image to be generated, the method computes a ray that goes from a virtual camera through the pixel into the dynamic scene. For individual ones of the rays, sample at least one point along the ray. For individual ones of the sampled points, a viewing direction being a direction of the corresponding ray, and E, query a machine learning model to produce colour and opacity values at the sampled point with the dynamic content of the scene as specified by E. For individual ones of the rays, apply a volume rendering method to the colour and opacity values computed along that ray, to produce a pixel value of the output image.Type: ApplicationFiled: December 13, 2022Publication date: April 13, 2023Inventors: Marek Adam KOWALSKI, Matthew Alastair JOHNSON, Jamie Daniel Joseph SHOTTON
-
Patent number: 11551405Abstract: Computing an output image of a dynamic scene. A value of E is selected which is a parameter describing desired dynamic content of the scene in the output image. Using selected intrinsic camera parameters and a selected viewpoint, for individual pixels of the output image to be generated, the method computes a ray that goes from a virtual camera through the pixel into the dynamic scene. For individual ones of the rays, sample at least one point along the ray. For individual ones of the sampled points, a viewing direction being a direction of the corresponding ray, and E, query a machine learning model to produce colour and opacity values at the sampled point with the dynamic content of the scene as specified by E. For individual ones of the rays, apply a volume rendering method to the colour and opacity values computed along that ray, to produce a pixel value of the output image.Type: GrantFiled: July 13, 2020Date of Patent: January 10, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Marek Adam Kowalski, Matthew Alastair Johnson, Jamie Daniel Joseph Shotton
-
Publication number: 20220284655Abstract: There is a region of interest of a synthetic image depicting an object from a class of objects. A trained neural image generator, having been trained to map embeddings from a latent space to photorealistic images of objects in the class, is accessed. A first embedding is computed from the latent space, the first embedding corresponding to an image which is similar to the region of interest while maintaining photorealistic appearance. A second embedding is computed from the latent space, the second embedding corresponding to an image which matches the synthetic image. Blending of the first embedding and the second embedding is done to form a blended embedding. At least one output image is generated from the blended embedding, the output image being more photorealistic than the synthetic image.Type: ApplicationFiled: May 23, 2022Publication date: September 8, 2022Inventors: Stephan Joachim GARBIN, Marek Adam KOWALSKI, Matthew Alastair JOHNSON, Tadas BALTRUSAITIS, Martin DE LA GORCE, Virginia ESTELLERS CASAS, Sebastian Karol DZIADZIO, Jamie Daniel Joseph SHOTTON
-
Patent number: 11354846Abstract: There is a region of interest of a synthetic image depicting an object from a class of objects. A trained neural image generator, having been trained to map embeddings from a latent space to photorealistic images of objects in the class, is accessed. A first embedding is computed from the latent space, the first embedding corresponding to an image which is similar to the region of interest while maintaining photorealistic appearance. A second embedding is computed from the latent space, the second embedding corresponding to an image which matches the synthetic image. Blending of the first embedding and the second embedding is done to form a blended embedding. At least one output image is generated from the blended embedding, the output image being more photorealistic than the synthetic image.Type: GrantFiled: June 29, 2020Date of Patent: June 7, 2022Assignee: Microsoft Technology Licensing, LLCInventors: Stephan Joachim Garbin, Marek Adam Kowalski, Matthew Alastair Johnson, Tadas Baltrusaitis, Martin De La Gorce, Virginia Estellers Casas, Sebastian Karol Dziadzio, Jamie Daniel Joseph Shotton
-
Publication number: 20210390761Abstract: Computing an output image of a dynamic scene. A value of E is selected which is a parameter describing desired dynamic content of the scene in the output image. Using selected intrinsic camera parameters and a selected viewpoint, for individual pixels of the output image to be generated, the method computes a ray that goes from a virtual camera through the pixel into the dynamic scene. For individual ones of the rays, sample at least one point along the ray. For individual ones of the sampled points, a viewing direction being a direction of the corresponding ray, and E, query a machine learning model to produce colour and opacity values at the sampled point with the dynamic content of the scene as specified by E. For individual ones of the rays, apply a volume rendering method to the colour and opacity values computed along that ray, to produce a pixel value of the output image.Type: ApplicationFiled: July 13, 2020Publication date: December 16, 2021Inventors: Marek Adam KOWALSKI, Matthew Alastair JOHNSON, Jamie Daniel Joseph SHOTTON
-
Publication number: 20210343063Abstract: There is a region of interest of a synthetic image depicting an object from a class of objects. A trained neural image generator, having been trained to map embeddings from a latent space to photorealistic images of objects in the class, is accessed. A first embedding is computed from the latent space, the first embedding corresponding to an image which is similar to the region of interest while maintaining photorealistic appearance. A second embedding is computed from the latent space, the second embedding corresponding to an image which matches the synthetic image. Blending of the first embedding and the second embedding is done to form a blended embedding. At least one output image is generated from the blended embedding, the output image being more photorealistic than the synthetic image.Type: ApplicationFiled: June 29, 2020Publication date: November 4, 2021Inventors: Stephan Joachim GARBIN, Marek Adam KOWALSKI, Matthew Alastair JOHNSON, Tadas BALTRUSAITIS, Martin DE LA GORCE, Virginia ESTELLERS CASAS, Sebastian Karol DZIADZIO, Jamie Daniel Joseph SHOTTON
-
Patent number: 11107242Abstract: In various examples there is an apparatus for detecting position and orientation of an object. The apparatus comprises a memory storing at least one frame of captured sensor data depicting the object. The apparatus also comprises a trained machine learning system configured to receive the frame of the sensor data and to compute a plurality of two dimensional positions in the frame. Each predicted two dimensional position is a position of sensor data in the frame depicting a keypoint, where a keypoint is a pre-specified 3D position relative to the object. At least one of the keypoints is a floating keypoint depicting a pre-specified position relative to the object, lying inside or outside the object's surface. The apparatus comprises a pose detector which computes the three dimensional position and orientation of the object using the predicted two dimensional positions and outputs the computed three dimensional position and orientation.Type: GrantFiled: March 22, 2019Date of Patent: August 31, 2021Assignee: Microsoft Technology Licensing, LLCInventors: Andrew William Fitzgibbon, Erroll William Wood, Jingjing Shen, Thomas Joseph Cashman, Jamie Daniel Joseph Shotton
-
Patent number: 10832163Abstract: Memory facilitation using directed acyclic graphs is described, for example, where a plurality of directed acyclic graphs are trained for gesture recognition from human skeletal data, or to estimate human body joint positions from depth images for gesture detection. In various examples directed acyclic graphs are grown during training using a training objective which takes into account both connection patterns between nodes and split function parameter values. For example, a layer of child nodes is grown and connected to a parent layer of nodes using an initialization strategy. In examples, various local search processes are used to find good combinations of connection patterns and split function parameters.Type: GrantFiled: October 28, 2016Date of Patent: November 10, 2020Assignee: Microsoft Technology Licensing, LLCInventors: Jamie Daniel Joseph Shotton, Toby Sharp, Pushmeet Kohli, Reinhard Sebastian Bernhard Nowozin, John Michael Winn, Antonio Criminisi
-
Patent number: 10761612Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.Type: GrantFiled: May 15, 2019Date of Patent: September 1, 2020Assignee: Microsoft Technology Licensing, LLCInventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
-
Publication number: 20200226786Abstract: In various examples there is an apparatus for detecting position and orientation of an object. The apparatus comprises a memory storing at least one frame of captured sensor data depicting the object. The apparatus also comprises a trained machine learning system configured to receive the frame of the sensor data and to compute a plurality of two dimensional positions in the frame. Each predicted two dimensional position is a position of sensor data in the frame depicting a keypoint, where a keypoint is a pre-specified 3D position relative to the object. At least one of the keypoints is a floating keypoint depicting a pre-specified position relative to the object, lying inside or outside the object's surface. The apparatus comprises a pose detector which computes the three dimensional position and orientation of the object using the predicted two dimensional positions and outputs the computed three dimensional position and orientation.Type: ApplicationFiled: March 22, 2019Publication date: July 16, 2020Inventors: Andrew William FITZGIBBON, Erroll William WOOD, Jingjing SHEN, Thomas Joseph CASHMAN, Jamie Daniel Joseph SHOTTON
-
Publication number: 20190392587Abstract: A system to predict a location of a feature point of an articulated object from a plurality of data points relating to the articulated object of which some possess and some are missing 2D location data. The data points are input into a machine learning model that is trained to predict 2D location data for each feature point of the articulated object that was missing location data.Type: ApplicationFiled: August 9, 2018Publication date: December 26, 2019Inventors: Sebastian NOWOZIN, Federica BOGO, Jamie Daniel Joseph SHOTTON, Jan STUEHMER
-
Publication number: 20190278380Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.Type: ApplicationFiled: May 15, 2019Publication date: September 12, 2019Applicant: Microsoft Technology Licensing, LLCInventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
-
Patent number: 10331222Abstract: In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.Type: GrantFiled: May 24, 2016Date of Patent: June 25, 2019Assignee: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: David Kim, Otmar D. Hilliges, Shahram Izadi, Patrick L. Olivier, Jamie Daniel Joseph Shotton, Pushmeet Kohli, David G. Molyneaux, Stephen E. Hodges, Andrew W. Fitzgibbon
-
Patent number: 10311282Abstract: Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.Type: GrantFiled: September 11, 2017Date of Patent: June 4, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Jamie Daniel Joseph Shotton, Cem Keskin, Christoph Rhemann, Toby Sharp, Duncan Paul Robertson, Pushmeet Kohli, Andrew William Fitzgibbon, Shahram Izadi
-
Patent number: 10304258Abstract: A ground truth engine is described which has a memory holding a plurality of captured images depicting an articulated item. A processor of the engine is configured to access a parameterized, three dimensional (3D) model of the item. An optimizer of the ground truth engine is configured to compute ground truth values of the parameters of the 3D model for individual ones of the captured images, such that the articulated item depicted in the captured image fits the 3D model, the optimizer configured to take into account feedback data from one or more humans, about accuracy of a plurality of the computed values of the parameters.Type: GrantFiled: July 24, 2017Date of Patent: May 28, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Lucas Bordeaux, Thomas Joseph Cashman, Federica Bogo, Jamie Daniel Joseph Shotton, Andrew William Fitzgibbon
-
Patent number: 10235605Abstract: Image labeling is described, for example, to recognize body organs in a medical image, to label body parts in a depth image of a game player, to label objects in a video of a scene. In various embodiments an automated classifier uses geodesic features of an image, and optionally other types of features, to semantically segment an image. For example, the geodesic features relate to a distance between image elements, the distance taking into account information about image content between the image elements. In some examples the automated classifier is an entangled random decision forest in which data accumulated at earlier tree levels is used to make decisions at later tree levels. In some examples the automated classifier has auto-context by comprising two or more random decision forests. In various examples parallel processing and look up procedures are used.Type: GrantFiled: April 10, 2013Date of Patent: March 19, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Antonio Criminisi, Peter Kontschieder, Pushmeet Kohli, Jamie Daniel Joseph Shotton
-
Patent number: 10218882Abstract: A computing device has an input configured to receive data captured by at least one capture device where the data depicts at least part of an object moving in an environment. The computing device has a tracker configured to track a real-world position and orientation of the object using the captured data. A processor at the computing device is configured to compute and output feedback about performance of the tracker, where the feedback encourages a user to adjust movement of the object for improved tracking of the object by the tracker.Type: GrantFiled: December 31, 2015Date of Patent: February 26, 2019Assignee: Microsoft Technology Licensing, LLCInventors: Jamie Daniel Joseph Shotton, Andrew William Fitzgibbon, Jonathan James Taylor, Richard Malcolm Banks, David Sweeney, Robert Corish, Abigail Jane Sellen, Eduardo Alberto Soto
-
Patent number: 10210382Abstract: Techniques for human body pose estimation are disclosed herein. Depth map images from a depth camera may be processed to calculate a probability that each pixel of the depth map is associated with one or more segments or body parts of a body. Body parts may then be constructed of the pixels and processed to define joints or nodes of those body parts. The nodes or joints may be provided to a system which may construct a model of the body from the various nodes or joints.Type: GrantFiled: December 22, 2015Date of Patent: February 19, 2019Assignee: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Jamie Daniel Joseph Shotton, Andrew William Fitzgibbon
-
Publication number: 20190026952Abstract: A ground truth engine is described which has a memory holding a plurality of captured images depicting an articulated item. A processor of the engine is configured to access a parameterized, three dimensional (3D) model of the item. An optimizer of the ground truth engine is configured to compute ground truth values of the parameters of the 3D model for individual ones of the captured images, such that the articulated item depicted in the captured image fits the 3D model, the optimizer configured to take into account feedback data from one or more humans, about accuracy of a plurality of the computed values of the parameters.Type: ApplicationFiled: July 24, 2017Publication date: January 24, 2019Inventors: Lucas BORDEAUX, Thomas Joseph CASHMAN, Federica BOGO, Jamie Daniel Joseph SHOTTON, Andrew William FITZGIBBON