Patents by Inventor Marc Levoy

Marc Levoy has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11599747
    Abstract: Apparatus and methods related to using machine learning to determine depth maps for dual pixel images of objects are provided. A computing device can receive a dual pixel image of at least a foreground object. The dual pixel image can include a plurality of dual pixels. A dual pixel of the plurality of dual pixels can include a left-side pixel and a right-side pixel that both represent light incident on a single dual pixel element used to capture the dual pixel image. The computing device can be used to train a machine learning system to determine a depth map associated with the dual pixel image. The computing device can provide the trained machine learning system.
    Type: Grant
    Filed: November 6, 2020
    Date of Patent: March 7, 2023
    Assignee: Google LLC
    Inventors: Yael Pritch Knaan, Marc Levoy, Neal Wadhwa, Rahul Garg, Sameer Ansari, Jiawen Chen
  • Publication number: 20220343525
    Abstract: Example implementations relate to joint depth prediction from dual cameras and dual pixels. An example method may involve obtaining a first set of depth information representing a scene from a first source and a second set of depth information representing the scene from a second source. The method may further involve determining, using a neural network, a joint depth map that conveys respective depths for elements in the scene. The neural network may determine the joint depth map based on a combination of the first set of depth information and the second set of depth information. In addition, the method may involve modifying an image representing the scene based on the joint depth map. For example, background portions of the image may be partially blurred based on the joint depth map.
    Type: Application
    Filed: April 27, 2020
    Publication date: October 27, 2022
    Inventors: Rahul GARG, Neal WADHWA, Sean FANELLO, Christian HAENE, Yinda ZHANG, Sergio Orts ESCOLANO, Yael Pritch KNAAN, Marc LEVOY, Shahram IZADI
  • Publication number: 20220076018
    Abstract: Apparatus and methods related to photography are provided. A computing device can receive an input image. An object detector of the computing device can determine an object region of interest of the input image that is associated with an object detected in the input image. A trained machine learning algorithm can determine an output photographic region of interest for the input image based on the object region of interest and the input image. The machine learning algorithm can be trained to identify an output photographic region of interest that is suitable for use by a photographic function for image generation. The computing device can generate an output related to the output photographic region of interest.
    Type: Application
    Filed: January 15, 2019
    Publication date: March 10, 2022
    Inventors: Ryan Geiss, Ruiduo Yang, Marc Levoy
  • Patent number: 11210799
    Abstract: A camera may capture an image of a scene and use the image to generate a first and a second subpixel image of the scene. The pair of subpixel images may be represented by a first set of subpixels and a second set of subpixels from the image respectively. Each pixel of the image may include two green subpixels that are respectively represented in the first and second subpixel images. The camera may determine a disparity between a portion of the scene as represented by the pair of subpixel images and may estimate a depth map of the scene that indicates a depth of the portion relative to other portions of the scene based on the disparity and a baseline distance between the two green subpixels. A new version of the image may be generated with a focus upon the portion and with the other portions of the scene blurred.
    Type: Grant
    Filed: December 5, 2017
    Date of Patent: December 28, 2021
    Assignee: Google LLC
    Inventors: David Jacobs, Rahul Garg, Yael Pritch Knaan, Neal Wadhwa, Marc Levoy
  • Publication number: 20210056349
    Abstract: Apparatus and methods related to using machine learning to determine depth maps for dual pixel images of objects are provided. A computing device can receive a dual pixel image of at least a foreground object. The dual pixel image can include a plurality of dual pixels. A dual pixel of the plurality of dual pixels can include a left-side pixel and a right-side pixel that both represent light incident on a single dual pixel element used to capture the dual pixel image. The computing device can be used to train a machine learning system to determine a depth map associated with the dual pixel image. The computing device can provide the trained machine learning system.
    Type: Application
    Filed: November 6, 2020
    Publication date: February 25, 2021
    Inventors: Yael Pritch Knaan, Marc Levoy, Neal Wadhwa, Rahul Garg, Sameer Ansari, Jiawen Chen
  • Patent number: 10860889
    Abstract: Apparatus and methods related to using machine learning to determine depth maps for dual pixel images of objects are provided. A computing device can receive a dual pixel image of at least a foreground object. The dual pixel image can include a plurality of dual pixels. A dual pixel of the plurality of dual pixels can include a left-side pixel and a right-side pixel that both represent light incident on a single dual pixel element used to capture the dual pixel image. The computing device can be used to train a machine learning system to determine a depth map associated with the dual pixel image. The computing device can provide the trained machine learning system.
    Type: Grant
    Filed: January 11, 2019
    Date of Patent: December 8, 2020
    Assignee: Google LLC
    Inventors: Yael Pritch Knaan, Marc Levoy, Neal Wadhwa, Rahul Garg, Sameer Ansari, Jiawen Chen
  • Publication number: 20200242788
    Abstract: A camera may capture an image of a scene and use the image to generate a first and a second subpixel image of the scene. The pair of subpixel images may be represented by a first set of subpixels and a second set of subpixels from the image respectively. Each pixel of the image may include two green subpixels that are respectively represented in the first and second subpixel images. The camera may determine a disparity between a portion of the scene as represented by the pair of subpixel images and may estimate a depth map of the scene that indicates a depth of the portion relative to other portions of the scene based on the disparity and a baseline distance between the two green subpixels. A new version of the image may be generated with a focus upon the portion and with the other portions of the scene blurred.
    Type: Application
    Filed: December 5, 2017
    Publication date: July 30, 2020
    Inventors: David Jacobs, Rahul Garg, Yael Pritch Knaan, Neal Wadhwa, Marc Levoy
  • Publication number: 20200226419
    Abstract: Apparatus and methods related to using machine learning to determine depth maps for dual pixel images of objects are provided. A computing device can receive a dual pixel image of at least a foreground object. The dual pixel image can include a plurality of dual pixels. A dual pixel of the plurality of dual pixels can include a left-side pixel and a right-side pixel that both represent light incident on a single dual pixel element used to capture the dual pixel image. The computing device can be used to train a machine learning system to determine a depth map associated with the dual pixel image. The computing device can provide the trained machine learning system.
    Type: Application
    Filed: January 11, 2019
    Publication date: July 16, 2020
    Inventors: Yael Pritch Knaan, Marc Levoy, Neal Wadhwa, Rahul Garg, Sameer Ansari, Jiawen Chen
  • Patent number: 9615012
    Abstract: A device may operate a first image-capture system to capture first image data of a scene. While the first image-capture system is capturing the first image data, the device may operate a second image-capture system to determine an updated value for the first image setting, and send an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data.
    Type: Grant
    Filed: September 30, 2013
    Date of Patent: April 4, 2017
    Assignee: Google Inc.
    Inventors: Ryan Burgess Geiss, Marc Levoy, Roman Lewkow
  • Patent number: 9013592
    Abstract: An apparatus, method and computer program product for presenting burst images are provided. The apparatus may include a processor that may be configured to receive a plurality of burst images. Each burst image may differ from the other burst images based on a variable parameter. The processor may also be configured to provide for a presentation of a sample burst image. In this regard, the sample burst image may be one of the plurality of burst images. The processor may be further configured to receive a selected location within the presentation of the sample burst image and provide for a presentation of a plurality of burst image fragments associated with each of the plurality of burst images. In this regard, the burst image fragments may be portions of each of the burst images, where the areas of each burst image may be determined based on the selected location.
    Type: Grant
    Filed: July 17, 2013
    Date of Patent: April 21, 2015
    Assignee: Nokia Corporation
    Inventors: Marc Levoy, Natasha Gelfand, Wei-Chao Chen, Kari Antero Pulli
  • Publication number: 20150092066
    Abstract: A device may operate a first image-capture system to capture first image data of a scene. While the first image-capture system is capturing the first image data, the device may operate a second image-capture system to determine an updated value for the first image setting, and send an instruction to the first image-capture system that indicates to use the updated value for the first image setting to continue to capture the first image data.
    Type: Application
    Filed: September 30, 2013
    Publication date: April 2, 2015
    Applicant: Google Inc.
    Inventors: Ryan Burgess Geiss, Marc Levoy, Roman Lewkow
  • Patent number: 8953064
    Abstract: Image data is processed to facilitate focusing and/or optical correction. According to an example embodiment of the present invention, an imaging arrangement collects light data corresponding to light passing through a particular focal plane. The light data is collected using an approach that facilitates the determination of the direction from which various portions of the light incident upon a portion of the focal plane emanate from. Using this directional information in connection with value of the light as detected by photosensors, an image represented by the light is selectively focused and/or corrected.
    Type: Grant
    Filed: September 16, 2014
    Date of Patent: February 10, 2015
    Assignee: The Board of Trustees of the Leland Stanford Junior University
    Inventors: Yi-Ren Ng, Patrick M. Hanrahan, Marc A. Levoy, Mark A. Horowitz
  • Publication number: 20150029388
    Abstract: Image data is processed to facilitate focusing and/or optical correction. According to an example embodiment of the present invention, an imaging arrangement collects light data corresponding to light passing through a particular focal plane. The light data is collected using an approach that facilitates the determination of the direction from which various portions of the light incident upon a portion of the focal, plane emanate from. Using this directional information in connection with value of the light as detected by photosensors, an image represented by the light is selectively focused and/or corrected.
    Type: Application
    Filed: September 16, 2014
    Publication date: January 29, 2015
    Applicant: THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY
    Inventors: Yi-Ren Ng, Patrick M. Hanrahan, Marc A. Levoy, Mark A. Horowitz
  • Patent number: 8866957
    Abstract: Image data is processed to facilitate focusing and/or optical correction. According to an example embodiment of the present invention, an imaging arrangement collects light data corresponding to light passing through a particular focal plane. The light data is collected using an approach that facilitates the determination of the direction from which various portions of the light incident upon a portion of the focal plane emanate from. Using this directional information in connection with value of the light as detected by photosensors, an image represented by the light is selectively focused and/or corrected.
    Type: Grant
    Filed: August 26, 2013
    Date of Patent: October 21, 2014
    Assignee: The Board of Trustees of the Leland Stanford Junior University
    Inventors: Yi-Ren Ng, Partrick M. Hanrahan, Marc A. Levoy, Mark A. Horowitz
  • Publication number: 20140049663
    Abstract: Image data is processed to facilitate focusing and/or optical correction. According to an example embodiment of the present invention, an imaging arrangement collects light data corresponding to light passing through a particular focal plane. The light data is collected using an approach that facilitates the determination of the direction from which various portions of the light incident upon a portion of the focal plane emanate from. Using this directional information in connection with value of the light as detected by photosensors, an image represented by the light is selectively focused and/or corrected.
    Type: Application
    Filed: August 26, 2013
    Publication date: February 20, 2014
    Applicant: THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY
    Inventors: Yi-Ren Ng, Patrick M. Hanrahan, Marc A. Levoy, Mark A. Horowitz
  • Publication number: 20140028892
    Abstract: Image data is processed to facilitate focusing and/or optical correction. According to an example embodiment of the present invention, an imaging arrangement collects light data corresponding to light passing through a particular focal plane. The light data is collected using an approach that facilitates the determination of the direction from which various portions of the light incident upon a portion of the focal plane emanate from. Using this directional information in connection with value of the light as detected by photosensors, an image represented by the light is selectively focused and/or corrected.
    Type: Application
    Filed: August 26, 2013
    Publication date: January 30, 2014
    Applicant: THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY
    Inventors: Yi-Ren Ng, Patrick M. Hanrahan, Marc A. Levoy, Mark A. Horowitz
  • Publication number: 20130300910
    Abstract: An apparatus, method and computer program product for presenting burst images are provided. The apparatus may include a processor that may be configured to receive a plurality of burst images. Each burst image may differ from the other burst images based on a variable parameter. The processor may also be configured to provide for a presentation of a sample burst image. In this regard, the sample burst image may be one of the plurality of burst images. The processor may be further configured to receive a selected location within the presentation of the sample burst image and provide for a presentation of a plurality of burst image fragments associated with each of the plurality of burst images. In this regard, the burst image fragments may be portions of each of the burst images, where the areas of each burst image may be determined based on the selected location.
    Type: Application
    Filed: July 17, 2013
    Publication date: November 14, 2013
    Inventors: Marc Levoy, Natasha Gelfand, Wei-Chao Chen, Kari Antero Pulli
  • Patent number: 8497920
    Abstract: An apparatus for presenting burst images is provided. The apparatus may include a processor that may be configured to receive a plurality of burst images. Each burst image may differ from the other burst images based on a variable parameter, such as, but not limited to, exposure, focus, and/or time, or to the state of a target, such as the varying facial expression of a person. The processor may also be configured to provide for a presentation of a sample burst image. In this regard, the sample burst image may be one of the plurality of burst images. The processor may be further configured to receive a selected location within the presentation of the sample burst image and provide for a presentation of a plurality of burst image fragments associated with each of the plurality of burst images. In this regard, the burst image fragments may be portions of each of the burst images, where the areas of each burst image may be determined based on the selected location.
    Type: Grant
    Filed: June 11, 2008
    Date of Patent: July 30, 2013
    Assignee: Nokia Corporation
    Inventors: Marc Levoy, Natasha Gelfand, Wei-Chao Chen, Kari Antero Pulli
  • Patent number: 8411962
    Abstract: A computing device may select a source tile from a source image. From the source tile, the computing device may select a first rectangular feature and a second rectangular feature. Based on the first and second rectangular features, the computing device may calculate a source feature vector. The computing device may also select a search area of a target image, and a target tile within the within the search area. Based on the target tile, the computing device may calculate a target feature vector. The computing device may determine that a difference between the source feature vector and the target feature vector is below an error threshold, and based on this determination, further determine a mapping between the source image and the target image. The computing device may then apply the mapping to the source image to produce a transformed source image.
    Type: Grant
    Filed: April 27, 2012
    Date of Patent: April 2, 2013
    Assignee: Google Inc.
    Inventors: Ryan Geiss, Marc Levoy
  • Patent number: D985011
    Type: Grant
    Filed: October 10, 2019
    Date of Patent: May 2, 2023
    Assignee: GOOGLE LLC
    Inventors: Michelle Chen, Ryan Geiss, Marc Levoy, Kelly Tsai, Chorong Johnston, Alexander Schiffhauer, Samuel Hasinoff