Patents by Inventor Kurt Akeley

Kurt Akeley has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20190124318
    Abstract: A light-field camera may generate four-dimensional light-field data indicative of incoming light. The light-field camera may have an aperture configured to receive the incoming light, an image sensor, and a microlens array configured to redirect the incoming light at the image sensor. The image sensor may receive the incoming light and, based on the incoming light, generate the four-dimensional light-field data, which may have first and second spatial dimensions and first and second angular dimensions. The first angular dimension may have a first resolution higher than a second resolution of the second angular dimension.
    Type: Application
    Filed: July 11, 2018
    Publication date: April 25, 2019
    Inventors: Colvin PITTS, Chia-Kai LIANG, Kurt AKELEY
  • Patent number: 10154197
    Abstract: An image capture device, such as a camera, has multiple modes including a light field image capture mode, a conventional 2D image capture mode, and at least one intermediate image capture mode. By changing position and/or properties of the microlens array (MLA) in front of the image sensor, changes in 2D spatial resolution and angular resolution can be attained. In at least one embodiment, such changes can be performed in a continuous manner, allowing a continuum of intermediate modes to be attained.
    Type: Grant
    Filed: July 6, 2016
    Date of Patent: December 11, 2018
    Assignee: GOOGLE LLC
    Inventors: Jerome Chandra Bhat, Brandon Elliott Merle Clarke, Graham Butler Myhre, Ravi Kiran Nalla, Steven David Oliver, Tony Yip Pang Poon, William D. Houck, II, Colvin Pitts, Yi-Ren Ng, Kurt Akeley
  • Publication number: 20180329602
    Abstract: Video data of an environment may be prepared for presentation to a user in a virtual reality or augmented reality experience. According to one method, a plurality of locations distributed throughout a viewing volume may be designated, at which a plurality of vantages are to be positioned to facilitate viewing of the environment from proximate the locations. For each location, a plurality of images of the environment, captured from viewpoints proximate the location, may be retrieved. For each location, the images may be reprojected to a three-dimensional shape and combined to generate a combined image. The combined image may be applied to one or more surfaces of the three-dimensional shape to generate a vantage. The vantages may be stored such that the vantages can be used to generate viewpoint video of the scene, as viewed from a virtual viewpoint corresponding to an actual viewer's viewpoint within the viewing volume.
    Type: Application
    Filed: May 9, 2017
    Publication date: November 15, 2018
    Inventor: Kurt Akeley
  • Publication number: 20180329485
    Abstract: A virtual reality or augmented reality experience may be presented for a viewer through the use of input including only three degrees of freedom. The input may include orientation data indicative of a viewer orientation at which a head of the viewer is oriented. The viewer orientation may be mapped to an estimated viewer location. Viewpoint video may be generated of a scene as viewed from a virtual viewpoint with a virtual location corresponding to the estimated viewer location, from along the viewer orientation. The viewpoint video may be displayed for the viewer. In some embodiments, mapping may be carried out by defining a ray at the viewer orientation, locating an intersection of the ray with a three-dimensional shape, and, based on a location of the intersection, generating the estimated viewer location. The shape may be generated via calibration with a device that receives input including six degrees of freedom.
    Type: Application
    Filed: February 15, 2018
    Publication date: November 15, 2018
    Inventors: Trevor Carothers, Kurt Akeley
  • Patent number: 10038909
    Abstract: RAW images and/or light field images may be compressed through the use of specialized techniques. The color depth of a light field image may be reduced through the use of a bit reduction algorithm such as a K-means algorithm. The image may then be retiled to group pixels of similar intensities and/or colors. The retiled image may be padded with extra pixel rows and/or pixel columns as needed, and compressed through the use of an image compression algorithm. The compressed image may be assembled with metadata pertinent to the manner in which compression was done to form a compressed image file. The compressed image file may be decompressed by following the compression method in reverse.
    Type: Grant
    Filed: July 6, 2016
    Date of Patent: July 31, 2018
    Assignee: Google LLC
    Inventors: Kurt Akeley, Brendan Bevensee, Colvin Pitts, Timothy James Knight, Carl Warren Craddock, Chia-Kai Liang
  • Patent number: 10033986
    Abstract: A light-field camera may generate four-dimensional light-field data indicative of incoming light. The light-field camera may have an aperture configured to receive the incoming light, an image sensor, and a microlens array configured to redirect the incoming light at the image sensor. The image sensor may receive the incoming light and, based on the incoming light, generate the four-dimensional light-field data, which may have first and second spatial dimensions and first and second angular dimensions. The first angular dimension may have a first resolution higher than a second resolution of the second angular dimension.
    Type: Grant
    Filed: April 29, 2016
    Date of Patent: July 24, 2018
    Assignee: Google LLC
    Inventors: Colvin Pitts, Chia-Kai Liang, Kurt Akeley
  • Publication number: 20180097867
    Abstract: A video stream of a scene for a virtual reality or augmented reality experience may be captured by one or more image capture devices. Data from the video stream may be retrieved, including base vantage data with base vantage color data depicting the scene from a base vantage location, and target vantage data with target vantage color data depicting the scene from a target vantage location. The base vantage data may be reprojected to the target vantage location to obtain reprojected target vantage data. The reprojected target vantage data may be compared with the target vantage data to obtain residual data. The residual data may be compressed by removing a subset of the residual data that is likely to be less viewer-discernable than a remainder of the residual data. A compressed video stream may be stored, including the base vantage data and the compressed residual data.
    Type: Application
    Filed: December 5, 2017
    Publication date: April 5, 2018
    Inventors: Derek Pang, Colvin Pitts, Kurt Akeley
  • Publication number: 20180089903
    Abstract: A virtual reality or augmented reality experience of a scene may be presented to a viewer using layered data retrieval and/or processing. A first layer of a video stream may be retrieved, and a first viewer position and/or orientation may be received. The first layer may be processed to generate first viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation. The first viewpoint video may be displayed for the viewer. Then, a second layer of the video stream may be retrieved, and a second viewer position and/or orientation may be received. The second layer may be processed to generate second viewpoint video of the scene from a second virtual viewpoint corresponding to the second viewer position and/or orientation, with higher quality than the first viewpoint video. The second viewpoint video may be displayed for the viewer.
    Type: Application
    Filed: October 11, 2017
    Publication date: March 29, 2018
    Inventors: Derek Pang, Colvin Pitts, Kurt Akeley
  • Publication number: 20180082405
    Abstract: An image such as a light-field image may be processed to provide depth-based blurring. The image may be received in a data store. At an input device, first and second user input may be received to designate a first focus depth and a second focus depth different from the first focus depth, respectively. A processor may identify one or more foreground portions of the image that have one or more foreground portion depths, each of which is less than the first focus depth. The processor may also identify one or more background portions of the image that have one or more background portion depths, each of which is greater than the second focus depth. The processor may also apply blurring to the one or more foreground portions and the one or more background portions to generate a processed image, which may be displayed on a display device.
    Type: Application
    Filed: November 28, 2017
    Publication date: March 22, 2018
    Inventors: Chia-Kai Liang, Kent Oberheu, Kurt Akeley, Garrett Girod, Nikhil Karnad, Francis A. Benevides, JR.
  • Publication number: 20180070067
    Abstract: According to various embodiments of the present invention, the optical systems of light field capture devices are optimized so as to improve captured light field image data. Optimizing optical systems of light field capture devices can result in captured light field image data (both still and video) that is cheaper and/or easier to process. Optical systems can be optimized to yield improved quality or resolution when using cheaper processing approaches whose computational costs fit within various processing and/or resource constraints. As such, the optical systems of light field cameras can be optimized to reduce size and/or cost and/or increase the quality of such optical systems.
    Type: Application
    Filed: November 13, 2017
    Publication date: March 8, 2018
    Inventors: Timothy Knight, Colvin Pitts, Kurt Akeley, Yuriy Romanenko, Carl (Warren) Craddock
  • Publication number: 20180070066
    Abstract: According to various embodiments of the present invention, the optical systems of light field capture devices are optimized so as to improve captured light field image data. Optimizing optical systems of light field capture devices can result in captured light field image data (both still and video) that is cheaper and/or easier to process. Optical systems can be optimized to yield improved quality or resolution when using cheaper processing approaches whose computational costs fit within various processing and/or resource constraints. As such, the optical systems of light field cameras can be optimized to reduce size and/or cost and/or increase the quality of such optical systems.
    Type: Application
    Filed: November 13, 2017
    Publication date: March 8, 2018
    Inventors: Timothy Knight, Colvin Pitts, Kurt Akeley, Yuriy Romanenko, Carl (Warren) Craddock
  • Publication number: 20180033209
    Abstract: Video data of an environment may be prepared for stereoscopic presentation to a user in a virtual reality or augmented reality experience. According to one method, a plurality of locations distributed throughout a viewing volume may be designated, at which a plurality of vantages are to be positioned to facilitate viewing of the environment from proximate the locations. For each location, a plurality of images of the environment, captured from viewpoints proximate the location, may be retrieved. For each location, the images may be reprojected to a three-dimensional shape and combined to generate a combined image. The combined image may be applied to one or more surfaces of the three-dimensional shape to generate a vantage. The vantages may be stored such that the vantages can be used to generate stereoscopic viewpoint video of the scene, as viewed from at least two virtual viewpoints corresponding to viewpoints of an actual viewer's eyes within the viewing volume.
    Type: Application
    Filed: October 11, 2017
    Publication date: February 1, 2018
    Inventor: Kurt Akeley
  • Publication number: 20180035134
    Abstract: A virtual reality or augmented reality experience of a scene may be decoded for playback for a viewer through a combination of CPU and GPU processing. A video stream may be retrieved from a data store. A first viewer position and/or orientation may be received from an input device, such as the sensor package on a head-mounted display (HMD). At a processor, the video stream may be partially decoded to generate a partially-decoded bitstream. At a graphics processor, the partially-decoded bitstream may be further decoded to generate viewpoint video of the scene from a first virtual viewpoint corresponding to the first viewer position and/or orientation. The viewpoint video may be displayed on a display device, such as screen of the HMD.
    Type: Application
    Filed: October 11, 2017
    Publication date: February 1, 2018
    Inventors: Derek Pang, Colvin Pitts, Kurt Akeley, Zeyar Htet
  • Publication number: 20180020204
    Abstract: A video stream for a scene for a virtual reality or augmented reality experience may be stored and delivered to a viewer. The video stream may be divided into a plurality of units based on time segmentation, viewpoint segmentation, and/or view orientation segmentation. Each of the units may be divided into a plurality of sub-units based on a different segmentation from the units, via time segmentation, viewpoint segmentation, and/or view orientation segmentation. At least a portion of the video stream may be stored in a file that includes a plurality of the units. Each unit may be a group of pictures that is a sequence of successive frames in time. Each sub-unit may be a vantage defining a viewpoint from which the scene is viewable. Each vantage may be further divided into tiles, each of which is part of the vantage, limited to one or more particular view orientations.
    Type: Application
    Filed: September 15, 2017
    Publication date: January 18, 2018
    Inventors: Derek Pang, Colvin Pitts, Kurt Akeley
  • Patent number: 9866810
    Abstract: According to various embodiments of the present invention, the optical systems of light field capture devices are optimized so as to improve captured light field image data. Optimizing optical systems of light field capture devices can result in captured light field image data (both still and video) that is cheaper and/or easier to process. Optical systems can be optimized to yield improved quality or resolution when using cheaper processing approaches whose computational costs fit within various processing and/or resource constraints. As such, the optical systems of light field cameras can be optimized to reduce size and/or cost and/or increase the quality of such optical systems.
    Type: Grant
    Filed: February 22, 2016
    Date of Patent: January 9, 2018
    Assignee: Lytro, Inc.
    Inventors: Timothy Knight, Colvin Pitts, Kurt Akeley, Yuriy Romanenko, Carl (Warren) Craddock
  • Patent number: 9858649
    Abstract: An image such as a light-field image may be processed to provide depth-based blurring. The image may be received in a data store. At an input device, first and second user input may be received to designate a first focus depth and a second focus depth different from the first focus depth, respectively. A processor may identify one or more foreground portions of the image that have one or more foreground portion depths, each of which is less than the first focus depth. The processor may also identify one or more background portions of the image that have one or more background portion depths, each of which is greater than the second focus depth. The processor may also apply blurring to the one or more foreground portions and the one or more background portions to generate a processed image, which may be displayed on a display device.
    Type: Grant
    Filed: September 30, 2015
    Date of Patent: January 2, 2018
    Assignee: Lytro, Inc.
    Inventors: Chia-Kai Liang, Kent Oberheu, Kurt Akeley, Garrett Girod, Nikhil Karnad, Francis A. Benevides, Jr.
  • Publication number: 20170332000
    Abstract: A high dynamic range light-field image may be captured through the use of a light-field imaging system. In a first sensor of the light-field imaging system, first image data may be captured at a first exposure level. In the first sensor or in a second sensor of the light-field imaging system, second imaging data may be captured at a second exposure level greater than the first exposure level. In a data store, the first image data and the second image data may be received. In a processor, the first image data and the second image data may be combined to generate a light-field image with high dynamic range.
    Type: Application
    Filed: May 10, 2016
    Publication date: November 16, 2017
    Inventors: Zejing Wang, Kurt Akeley, Colvin Pitts, Jon Karafin
  • Publication number: 20170244948
    Abstract: An environment may be displayed from a viewpoint. According to one method, volumetric video data may be acquired depicting the environment, for example, using a tiled camera array. A plurality of vantages may be distributed throughout a viewing volume from which the environment is to be viewed. The volumetric video data may be used to generate video data for each vantage, representing the view of the environment from that vantage. User input may be received designating a viewpoint within the viewing volume. From among the plurality of vantages, a subset nearest to the viewpoint may be identified. The video data from the subset may be retrieved and combined to generate viewpoint video data depicting the environment from the viewpoint. The viewpoint video data may be displayed for the viewer to display a view of the environment from the viewpoint selected by the user.
    Type: Application
    Filed: May 9, 2017
    Publication date: August 24, 2017
    Inventors: Derek Pang, Colvin Pitts, Kurt Akeley
  • Publication number: 20170091906
    Abstract: An image such as a light-field image may be processed to provide depth-based blurring. The image may be received in a data store. At an input device, first and second user input may be received to designate a first focus depth and a second focus depth different from the first focus depth, respectively. A processor may identify one or more foreground portions of the image that have one or more foreground portion depths, each of which is less than the first focus depth. The processor may also identify one or more background portions of the image that have one or more background portion depths, each of which is greater than the second focus depth. The processor may also apply blurring to the one or more foreground portions and the one or more background portions to generate a processed image, which may be displayed on a display device.
    Type: Application
    Filed: September 30, 2015
    Publication date: March 30, 2017
    Inventors: Chia-Kai Liang, Kent Oberheu, Kurt Akeley, Garrett Girod, Nikhil Karnad, Francis A. Benevides, JR.
  • Publication number: 20160353082
    Abstract: A light-field camera may generate four-dimensional light-field data indicative of incoming light. The light-field camera may have an aperture configured to receive the incoming light, an image sensor, and a microlens array configured to redirect the incoming light at the image sensor. The image sensor may receive the incoming light and, based on the incoming light, generate the four-dimensional light-field data, which may have first and second spatial dimensions and first and second angular dimensions. The first angular dimension may have a first resolution higher than a second resolution of the second angular dimension.
    Type: Application
    Filed: April 29, 2016
    Publication date: December 1, 2016
    Inventors: Colvin Pitts, Chia-Kai Liang, Kurt Akeley