Patents by Inventor Paul M Hubel

Paul M Hubel has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10402952
    Abstract: Systems, methods, and computer readable media to improve the operation of electronic display systems. Techniques for inverse tone mapping operations for selected standard dynamic range (SDR) images are described. The converted images may be presented on high dynamic range (HDR) displays so as to increase a user's viewing experience (through an expanded dynamic range) while preserving the artistic content of the displayed information. Techniques disclosed herein selectively transform SDR images to HDR images by determining if the SDR images were created from HDR images (e.g., through the fusion of multiple SDR images) and if their quality is such as to permit the conversion without introducing unwanted visual artifacts. The proposed techniques apply a sigmoidal inverse tone mapping function configured to provide a perceptual-based tone mapping. Values for the function's tuning parameters may be set based on what may be determined about the original HDR-to-SDR mapping operation.
    Type: Grant
    Filed: August 2, 2017
    Date of Patent: September 3, 2019
    Assignee: Apple Inc.
    Inventors: Teun R. Baar, Jiaying Wu, Nicolas P. Bonnier, Garrett M. Johnson, Paul M. Hubel
  • Publication number: 20190222766
    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
    Type: Application
    Filed: March 22, 2019
    Publication date: July 18, 2019
    Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
  • Patent number: 10298864
    Abstract: Foreign lighting effects, such as lens flare, are very common in natural images. In a two-camera-system, the two images may be fused together to generate one image of a better quality. However, there are frequently different foreign light patterns in the two images that form the image pair, e.g., due to the difference in lens design, sensor and position, etc. Directly fusing such pairs of images will result in non-photorealistic images, with composed foreign light patterns from both images from the image pair. This disclosure describes a general foreign light mitigation scheme to detect all kinds of foreign light region mismatches. The detected foreign light mismatch regions may be deemphasized or excluded in the fusion step, in order to create a fused image that keeps a natural-looking foreign light pattern that is close to what was seen by the user of an image capture device during an image capture preview mode.
    Type: Grant
    Filed: June 8, 2017
    Date of Patent: May 21, 2019
    Assignee: Apple Inc.
    Inventors: Xiaoxing Li, Paul M. Hubel, Marius Tico, Lech J. Szumilas
  • Patent number: 10277820
    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
    Type: Grant
    Filed: July 9, 2018
    Date of Patent: April 30, 2019
    Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
  • Publication number: 20180350047
    Abstract: Systems, methods, and computer readable media to improve the operation of electronic display systems are disclosed. Techniques for inverse tone mapping operations for selected standard dynamic range (SDR) images are described. The converted images may be presented on high dynamic range (HDR) displays so as to increase a user's viewing experience (through an expanded dynamic range) while preserving the artistic content of the displayed information. Techniques disclosed herein selectively transform SDR images to HDR images by determining if the SDR images were created from HDR images (e.g., through the fusion of multiple SDR images) and if their quality is such as to permit the conversion without introducing unwanted visual artifacts. The proposed techniques apply a sigmoidal inverse tone mapping function configured to provide a perceptual-based tone mapping. Values for the function's tuning parameters may be set based on what may be determined about the original HDR-to-SDR mapping operation.
    Type: Application
    Filed: August 2, 2017
    Publication date: December 6, 2018
    Inventors: Teun R. Baar, Jiaying Wu, Nicolas P. Bonnier, Garrett M. Johnson, Paul M. Hubel
  • Publication number: 20180316864
    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
    Type: Application
    Filed: July 9, 2018
    Publication date: November 1, 2018
    Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
  • Publication number: 20180278913
    Abstract: Techniques are disclosed for capturing stereoscopic images using one or more high color density or “full color” image sensors and one or more low color density or “sparse color” image sensors. Low color density image sensors, may include substantially fewer color pixels than the sensor's total number of pixels, as well as fewer color pixels than the total number of color pixels on the full color image sensor. More particularly, the mostly-monochrome image captured by the low color density image sensor may be used to reduce noise and increase the spatial resolution of an imaging system's output image. In addition, the color pixels present in the low color density image sensor may be used to identify and fill in color pixel values, e.g., for regions occluded in the image captured using the full color image sensor. Optical Image Stabilization and/or split photodiodes may be employed on one or more sensors.
    Type: Application
    Filed: March 26, 2018
    Publication date: September 27, 2018
    Inventors: Ziv Attar, Paul M. Hubel, Ilana Volfin, Sebastien Marineau-Mes
  • Patent number: 10033927
    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
    Type: Grant
    Filed: October 30, 2017
    Date of Patent: July 24, 2018
    Assignee: Apple Inc.
    Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
  • Publication number: 20180068473
    Abstract: Image fusion techniques hide artifacts that can arise at seams between regions of different image quality. According to these techniques, image registration may be performed on multiple images having at least a portion of image content in common. A first image may be warped to a spatial domain of a second image based on the image registration. A fused image may be generated from a blend of the warped first image and the second image, wherein relative contributions of the warped first image and the second image are weighted according to a distribution pattern based on a size of a smaller of the pair of images. In this manner, contributions of the different images vary at seams that otherwise would appear.
    Type: Application
    Filed: September 6, 2016
    Publication date: March 8, 2018
    Inventors: Marius Tico, Lech J. Szumilas, Xiaoxing Li, Paul M. Hubel, Todd S. Sachs
  • Publication number: 20180070007
    Abstract: Techniques are described for automated analysis and filtering of image data. Image data is analyzed to identify regions of interest (ROIs) within the image content. The image data also may have depth estimates applied to content therein. One or more of the ROIs may be designated to possess a base depth, representing a depth of image content against which depths of other content may be compared. Moreover, the depth of the image content within a spatial area of an ROI may be set to be a consistent value, regardless of depth estimates that may have been assigned from other sources. Thereafter, other elements of image content may be assigned content adjustment values in gradients based on their relative depth in image content as compared to the base depth and, optionally, based on their spatial distance from the designated ROI. Image content may be adjusted based on the content adjustment values.
    Type: Application
    Filed: September 6, 2017
    Publication date: March 8, 2018
    Inventors: Claus Molgaard, Paul M. Hubel, Ziv Attar, Ilana Volfin
  • Publication number: 20180070015
    Abstract: Techniques are disclosed for managing image capture and processing in a multi-camera imaging system. In such a system, a pair of cameras each may output a sequence of frames representing captured image data. The cameras' output may be synchronized to each other to cause synchronism in the image capture operations of the cameras. The system may assess image quality of frames output from the cameras and, based on the image quality, designate a pair of the frames to serve as a “reference frame pair.” Thus, one frame from the first camera and a paired frame from the second camera will be designated as the reference frame pair. The system may adjust each reference frame in the pair using other frames from their respective cameras. The reference frames also may be processed by other operations within the system, such as image fusion.
    Type: Application
    Filed: September 6, 2017
    Publication date: March 8, 2018
    Inventors: Paul M. Hubel, Marius Tico, Ting Chen
  • Publication number: 20180063441
    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
    Type: Application
    Filed: October 30, 2017
    Publication date: March 1, 2018
    Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
  • Patent number: 9860447
    Abstract: In some embodiments, a plurality of images is captured through an image sensor of a camera during a period of time. The plurality of images is registered to calculate an image-sensor-based measure of movement of the camera during the period of time. The image-sensor-based measure of movement is compared to a motion-sensor-based measure of movement of the camera during the period of time, and a systematic offset between the motion-sensor-based measure of movement and the image-sensor-based measure of movement is calculated.
    Type: Grant
    Filed: September 29, 2014
    Date of Patent: January 2, 2018
    Assignee: Apple Inc.
    Inventors: David C. Beard, Paul M. Hubel
  • Patent number: 9860446
    Abstract: Lens flare mitigation techniques determine which pixels in images of a sequence of images are likely to be pixels affected by lens flare. Once the lens flare areas of the images are determined, unwanted lens flare effects may be mitigated by various approaches, including reducing border artifacts along a seam between successive images, discarding entire images of the sequence that contain lens flare areas, and using tone-mapping to reduce the visibility of lens flare.
    Type: Grant
    Filed: February 24, 2017
    Date of Patent: January 2, 2018
    Assignee: Apple Inc.
    Inventors: Marius Tico, Paul M. Hubel, Frank Doepke, Todd S. Sachs
  • Publication number: 20170359534
    Abstract: Foreign lighting effects, such as lens flare, are very common in natural images. In a two-camera-system, the two images may be fused together to generate one image of a better quality. However, there are frequently different foreign light patterns in the two images that form the image pair, e.g., due to the difference in lens design, sensor and position, etc. Directly fusing such pairs of images will result in non-photorealistic images, with composed foreign light patterns from both images from the image pair. This disclosure describes a general foreign light mitigation scheme to detect all kinds of foreign light region mismatches. The detected foreign light mismatch regions may be deemphasized or excluded in the fusion step, in order to create a fused image that keeps a natural-looking foreign light pattern that is close to what was seen by the user of an image capture device during an image capture preview mode.
    Type: Application
    Filed: June 8, 2017
    Publication date: December 14, 2017
    Inventors: Xiaoxing Li, Paul M. Hubel, Marius Tico, Lech J. Szumilas
  • Publication number: 20170358101
    Abstract: Depth determination includes obtaining a first image of a scene captured by a camera at a first position, obtaining a second image of the scene captured by the camera at a second position directed by an optical image stabilization (OIS) actuator, determining a virtual baseline between the camera at the first position and the second position, and determining a depth of the scene based on the first image, the second image, and the virtual baseline.
    Type: Application
    Filed: June 9, 2017
    Publication date: December 14, 2017
    Inventors: Thomas E. Bishop, Benjamin A. Darling, Kevin A. Gross, Marius Tico, Paul M. Hubel, Todd S. Sachs
  • Patent number: 9843730
    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
    Type: Grant
    Filed: May 5, 2017
    Date of Patent: December 12, 2017
    Assignee: Apple Inc.
    Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
  • Publication number: 20170237905
    Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.
    Type: Application
    Filed: May 5, 2017
    Publication date: August 17, 2017
    Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
  • Patent number: 9692995
    Abstract: Lens flare mitigation techniques determine which pixels in images of a sequence of images are likely to be pixels affected by lens flare. Once the lens flare areas of the images are determined, unwanted lens flare effects may be mitigated by various approaches, including reducing border artifacts along a seam between successive images, discarding entire images of the sequence that contain lens flare areas, and using tone-mapping to reduce the visibility of lens flare.
    Type: Grant
    Filed: February 1, 2016
    Date of Patent: June 27, 2017
    Assignee: Apple Inc.
    Inventors: Marius Tico, Paul M. Hubel, Frank Doepke, Todd S. Sachs
  • Patent number: 9686539
    Abstract: Systems, methods, and computer readable media for calibrating two cameras (image capture units) using a non-standard, and initially unknown, calibration object are described. More particularly, an iterative approach to determine the structure and pose of an target object in an unconstrained environment are disclosed. The target object may be any of a number of predetermined objects such as a specific three dimensional (3D) shape, a specific type of animal (e.g., dogs), or the face of an arbitrary human. Virtually any object whose structure may be expressed in terms of a relatively low dimensional parametrized model may be used as a target object. The identified object (i.e., its pose and shape) may be used as input to a bundle adjustment operation resulting in camera calibration.
    Type: Grant
    Filed: June 12, 2016
    Date of Patent: June 20, 2017
    Assignee: Apple Inc.
    Inventors: Marco Zuliani, Claus Molgaard, Paul M. Hubel