Patents by Inventor Anita Nariani Schulze
Anita Nariani Schulze has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11562470Abstract: Devices, methods, and computer-readable media are disclosed describing an adaptive approach for image bracket selection and fusion, e.g., to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV? images, EV+ images, long exposure (or synthetic long exposure) images, EV0/EV? image pairs, etc., which are received according to a particular pattern. When a capture request is received, a set of rules and/or a decision tree may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. A noise reduction process may optionally be performed on the selected images before (or after) the registration and fusion operations.Type: GrantFiled: April 7, 2021Date of Patent: January 24, 2023Assignee: Apple IncInventors: Marius Tico, Alok Deshpande, Anita Nariani Schulze, Hao Sun, Jun Hu, Justin Michael Faryabi-Allen, Morten Poulsen, Tak Shing Wong, Wu Cheng
-
Publication number: 20210256669Abstract: Devices, methods, and computer-readable media are disclosed describing an adaptive approach for image bracket selection and fusion, e.g., to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV? images, EV+ images, long exposure (or synthetic long exposure) images, EV0/EV? image pairs, etc., which are received according to a particular pattern. When a capture request is received, a set of rules and/or a decision tree may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. A noise reduction process may optionally be performed on the selected images before (or after) the registration and fusion operations.Type: ApplicationFiled: April 7, 2021Publication date: August 19, 2021Inventors: Marius Tico, Alok Deshpande, Anita Nariani Schulze, Hao Sun, Jun Hu, Justin Michael Faryabi-Allen, Morten Poulsen, Tak Shing Wong, Wu Cheng
-
Patent number: 10977778Abstract: Devices, methods, and computer-readable media are disclosed describing an adaptive approach for image bracket selection and fusion, e.g., to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV? images, EV+ images, long exposure (or synthetic long exposure) images, EV0/EV? image pairs, etc., which are received according to a particular pattern. When a capture request is received, a set of rules and/or a decision tree may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. A noise reduction process may optionally be performed on the selected images before (or after) the registration and fusion operations.Type: GrantFiled: May 30, 2019Date of Patent: April 13, 2021Assignee: Apple Inc.Inventors: Marius Tico, Alok Deshpande, Anita Nariani Schulze, Hao Sun, Jun Hu, Justin Michael Faryabi-Allen, Morten Poulsen, Tak Shing Wong, Wu Cheng
-
Publication number: 20190370948Abstract: Devices, methods, and computer-readable media are disclosed describing an adaptive approach for image bracket selection and fusion, e.g., to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions. An incoming image stream may be obtained from an image capture device, wherein the incoming image stream comprises a variety of differently-exposed captures, e.g., EV0 images, EV? images, EV+ images, long exposure (or synthetic long exposure) images, EV0/EV? image pairs, etc., which are received according to a particular pattern. When a capture request is received, a set of rules and/or a decision tree may be used to evaluate one or more capture conditions associated with the images from the incoming image stream and determine which two or more images to select for a fusion operation. A noise reduction process may optionally be performed on the selected images before (or after) the registration and fusion operations.Type: ApplicationFiled: May 30, 2019Publication date: December 5, 2019Inventors: Marius Tico, Alok Deshpande, Anita Nariani Schulze, Hao Sun, Jun Hu, Justin Michael Faryabi-Allen, Morten Poulsen, Tak Shing Wong, Wu Cheng
-
Patent number: 9686470Abstract: Techniques to detect subject and camera motion in a set of consecutively captured image frames are disclosed. More particularly, techniques disclosed herein temporally track two sets of downscaled images to detect motion. One set may contain higher resolution and the other set lower resolution of the same images. For each set, a coefficient of variation may be computed across the set of images for each sample in the downscaled image to detect motion and generate a change mask. The information in the change mask can be used for various applications, including determining how to capture a next image in the sequence.Type: GrantFiled: September 30, 2014Date of Patent: June 20, 2017Assignee: Apple Inc.Inventors: Anita Nariani-Schulze, Benjamin M. Olson, Ralph Brunner, Suk Hwan Lim, Frank Doepke
-
Patent number: 9491360Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.Type: GrantFiled: June 6, 2013Date of Patent: November 8, 2016Assignee: Apple Inc.Inventors: Anita Nariani Schulze, Rolf Toft, Paul M. Hubel, Marius Tico, Jianping Zhou, Ralph Brunner, Claus Molgaard
-
Patent number: 9384552Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel approach to pixel-based registration of non-reference images to a reference frame in a set of commonly captured images is disclosed which makes use of pyramid decomposition to more efficiently detect corners. The disclosed pixel-based registration operation may also be combined with motion sensor data-based registration approaches to register non-reference images with respect to the reference frame. When the registered non-reference images are combined with the pre-selected reference image, the resulting image is a quality stabilized image.Type: GrantFiled: June 6, 2013Date of Patent: July 5, 2016Assignee: Apple Inc.Inventors: Jianping Zhou, Paul Hubel, Marius Tico, Anita Nariani Schulze, Rolf Toft
-
Patent number: 9350916Abstract: Systems, methods, and computer readable media to improve image stabilization operations and other image processing operations are described. A novel combination of interleaved image capture and image processing operations, e.g., image registration operations, may be employed on a bracketed capture of still images. Such techniques may result in improved camera performance and processing efficiency, as well as decreased shot-to-shot time intervals. In another embodiment, an image fusion portion of an image post-processing pipeline may also be performed in an interleaved fashion, such that each image in the sequence of obtained bracketed images may be incrementally added to an output composite image after it has been aligned with the preceding image or images from the sequence.Type: GrantFiled: May 28, 2013Date of Patent: May 24, 2016Assignee: Apple Inc.Inventors: Russell Pflughaupt, Rolf Toft, Paul Hubel, Anita Nariani Schulze
-
Patent number: 9262684Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. Novel approaches for fusing non-reference images with a pre-selected reference frame in a set of commonly captured images are disclosed. The fusing approach may use a soft transition by using a weighted average for ghost/non-ghost pixels to avoid sudden transition between neighborhood and almost similar pixels. Additionally, the ghost/non-ghost decision can be made based on a set of neighboring pixels rather than independently for each pixel. An alternative approach may involve performing a multi-resolution decomposition of all the captured images, using temporal fusion, spatio-temporal fusion, or combinations thereof, at each level and combining the different levels to generate an output image.Type: GrantFiled: June 6, 2013Date of Patent: February 16, 2016Assignee: Apple Inc.Inventors: Marius Tico, Jianping Zhou, Anita Nariani Schulze, Rolf Toft, Paul Hubel, Wei Sun
-
Patent number: 9215355Abstract: Calculating a temporal filter values to filter video data for noise and ghosting artifacts. A sensor specific noise model may be derived using the video capture settings of the video source to estimate the noise variance for an image, and to determine the amount of the temporal filtering to be applied to a frame or pixel. The global motion and sensor specific noise model may be used to populate filter coefficient look-up tables based on the local motion and luma values of each pixel in the frame. The maximum absolute difference between frames may be used to estimate global motion between two frames such that filter strength may be reduced for frames exhibiting high global motion and increased for frames exhibiting low global motion. According to an embodiment, the increase in temporal filtering between frames may be capped to avoid sudden visible noise differences between frames.Type: GrantFiled: September 14, 2012Date of Patent: December 15, 2015Assignee: Apple Inc.Inventors: Ting Chen, Yongfang Liang, Anita Nariani Schulze
-
Publication number: 20150350547Abstract: Techniques to detect subject and camera motion in a set of consecutively captured image frames are disclosed. More particularly, techniques disclosed herein temporally track two sets of downscaled images to detect motion. One set may contain higher resolution and the other set lower resolution of the same images. For each set, a coefficient of variation may be computed across the set of images for each sample in the downscaled image to detect motion and generate a change mask. The information in the change mask can be used for various applications, including determining how to capture a next image in the sequence.Type: ApplicationFiled: September 30, 2014Publication date: December 3, 2015Inventors: Anita Nariani-Schulze, Benjamin M. Olson, Ralph Brunner, Suk Hwan Lim, Frank Doepke
-
Publication number: 20140363087Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. Novel approaches for fusing non-reference images with a pre-selected reference frame in a set of commonly captured images are disclosed. The fusing approach may use a soft transition by using a weighted average for ghost/non-ghost pixels to avoid sudden transition between neighborhood and almost similar pixels. Additionally, the ghost/non-ghost decision can be made based on a set of neighboring pixels rather than independently for each pixel. An alternative approach may involve performing a multi-resolution decomposition of all the captured images, using temporal fusion, spatio-temporal fusion, or combinations thereof, at each level and combining the different levels to generate an output image.Type: ApplicationFiled: June 6, 2013Publication date: December 11, 2014Inventors: Marius Tico, Jianping Zhou, Anita Nariani Schulze, Rolf Toft, Paul Hubel, Wei Sun
-
Publication number: 20140363096Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel approach to pixel-based registration of non-reference images to a reference frame in a set of commonly captured images is disclosed which makes use of pyramid decomposition to more efficiently detect corners. The disclosed pixel-based registration operation may also be combined with motion sensor data-based registration approaches to register non-reference images with respect to the reference frame. When the registered non-reference images are combined with the pre-selected reference image, the resulting image is a quality stabilized image.Type: ApplicationFiled: June 6, 2013Publication date: December 11, 2014Inventors: Jianping Zhou, Paul Hubel, Marius Tico, Anita Nariani Schulze, Rolf Toft
-
Publication number: 20140362256Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.Type: ApplicationFiled: June 6, 2013Publication date: December 11, 2014Inventors: Anita Nariani Schulze, Rolf Toft, Paul M. Hubel, Marius Tico, Jianping Zhou, Ralph Brunner, Claus Molgaard
-
Publication number: 20140354842Abstract: Systems, methods, and computer readable media to improve image stabilization operations and other image processing operations are described. A novel combination of interleaved image capture and image processing operations, e.g., image registration operations, may be employed on a bracketed capture of still images. Such techniques may result in improved camera performance and processing efficiency, as well as decreased shot-to-shot time intervals. In another embodiment, an image fusion portion of an image post-processing pipeline may also be performed in an interleaved fashion, such that each image in the sequence of obtained bracketed images may be incrementally added to an output composite image after it has been aligned with the preceding image or images from the sequence.Type: ApplicationFiled: May 28, 2013Publication date: December 4, 2014Inventors: Russell Pflughaupt, Rolf Toft, Paul Hubel, Anita Nariani Schulze
-
Publication number: 20130083246Abstract: Calculating a temporal filter values to filter video data for noise and ghosting artifacts. A sensor specific noise model may be derived using the video capture settings of the video source to estimate the noise variance for an image, and to determine the amount of the temporal filtering to be applied to a frame or pixel. The global motion and sensor specific noise model may be used to populate filter coefficient look-up tables based on the local motion and luma values of each pixel in the frame. The maximum absolute difference between frames may be used to estimate global motion between two frames such that filter strength may be reduced for frames exhibiting high global motion and increased for frames exhibiting low global motion. According to an embodiment, the increase in temporal filtering between frames may be capped to avoid sudden visible noise differences between frames.Type: ApplicationFiled: September 14, 2012Publication date: April 4, 2013Applicant: APPLE INC.Inventors: Ting Chen, Yongfang Liang, Anita Nariani Schulze