Patents by Inventor Ruiwen Zhen

Ruiwen Zhen has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11727540
    Abstract: Various image sharpening techniques are disclosed. For example, a method for image sharpening includes obtaining, using at least one sensor of an electronic device, an image that includes visual content. The method also includes generating an edge map that indicates edges of the visual content within the image. The method further includes applying a high-pass signal and an adaptive gain based on the edge map to sharpen the image. The method also includes generating a bright halo mask and a dark halo mask based on the edge map, where the bright halo mask indicates an upper sharpening limit and the dark halo mask indicates a lower sharpening limit. In addition, the method includes modifying a level of sharpening at one or more of the edges within the sharpened image to provide halo artifact reduction based on the bright halo mask and the dark halo mask.
    Type: Grant
    Filed: December 31, 2020
    Date of Patent: August 15, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Zhen Tong, Yibo Xu, John W. Glotzbach, Hamid R. Sheikh, Ruiwen Zhen, Baekdu Choi
  • Patent number: 11615510
    Abstract: An electronic device includes at least one imaging sensor and at least one processor coupled to the at least one imaging sensor. The at least one imaging sensor is configured to capture a burst of image frames. The at least one processor is configured to generate a low-resolution image from the burst of image frames. The at least one processor is also configured to estimate a blur kernel based on the burst of image frames. The at least one processor is further configured to perform deconvolution on the low-resolution image using the blur kernel to generate a deconvolved image. In addition, the at least one processor is configured to generate a high-resolution image using super resolution (SR) on the deconvolved image.
    Type: Grant
    Filed: December 31, 2020
    Date of Patent: March 28, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Baekdu Choi, Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh
  • Patent number: 11430094
    Abstract: A method includes obtaining multiple images of a scene including a first image associated with a first exposure value and a second image associated with a second exposure value. Each image includes image data in each of multiple channels of a color space, and the first exposure value is greater than the second exposure value. The method also includes decomposing each channel of each image into a base layer and a detail layer. The method further includes generating for each image, a blending weight map for the base layer and a blending weight map for the detail layer. The method includes combining each base layer and each detail layer of each channel of each image based on the blending weight map for the base layer for the image and the blending weight map for the detail layer of the image to obtain an HDR image of the scene.
    Type: Grant
    Filed: November 24, 2020
    Date of Patent: August 30, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John Glotzbach, Hamid Sheikh
  • Patent number: 11418766
    Abstract: A method includes receiving a reference input frame and a non-reference input frame; separating a luma plane for a luma channel and a chroma plane for a chroma channel for each of the reference input frame and the non-reference input frame; generating a luma frame using the luma plane; generating a chroma frame using the chroma plane in a patch-based chroma blending; and combining the luma frame and the chroma frame into an output frame.
    Type: Grant
    Filed: December 17, 2019
    Date of Patent: August 16, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid Rahim Sheikh
  • Patent number: 11388348
    Abstract: A method includes obtaining, using at least one processor of an electronic device, a high dynamic range (HDR) input Bayer image. The method also includes generating, using the at least one processor of the electronic device, a plurality of synthesized images at different exposure levels based on the input Bayer image. The method further includes fusing, using the at least one processor of the electronic device, the synthesized images to generate a fused image. The method also includes generating, using the at least one processor of the electronic device, a gain map based on the fused image. In addition, the method includes applying, using the at least one processor of the electronic device, a gain based on the gain map to the input Bayer image.
    Type: Grant
    Filed: December 31, 2020
    Date of Patent: July 12, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Long N. Le, Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen
  • Publication number: 20220207658
    Abstract: Various image sharpening techniques are disclosed. For example, a method for image sharpening includes obtaining, using at least one sensor of an electronic device, an image that includes visual content. The method also includes generating an edge map that indicates edges of the visual content within the image. The method further includes applying a high-pass signal and an adaptive gain based on the edge map to sharpen the image. The method also includes generating a bright halo mask and a dark halo mask based on the edge map, where the bright halo mask indicates an upper sharpening limit and the dark halo mask indicates a lower sharpening limit. In addition, the method includes modifying a level of sharpening at one or more of the edges within the sharpened image to provide halo artifact reduction based on the bright halo mask and the dark halo mask.
    Type: Application
    Filed: December 31, 2020
    Publication date: June 30, 2022
    Inventors: Zhen Tong, Yibo Xu, John W. Glotzbach, Hamid R. Sheikh, Ruiwen Zhen, Baekdu Choi
  • Patent number: 11354781
    Abstract: A method includes obtaining, using at least one processor, an image of a scene including image data in each of multiple channels of a color space. The method also includes decomposing a channel of the image into a first base layer and a first detail layer. The method further includes generating a first enhanced detail layer by manipulating a characteristic of the first detail layer. The method also includes decomposing the first base layer into a second base layer and a second detail layer. The method further includes generating a second enhanced detail layer by manipulating a characteristic of the second detail layer. In addition, the method includes combining the first enhanced detail layer, the second enhanced detail layer, and the second base layer to obtain an enhanced image of the scene.
    Type: Grant
    Filed: November 24, 2020
    Date of Patent: June 7, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh
  • Publication number: 20220122223
    Abstract: An electronic device includes at least one imaging sensor and at least one processor coupled to the at least one imaging sensor. The at least one imaging sensor is configured to capture a burst of image frames. The at least one processor is configured to generate a low-resolution image from the burst of image frames. The at least one processor is also configured to estimate a blur kernel based on the burst of image frames. The at least one processor is further configured to perform deconvolution on the low-resolution image using the blur kernel to generate a deconvolved image. In addition, the at least one processor is configured to generate a high-resolution image using super resolution (SR) on the deconvolved image.
    Type: Application
    Filed: December 31, 2020
    Publication date: April 21, 2022
    Inventors: Baekdu Choi, Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh
  • Publication number: 20220020126
    Abstract: A method includes obtaining multiple images of a scene including a first image associated with a first exposure value and a second image associated with a second exposure value. Each image includes image data in each of multiple channels of a color space, and the first exposure value is greater than the second exposure value. The method also includes decomposing each channel of each image into a base layer and a detail layer. The method further includes generating for each image, a blending weight map for the base layer and a blending weight map for the detail layer. The method includes combining each base layer and each detail layer of each channel of each image based on the blending weight map for the base layer for the image and the blending weight map for the detail layer of the image to obtain an HDR image of the scene.
    Type: Application
    Filed: November 24, 2020
    Publication date: January 20, 2022
    Inventors: Ruiwen Zhen, John Glotzbach, Hamid Sheikh
  • Publication number: 20220020120
    Abstract: A method includes obtaining, using at least one processor, an image of a scene including image data in each of multiple channels of a color space. The method also includes decomposing a channel of the image into a first base layer and a first detail layer. The method further includes generating a first enhanced detail layer by manipulating a characteristic of the first detail layer. The method also includes decomposing the first base layer into a second base layer and a second detail layer. The method further includes generating a second enhanced detail layer by manipulating a characteristic of the second detail layer. In addition, the method includes combining the first enhanced detail layer, the second enhanced detail layer, and the second base layer to obtain an enhanced image of the scene.
    Type: Application
    Filed: November 24, 2020
    Publication date: January 20, 2022
    Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh
  • Publication number: 20220021802
    Abstract: A method includes obtaining, using at least one processor of an electronic device, a high dynamic range (HDR) input Bayer image. The method also includes generating, using the at least one processor of the electronic device, a plurality of synthesized images at different exposure levels based on the input Bayer image. The method further includes fusing, using the at least one processor of the electronic device, the synthesized images to generate a fused image. The method also includes generating, using the at least one processor of the electronic device, a gain map based on the fused image. In addition, the method includes applying, using the at least one processor of the electronic device, a gain based on the gain map to the input Bayer image.
    Type: Application
    Filed: December 31, 2020
    Publication date: January 20, 2022
    Inventors: Long N. Le, Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen
  • Patent number: 11200653
    Abstract: A method includes obtaining, using at least one sensor of an electronic device, multiple image frames of a scene. The multiple image frames include a first image frame and a second image frame captured using different exposures. The method also includes excluding, using at least one processor of the electronic device, pixels in the first and second image frames based on a coarse motion map. The method further includes generating, using the at least one processor, multiple local histogram match maps based on different portions of the first and second image frames. In addition, the method includes generating, using the at least one processor, an image of the scene using the local histogram match maps.
    Type: Grant
    Filed: December 9, 2019
    Date of Patent: December 14, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Zhen Tong, John W. Glotzbach, Ruiwen Zhen, Hamid R. Sheikh
  • Patent number: 11151731
    Abstract: A method includes receiving a reference image and a non-reference image; dividing the reference image into a plurality of tiles; determining, using an electronic device, a motion vector map using coarse-to-fine based motion vector estimation; and generating an output frame using the motion vector map with the reference image and the non-reference image.
    Type: Grant
    Filed: December 26, 2019
    Date of Patent: October 19, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh
  • Patent number: 11128809
    Abstract: An electronic device, method, and computer readable medium for compositing high dynamic range frames are provided. The electronic device includes a camera, and a processor coupled to the camera. The processor registers the plurality of multi-exposure frames with a hybrid of matched features to align non-reference frames with a reference frame; generates blending maps of the plurality of multi-exposure frames to reduce moving ghost artifacts and identify local areas that are well-exposed in the plurality of multi-exposure frames; and blends the plurality of multi-exposure frames weighted by the blending maps using a two-step weight-constrained exposure fusion technique into a high dynamic range (HDR) frame.
    Type: Grant
    Filed: February 15, 2019
    Date of Patent: September 21, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John Glotzbach, Ibrahim Pekkucuksen, Hamid R. Sheikh
  • Patent number: 11107205
    Abstract: A method includes obtaining multiple image frames of a scene using at least one camera of an electronic device. The method also includes using a convolutional neural network to generate blending maps associated with the image frames. The blending maps contain or are based on both a measure of motion in the image frames and a measure of how well exposed different portions of the image frames are. The method further includes generating a final image of the scene using at least some of the image frames and at least some of the blending maps. The final image of the scene may be generated by blending the at least some of the image frames using the at least some of the blending maps, and the final image of the scene may include image details that are lost in at least one of the image frames due to over-exposure or under-exposure.
    Type: Grant
    Filed: February 18, 2019
    Date of Patent: August 31, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Yuting Hu, Ruiwen Zhen, John W. Glotzbach, Ibrahim Pekkucuksen, Hamid R. Sheikh
  • Patent number: 11095829
    Abstract: A method includes obtaining multiple image frames of a scene using at least one sensor of an electronic device. The multiple image frames include a first image frame and a second image frame having a longer exposure than the first image frame. The method also includes generating a label map that identifies pixels in the multiple image frames that are to be used in an image. The method further includes generating the image of the scene using the pixels extracted from the image frames based on the label map. The label map may include multiple labels, and each label may be associated with at least one corresponding pixel and may include a discrete value that identifies one of the multiple image frames from which the at least one corresponding pixel is extracted.
    Type: Grant
    Filed: August 19, 2019
    Date of Patent: August 17, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen, Zhen Tong, Long N. Le
  • Patent number: 11064133
    Abstract: A method includes obtaining multiple image frames of a scene using at least one sensor of an electronic device. The multiple image frames include a first image frame and a second image frame having a longer exposure than the first image frame. The method also includes generating a label map that identifies pixels in the multiple image frames that are to be used in an image. The method further includes generating the image of the scene using the pixels extracted from the image frames based on the label map. The label map may include multiple labels, and each label may be associated with at least one corresponding pixel and may include a discrete value that identifies one of the multiple image frames from which the at least one corresponding pixel is extracted.
    Type: Grant
    Filed: August 19, 2019
    Date of Patent: July 13, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen, Zhen Tong, Long N. Le
  • Publication number: 20210185287
    Abstract: A method includes receiving a reference input frame and a non-reference input frame; separating a luma plane for a luma channel and a chroma plane for a chroma channel for each of the reference input frame and the non-reference input frame; generating a luma frame using the luma plane; generating a chroma frame using the chroma plane in a patch-based chroma blending; and combining the luma frame and the chroma frame into an output frame.
    Type: Application
    Filed: December 17, 2019
    Publication date: June 17, 2021
    Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid Rahim Sheikh
  • Patent number: 10972672
    Abstract: A method of generating an image from multiple cameras having different focal lengths is described. The method comprising receiving a wide image and a tele image; aligning the wide image and the tele image to overlap a common field of view; correcting for photometric differences between the wide image and the tele image; selecting a stitching seam for the wide image and the tele image; and joining the wide image and the tele image to generate a composite image, wherein a first portion of the composite image on one side of the stitching seam is from the wide image and a second portion of the composite image on the other side of the stitching seam is from the tele image. An electronic device for generating an image is also described.
    Type: Grant
    Filed: February 17, 2018
    Date of Patent: April 6, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Ruiwen Zhen, John W. Glotzbach, Raja Bala, Hamid Rahim Sheikh
  • Patent number: 10949959
    Abstract: A method of processing image data in a composite image is described. The method comprises analyzing a multi-level blending of images of the composite image; identifying sharp transition boundary areas of the composite image; applying less attenuation for higher levels of multi-level blending of the composite image in the sharp transition boundary areas; and applying more attenuation for lower levels of multi-level blending of the composite image in the sharp transition boundary areas.
    Type: Grant
    Filed: February 18, 2019
    Date of Patent: March 16, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: John W Glotzbach, Ruiwen Zhen, Ibrahim Pekkucuksen, Hamid Rahim Sheikh