Patents by Inventor Long N. Le
Long N. Le has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230034109Abstract: A method includes obtaining a blended red-green-blue (RGB) image frame of a scene. The method also includes performing, using at least one processing device of an electronic device, an interband denoising operation to remove at least one of noise and one or more artifacts from the blended RGB image frame in order to produce a denoised RGB image frame. Performing the interband denoising operation includes performing filtering of red, green, and blue color channels of the blended RGB image frame to remove at least one of the noise and the one or more artifacts from the blended RGB image frame. The filtering of the red and blue color channels of the blended RGB image frame is based on image data of at least one of the green color channel and a white color channel of the blended RGB image frame.Type: ApplicationFiled: January 27, 2022Publication date: February 2, 2023Inventors: Long N. Le, Yibo Xu, John W. Glotzbach, Zhen Tong, Hamid R. Sheikh
-
Publication number: 20230035482Abstract: A method includes obtaining multiple input image frames and determining how to warp at least one of the input image frames. The method also includes performing an intraband demosaic-warp operation to reconstruct image data in different color channels of the input image frames and warp the at least one input image frame to produce RGB input image frames. The method further includes blending the RGB input image frames to produce a blended RGB image frame, performing an interband denoising operation to produce a denoised RGB image frame, and performing an interband sharpening operation to produce a sharpened RGB image frame. In addition, the method includes performing an interband demosaic operation to substantially equalize high-frequency content in different color channels of the sharpened RGB image frame to produce an equalized sharpened RGB image frame and generating a final image of the scene based on the equalized sharpened RGB image frame.Type: ApplicationFiled: January 27, 2022Publication date: February 2, 2023Inventors: John W. Glotzbach, Long N. Le, Zhen Tong, Hamid R. Sheikh, Yibo Xu
-
Publication number: 20230021726Abstract: A method includes obtaining multiple images of a scene using at least one red-green-blue-white (RGBW) image sensor. The method also includes generating multi-channel frames at different exposure levels from the images. The method further includes estimating motion across exposure differences between the different exposure levels using a white channel of the multi-channel frames as a guidance signal to generate multiple motion maps. The method also includes estimating saturation across the exposure differences between the different exposure levels to generate multiple saturation maps. The method further includes using the generated motion maps and saturation maps to recover saturations from the different exposure levels and generate a saturation-free RGBW frame. In addition, the method includes processing the saturation-free RGBW frame to generate a final image of the scene.Type: ApplicationFiled: January 28, 2022Publication date: January 26, 2023Inventors: Long N. Le, Hamid R. Sheikh, John W. Glotzbach
-
Patent number: 11388348Abstract: A method includes obtaining, using at least one processor of an electronic device, a high dynamic range (HDR) input Bayer image. The method also includes generating, using the at least one processor of the electronic device, a plurality of synthesized images at different exposure levels based on the input Bayer image. The method further includes fusing, using the at least one processor of the electronic device, the synthesized images to generate a fused image. The method also includes generating, using the at least one processor of the electronic device, a gain map based on the fused image. In addition, the method includes applying, using the at least one processor of the electronic device, a gain based on the gain map to the input Bayer image.Type: GrantFiled: December 31, 2020Date of Patent: July 12, 2022Assignee: Samsung Electronics Co., Ltd.Inventors: Long N. Le, Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen
-
Publication number: 20220198722Abstract: A method includes capturing a plurality of first images using a first image sensor and capturing a plurality of second images using a second image sensor. The method also includes estimating depth information based on at least one of the first images and at least one of the second images. The method further includes obtaining information related to a lighting direction from an artificial intelligence (AI) light director, where the AI light director is trained to determine one or more lighting directions. In addition, the method includes generating at least one relit image using at least one of the first and second images based on the obtained information related to the lighting direction and the estimated depth information.Type: ApplicationFiled: December 30, 2021Publication date: June 23, 2022Inventors: Deboleena Roy, Long N. Le, Hamid R. Sheikh
-
Patent number: 11276154Abstract: A method includes capturing a plurality of first images using a first image sensor and a plurality of second images using a second image sensor. The method also includes generating a short single view, short depth map, long single view, and long depth map from the first and second images. The method further includes generating a multi-frame depth map using the short single view, short depth map, long single view, and long depth map. The method also includes generating a gain map using the multi-frame depth map and lighting conditions, where the lighting conditions introduce a relighting effect into the gain map. The method further includes generating a relit short single view and a relit long single view by applying the gain map to the short and long single views. In addition, the method includes generating a relit image by fusing the relit short and long single views.Type: GrantFiled: July 17, 2020Date of Patent: March 15, 2022Assignee: Samsung Electronics Co., Ltd.Inventors: Deboleena Roy, Long N. Le, Hamid R. Sheikh
-
Publication number: 20220021802Abstract: A method includes obtaining, using at least one processor of an electronic device, a high dynamic range (HDR) input Bayer image. The method also includes generating, using the at least one processor of the electronic device, a plurality of synthesized images at different exposure levels based on the input Bayer image. The method further includes fusing, using the at least one processor of the electronic device, the synthesized images to generate a fused image. The method also includes generating, using the at least one processor of the electronic device, a gain map based on the fused image. In addition, the method includes applying, using the at least one processor of the electronic device, a gain based on the gain map to the input Bayer image.Type: ApplicationFiled: December 31, 2020Publication date: January 20, 2022Inventors: Long N. Le, Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen
-
Publication number: 20220020128Abstract: A method includes capturing a plurality of first images using a first image sensor and a plurality of second images using a second image sensor. The method also includes generating a short single view, short depth map, long single view, and long depth map from the first and second images. The method further includes generating a multi-frame depth map using the short single view, short depth map, long single view, and long depth map. The method also includes generating a gain map using the multi-frame depth map and lighting conditions, where the lighting conditions introduce a relighting effect into the gain map. The method further includes generating a relit short single view and a relit long single view by applying the gain map to the short and long single views. In addition, the method includes generating a relit image by fusing the relit short and long single views.Type: ApplicationFiled: July 17, 2020Publication date: January 20, 2022Inventors: Deboleena Roy, Long N. Le, Hamid R. Sheikh
-
Patent number: 11095829Abstract: A method includes obtaining multiple image frames of a scene using at least one sensor of an electronic device. The multiple image frames include a first image frame and a second image frame having a longer exposure than the first image frame. The method also includes generating a label map that identifies pixels in the multiple image frames that are to be used in an image. The method further includes generating the image of the scene using the pixels extracted from the image frames based on the label map. The label map may include multiple labels, and each label may be associated with at least one corresponding pixel and may include a discrete value that identifies one of the multiple image frames from which the at least one corresponding pixel is extracted.Type: GrantFiled: August 19, 2019Date of Patent: August 17, 2021Assignee: Samsung Electronics Co., Ltd.Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen, Zhen Tong, Long N. Le
-
Patent number: 11064133Abstract: A method includes obtaining multiple image frames of a scene using at least one sensor of an electronic device. The multiple image frames include a first image frame and a second image frame having a longer exposure than the first image frame. The method also includes generating a label map that identifies pixels in the multiple image frames that are to be used in an image. The method further includes generating the image of the scene using the pixels extracted from the image frames based on the label map. The label map may include multiple labels, and each label may be associated with at least one corresponding pixel and may include a discrete value that identifies one of the multiple image frames from which the at least one corresponding pixel is extracted.Type: GrantFiled: August 19, 2019Date of Patent: July 13, 2021Assignee: Samsung Electronics Co., Ltd.Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen, Zhen Tong, Long N. Le
-
Patent number: 10944914Abstract: A method includes obtaining, using at least one image sensor of an electronic device, a first image frame of a scene. The method also includes using a convolutional neural network to generate, from the first image frame, multiple second image frames simulated to have different exposures. One or more objects in the scene in each second image frame are aligned with one or more corresponding objects in the scene in at least one other second image frame and are aligned with one or more corresponding objects in the scene in the first image frame. The method further includes blending the multiple second image frames to generate a final image of the scene.Type: GrantFiled: December 2, 2019Date of Patent: March 9, 2021Assignee: Samsung Electronics Co., Ltd.Inventors: Long N. Le, Hamid R. Sheikh, Zeeshan Nadir, John W. Glotzbach
-
Patent number: 10911691Abstract: A method includes obtaining, using at least one image sensor of an electronic device, multiple image frames of a scene. The multiple image frames include a plurality of short image frames at a first exposure level and a plurality of long image frames at a second exposure level longer than the first exposure level. The method also includes generating a short reference image frame and a long reference image frame using the multiple image frames. The method further includes selecting, using a processor of the electronic device, the short reference image frame or the long reference image frame as a reference frame, where the selection is based on an amount of saturated motion in the long image frame and an amount of a shadow region in the short image frame. In addition, the method includes generating a final image of the scene using the reference frame.Type: GrantFiled: December 18, 2019Date of Patent: February 2, 2021Assignee: Samsung Electronics Co., Ltd.Inventors: Long N. Le, Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh
-
Publication number: 20200396370Abstract: A method includes obtaining multiple image frames of a scene using at least one sensor of an electronic device. The multiple image frames include a first image frame and a second image frame having a longer exposure than the first image frame. The method also includes generating a label map that identifies pixels in the multiple image frames that are to be used in an image. The method further includes generating the image of the scene using the pixels extracted from the image frames based on the label map. The label map may include multiple labels, and each label may be associated with at least one corresponding pixel and may include a discrete value that identifies one of the multiple image frames from which the at least one corresponding pixel is extracted.Type: ApplicationFiled: August 19, 2019Publication date: December 17, 2020Inventors: Ruiwen Zhen, John W. Glotzbach, Hamid R. Sheikh, Ibrahim Pekkucuksen, Zhen Tong, Long N. Le
-
Publication number: 20200267299Abstract: A method includes capturing multiple ambient images of a scene using at least one camera of an electronic device and without using a flash of the electronic device. The method also includes capturing multiple flash images of the scene using the at least one camera of the electronic device and during firing of a pilot flash sequence using the flash. The method further includes analyzing multiple pairs of images to estimate exposure differences obtained using the flash, where each pair of images includes one of the ambient images and one of the flash images that are both captured using a common camera exposure and where different pairs of images are captured using different camera exposures. In addition, the method includes determining a flash strength for the scene based on the estimate of the exposure differences and firing the flash based on the determined flash strength.Type: ApplicationFiled: February 18, 2019Publication date: August 20, 2020Inventors: Long N. Le, Hamid R. Sheikh, John W. Glotzbach
-
Patent number: 10742892Abstract: A method includes capturing multiple ambient images of a scene using at least one camera of an electronic device and without using a flash of the electronic device. The method also includes capturing multiple flash images of the scene using the at least one camera of the electronic device and during firing of a pilot flash sequence using the flash. The method further includes analyzing multiple pairs of images to estimate exposure differences obtained using the flash, where each pair of images includes one of the ambient images and one of the flash images that are both captured using a common camera exposure and where different pairs of images are captured using different camera exposures. In addition, the method includes determining a flash strength for the scene based on the estimate of the exposure differences and firing the flash based on the determined flash strength.Type: GrantFiled: February 18, 2019Date of Patent: August 11, 2020Assignee: Samsung Electronics Co., Ltd.Inventors: Long N. Le, Hamid R. Sheikh, John W. Glotzbach
-
Patent number: 10554890Abstract: A method includes capturing multiple pairs of images of a scene at different exposures using at least one camera of an electronic device. Each pair of images includes (i) an ambient image of the scene captured without using a flash of the electronic device and (ii) a flash image of the scene captured using the flash of the electronic device. The method also includes rendering a final image of the scene with a bokeh that is determined using the multiple pairs of images. One of the ambient images or the flash images are captured in order of increasing exposure time, and the other of the ambient images or the flash images are captured in order of decreasing exposure time. The method may also include estimating a depth map associated with the scene using the pairs of images, where the bokeh is based on the depth map.Type: GrantFiled: February 18, 2019Date of Patent: February 4, 2020Assignee: Samsung Electronics Co., Ltd.Inventors: Long N. Le, John W. Glotzbach, Hamid R. Sheikh, Michael O. Polley