Patents by Inventor Chenchi LUO

Chenchi LUO has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11847771
    Abstract: A method includes obtaining at least one image and a ground truth map associated with the at least one image. The method also includes generating multiple optical maps using multiple algorithms and the at least one image. The method further includes, for each algorithm, identifying at least one score for the algorithm using one or more of the optical maps generated using the algorithm and the ground truth map. The ground truth map identifies one or more boundaries associated with one or more foreground objects in the at least one image. The scores identify how well the optical maps generated using the algorithms separate the one or more foreground objects from a background in the at least one image.
    Type: Grant
    Filed: April 30, 2021
    Date of Patent: December 19, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Yingmao Li, Chenchi Luo, Youngjun Yoo
  • Patent number: 11823353
    Abstract: A method includes processing, using at least one processor of an electronic device, multiple reference images of a scene using a first convolutional neural network (CNN) to generate a confidence map and a disparity map. The method also includes generating, using the at least one processor, an initial Bokeh image based on the disparity map and the reference images using a depth-of-field (DoF) renderer. The method further includes refining, using the at least one processor, the initial Bokeh image using a second CNN to generate a refined Bokeh image, where the second CNN uses the confidence map, the disparity map, and the reference images to generate the refined Bokeh image.
    Type: Grant
    Filed: November 30, 2020
    Date of Patent: November 21, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Yingmao Li, Kaimo Lin, Youngjun Yoo
  • Patent number: 11816855
    Abstract: A method includes obtaining at least three input image frames of a scene captured using at least three imaging sensors. The input image frames include a reference image frame and multiple non-reference image frames. The method also includes generating multiple disparity maps using the input image frames. Each disparity map is associated with the reference image frame and a different non-reference image frame. The method further includes generating multiple confidence maps using the input image frames. Each confidence map identifies weights associated with one of the disparity maps. In addition, the method includes generating a depth map of the scene using the disparity maps and the confidence maps. The imaging sensors are arranged to define multiple baseline directions, where each baseline direction extends between the imaging sensor used to capture the reference image frame and the imaging sensor used to capture a different non-reference image frame.
    Type: Grant
    Filed: September 21, 2020
    Date of Patent: November 14, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Yingmao Li, Kaimo Lin, Youngjun Yoo
  • Patent number: 11720782
    Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple calibration parameters associated with multiple sensors of a selected mobile device. The method also includes obtaining, using the at least one processor, an identification of multiple imaging tasks. The method further includes obtaining, using the at least one processor, multiple synthetically-generated scene images. In addition, the method includes generating, using the at least one processor, multiple training images and corresponding meta information based on the calibration parameters, the identification of the imaging tasks, and the scene images. The training images and corresponding meta information are generated concurrently, different ones of the training images correspond to different ones of the sensors, and different pieces of the meta information correspond to different ones of the imaging tasks.
    Type: Grant
    Filed: December 28, 2020
    Date of Patent: August 8, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Gyeongmin Choe, Yingmao Li, Zeeshan Nadir, Hamid R. Sheikh, John Seokjun Lee, Youngjun Yoo
  • Publication number: 20230245328
    Abstract: A method includes obtaining a first optical flow vector representing motion between consecutive video frames during a previous time step. The method also includes generating a first predicted optical flow vector from the first optical flow vector using a trained prediction model, where the first predicted optical flow vector represents predicted motion during a current time step. The method further includes refining the first predicted optical flow vector using a trained update model to generate a second optical flow vector representing motion during the current time step. The trained update model uses the first predicted optical flow vector, a video frame of the previous time step, and a video frame of the current time step to generate the second optical flow vector.
    Type: Application
    Filed: February 2, 2022
    Publication date: August 3, 2023
    Inventors: Yingmao Li, Chenchi Luo, Gyeongmin Choe, John Seokjun Lee
  • Patent number: 11631163
    Abstract: A method includes processing, using at least one processor of an electronic device, each of multiple images using a photometric augmentation engine, where the photometric augmentation engine performs one or more photometric augmentation operations. The method also includes applying, using the at least one processor, multiple layers of a convolutional neural network to each of the images, where each layer generates a corresponding feature map. The method further includes processing, using the at least one processor, at least one of the feature maps using at least one feature augmentation engine between consecutive layers of the multiple layers, where the at least one feature augmentation engine performs one or more feature augmentation operations.
    Type: Grant
    Filed: July 14, 2020
    Date of Patent: April 18, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Yingmao Li, Youngjun Yoo
  • Patent number: 11593637
    Abstract: A method, an electronic device, and computer readable medium are provided. The method includes receiving an input into a neural network that includes a kernel. The method also includes generating, during a convolution operation of the neural network, multiple panel matrices based on different portions of the input. The method additionally includes successively combining each of the multiple panel matrices with the kernel to generate an output. Generating the multiple panel matrices can include mapping elements within a moving window of the input onto columns of an indexing matrix, where a size of the window corresponds to the size of the kernel.
    Type: Grant
    Filed: April 30, 2019
    Date of Patent: February 28, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Yuming Zhu, Hyejung Kim, John Seokjun Lee, Manish Goel
  • Patent number: 11580399
    Abstract: An electronic device, method, and computer readable medium for 3D association of detected objects are provided. The electronic device includes a memory and at least one processor coupled to the memory. The at least one processor configured to convolve an input to a neural network with a basis kernel to generate a convolution result, scale the convolution result by a scalar to create a scaled convolution result, and combine the scaled convolution result with one or more of a plurality of scaled convolution results to generate an output feature map.
    Type: Grant
    Filed: April 30, 2019
    Date of Patent: February 14, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Manish Goel, David Liu
  • Patent number: 11556784
    Abstract: A method includes identifying, by at least one processor, multiple features of input data using a common feature extractor. The method also includes processing, by the at least one processor, at least some identified features using each of multiple pre-processing branches. Each pre-processing branch includes a first set of neural network layers and generates initial outputs associated with a different one of multiple data processing tasks. The method further includes combining, by the at least one processor, at least two initial outputs from at least two pre-processing branches to produce combined initial outputs. In addition, the method includes processing, by the at least one processor, at least some initial outputs or at least some combined initial outputs using each of multiple post-processing branches. Each post-processing branch includes a second set of neural network layers and generates final outputs associated with a different one of the multiple data processing tasks.
    Type: Grant
    Filed: November 22, 2019
    Date of Patent: January 17, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Yingmao Li, Youngjun Yoo
  • Patent number: 11532091
    Abstract: A method includes obtaining, using at least one processor, an input image frame. The method also includes identifying, using the at least one processor, one or more regions of the input image frame containing redundant information. In addition, the method includes performing, using the at least one processor, an image processing task using the input image frame. The image processing task is guided based on the one or more identified regions of the input image frame. The method may further include obtaining, using the at least one processor, a coarse depth map associated with the input image frame. Performing the image processing task may include refining the coarse depth map to produce a refined depth map, where the refining of the coarse depth map is guided based on the one or more identified regions of the input image frame.
    Type: Grant
    Filed: July 13, 2020
    Date of Patent: December 20, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Kushal Kardam Vyas, Yingmao Li, Chenchi Luo, George Q. Chen, Hamid R. Sheikh, Youngjun Yoo, Michael O. Polley
  • Patent number: 11503266
    Abstract: A method includes obtaining, using at least one processor, first and second input image frames, where the first and second input image frames are associated with first and second image planes, respectively. The method also includes obtaining, using the at least one processor, a depth map associated with the first input image frame. The method further includes producing another version of the depth map by performing one or more times: (a) projecting, using the at least one processor, the first input image frame to the second image plane in order to produce a projected image frame using (i) the depth map and (ii) information identifying a conversion from the first image plane to the second image plane and (b) adjusting, using the at least one processor, at least one of the depth map and the information identifying the conversion from the first image plane to the second image plane.
    Type: Grant
    Filed: March 6, 2020
    Date of Patent: November 15, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Kushal Kardam Vyas, Yingmao Li, Chenchi Luo, George Q. Chen, Hamid R. Sheikh, Youngjun Yoo, Michael O. Polley
  • Patent number: 11449968
    Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple video frames of a video stream and multiple depth frames corresponding to the multiple video frames. The method also includes generating, using the at least one processor, multiple blur kernel maps based on the multiple depth frames. The method further includes reducing, using the at least one processor, depth errors in each of the multiple blur kernel maps. The method also includes performing, using the at least one processor, temporal smoothing on the multiple blur kernel maps to suppress temporal artifacts between different ones of the multiple blur kernel maps. In addition, the method includes generating, using the at least one processor, blur effects in the video stream using the multiple blur kernel maps.
    Type: Grant
    Filed: December 31, 2020
    Date of Patent: September 20, 2022
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Kaimo Lin, Chenchi Luo, Yingmao Li, Youngjun Yoo
  • Publication number: 20220207655
    Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple video frames of a video stream and multiple depth frames corresponding to the multiple video frames. The method also includes generating, using the at least one processor, multiple blur kernel maps based on the multiple depth frames. The method further includes reducing, using the at least one processor, depth errors in each of the multiple blur kernel maps. The method also includes performing, using the at least one processor, temporal smoothing on the multiple blur kernel maps to suppress temporal artifacts between different ones of the multiple blur kernel maps. In addition, the method includes generating, using the at least one processor, blur effects in the video stream using the multiple blur kernel maps.
    Type: Application
    Filed: December 31, 2020
    Publication date: June 30, 2022
    Inventors: Kaimo Lin, Chenchi Luo, Yingmao Li, Youngjun Yoo
  • Publication number: 20220036513
    Abstract: A method includes processing, using at least one processor of an electronic device, multiple reference images of a scene using a first convolutional neural network (CNN) to generate a confidence map and a disparity map. The method also includes generating, using the at least one processor, an initial Bokeh image based on the disparity map and the reference images using a depth-of-field (DoF) renderer. The method further includes refining, using the at least one processor, the initial Bokeh image using a second CNN to generate a refined Bokeh image, where the second CNN uses the confidence map, the disparity map, and the reference images to generate the refined Bokeh image.
    Type: Application
    Filed: November 30, 2020
    Publication date: February 3, 2022
    Inventors: Chenchi Luo, Yingmao Li, Kaimo Lin, Youngjun Yoo
  • Publication number: 20220012903
    Abstract: A method includes obtaining, using at least one processor, an input image frame. The method also includes identifying, using the at least one processor, one or more regions of the input image frame containing redundant information. In addition, the method includes performing, using the at least one processor, an image processing task using the input image frame. The image processing task is guided based on the one or more identified regions of the input image frame. The method may further include obtaining, using the at least one processor, a coarse depth map associated with the input image frame. Performing the image processing task may include refining the coarse depth map to produce a refined depth map, where the refining of the coarse depth map is guided based on the one or more identified regions of the input image frame.
    Type: Application
    Filed: July 13, 2020
    Publication date: January 13, 2022
    Inventors: Kushal K. Vyas, Yingmao Li, Chenchi Luo, George Q. Chen, Hamid R. Sheikh, Youngjun Yoo, Michael O. Polley
  • Publication number: 20210390375
    Abstract: A method includes obtaining, using at least one processor of an electronic device, multiple calibration parameters associated with multiple sensors of a selected mobile device. The method also includes obtaining, using the at least one processor, an identification of multiple imaging tasks. The method further includes obtaining, using the at least one processor, multiple synthetically-generated scene images. In addition, the method includes generating, using the at least one processor, multiple training images and corresponding meta information based on the calibration parameters, the identification of the imaging tasks, and the scene images. The training images and corresponding meta information are generated concurrently, different ones of the training images correspond to different ones of the sensors, and different pieces of the meta information correspond to different ones of the imaging tasks.
    Type: Application
    Filed: December 28, 2020
    Publication date: December 16, 2021
    Inventors: Chenchi Luo, Gyeongmin Choe, Yingmao Li, Zeeshan Nadir, Hamid R. Sheikh, John Seokjun Lee, Youngjun Yoo
  • Patent number: 11195259
    Abstract: A method includes obtaining, using first and second image sensors of an electronic device, first and second images, respectively, of a scene. The method also includes obtaining, using an image depth sensor of the electronic device, a third image and a first depth map of the scene, the first depth map having a resolution lower than a resolution of the first and second images. The method further includes undistorting the first and second images using the third image and the first depth map. The method also includes rectifying the first and second images using the third image and the first depth map. The method further includes generating a disparity map using the first and second images that have been undistorted and rectified. In addition, the method includes generating a second depth map using the disparity map and the first depth map, where the second depth map has a resolution that is higher than the resolution of the first depth map.
    Type: Grant
    Filed: December 4, 2019
    Date of Patent: December 7, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Yingmao Li, Chenchi Luo, George Q. Chen, Youngjun Yoo
  • Publication number: 20210342998
    Abstract: A method includes obtaining at least one image and a ground truth map associated with the at least one image. The method also includes generating multiple optical maps using multiple algorithms and the at least one image. The method further includes, for each algorithm, identifying at least one score for the algorithm using one or more of the optical maps generated using the algorithm and the ground truth map. The ground truth map identifies one or more boundaries associated with one or more foreground objects in the at least one image. The scores identify how well the optical maps generated using the algorithms separate the one or more foreground objects from a background in the at least one image.
    Type: Application
    Filed: April 30, 2021
    Publication date: November 4, 2021
    Inventors: Yingmao Li, Chenchi Luo, Youngjun Yoo
  • Publication number: 20210334337
    Abstract: A method for performing a fundamental computational primitive in a device is provided, where the device includes a processor and a matrix multiplication accelerator (MMA). The method includes configuring a streaming engine in the device to stream data for the fundamental computational primitive from memory, configuring the MMA to format the data, and executing the fundamental computational primitive by the device.
    Type: Application
    Filed: July 4, 2021
    Publication date: October 28, 2021
    Inventors: Arthur John Redfern, Timothy David Anderson, Kai Chirca, Chenchi Luo, Zhenhua Yu
  • Patent number: 11132772
    Abstract: A method includes obtaining a first image of a scene using a first image sensor of an electronic device and a second image of the scene using a second image sensor of the electronic device. The method also includes generating a first feature map from the first image and a second feature map from the second image. The method further includes generating a third feature map based on the first feature map, the second feature map, and an asymmetric search window. The method additionally includes generating a depth map by restoring spatial resolution to the third feature map.
    Type: Grant
    Filed: December 12, 2019
    Date of Patent: September 28, 2021
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Chenchi Luo, Yingmao Li, Youngjun Yoo, George Q. Chen, Kaimo Lin, David D. Liu, Gyeongmin Choe