Patents by Inventor Chia-kai Liang
Chia-kai Liang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 12182967Abstract: The present disclosure describes systems and techniques for creating a super-resolution image (122) of a scene captured by a user device (102). Natural handheld motion (110) introduces, across multiple frames (204, 206, 208) of an image of a scene, sub-pixel offsets that enable the use of super-resolution computations (210) to form color planes (212, 214, 216), which are accumulated (218) and combined (220) to create a super-resolution image (122) of the scene.Type: GrantFiled: August 6, 2019Date of Patent: December 31, 2024Assignee: Google LLCInventors: Yi Hung Chen, Chia-Kai Liang, Bartlomiej Maciej Wronski, Peyman Milanfar, Ignacio Garcia Dorado
-
Patent number: 12167134Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: GrantFiled: June 16, 2023Date of Patent: December 10, 2024Assignee: Google LLCInventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Patent number: 12010440Abstract: The present disclosure describes systems and techniques directed to optical image stabilization movement to create a super-resolution image of a scene. The systems and techniques include a user device (102) introducing (502), through an optical image stabilization system (114), movement to one or more components of a camera system (112) of the user device (102). The user device (102) then captures (504) respective and multiple frames (306) of an image of a scene, where the respective and multiple frames (306) of the image of the scene have respective, sub-pixel offsets of the image of the scene across the multiple frames (306) as a result of the introduced movement to the one or more components of the camera system (112). The user device (102) performs (506), based on the respective, sub-pixel offsets of the image of the scene across the respective, multiple frames (306), super-resolution computations and creates (508) the super-resolution image of the scene based on the super-resolution computations.Type: GrantFiled: March 15, 2023Date of Patent: June 11, 2024Assignee: Google LLCInventors: Yi Hung Chen, Chia-Kai Liang, Bartlomiej Maciej Wronski, Peyman Milanfar, Ignacio Garcia Dorado
-
Publication number: 20240169498Abstract: Systems and methods for real-time image deblur and stabilization can utilize sensor data for estimating motion blur without the high computational cost of image analysis techniques. The estimated motion blur can then be utilized to generate a motion blur kernel for image correction. The systems and methods can further refine the correction by processing the motion blur kernel with a polynomial filter to generate a sharpening kernel. The systems and methods can provide for real-time correction even with minimal to no stabilization masking.Type: ApplicationFiled: July 22, 2021Publication date: May 23, 2024Inventors: Fuhao Shi, Mauricio Delbracio, Chia-Kai Liang, Damien Martin Kelly, Peyman Milanfar
-
Publication number: 20240161538Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions, and determining second costs associated with the warping mesh that include costs of edge-related transformations for preserving straightness of edges of the image. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.Type: ApplicationFiled: January 23, 2024Publication date: May 16, 2024Inventors: Yichang Shih, Chia-Kai Liang
-
Publication number: 20240107163Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: ApplicationFiled: November 28, 2023Publication date: March 28, 2024Inventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang
-
Patent number: 11922720Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions, and determining second costs associated with the warping mesh that include costs of edge-related transformations for preserving straightness of edges of the image. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.Type: GrantFiled: May 7, 2018Date of Patent: March 5, 2024Assignee: Google LLCInventors: YiChang Shih, Chia-Kai Liang
-
Patent number: 11856295Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: GrantFiled: October 29, 2021Date of Patent: December 26, 2023Assignee: Google LLCInventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang
-
Publication number: 20230336873Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: ApplicationFiled: June 16, 2023Publication date: October 19, 2023Inventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Publication number: 20230224596Abstract: The present disclosure describes systems and techniques directed to optical image stabilization movement to create a super-resolution image of a scene. The systems and techniques include a user device (102) introducing (502), through an optical image stabilization system (114), movement to one or more components of a camera system (112) of the user device (102). The user device (102) then captures (504) respective and multiple frames (306) of an image of a scene, where the respective and multiple frames (306) of the image of the scene have respective, sub-pixel offsets of the image of the scene across the multiple frames (306) as a result of the introduced movement to the one or more components of the camera system (112). The user device (102) performs (506), based on the respective, sub-pixel offsets of the image of the scene across the respective, multiple frames (306), super-resolution computations and creates (508) the super-resolution image of the scene based on the super-resolution computations.Type: ApplicationFiled: March 15, 2023Publication date: July 13, 2023Inventors: Yi Hung Chen, Chia-Kai Liang, Bartlomiej Maciej Wronski, Peyman Milanfar, Ignacio Garcia Dorado
-
Patent number: 11683586Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: GrantFiled: June 1, 2021Date of Patent: June 20, 2023Assignee: Google LLCInventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Patent number: 11611697Abstract: The present disclosure describes systems and techniques directed to optical image stabilization movement to create a super-resolution image of a scene. The systems and techniques include a user device (102) introducing (502), through an optical image stabilization system (114), movement to one or more components of a camera system (112) of the user device (102). The user device (102) then captures (504) respective and multiple frames (306) of an image of a scene, where the respective and multiple frames (306) of the image of the scene have respective, sub-pixel offsets of the image of the scene across the multiple frames (306) as a result of the introduced movement to the one or more components of the camera system (112). The user device (102) performs (506), based on the respective, sub-pixel offsets of the image of the scene across the respective, multiple frames (306), super-resolution computations and creates (508) the super-resolution image of the scene based on the super-resolution computations.Type: GrantFiled: August 6, 2019Date of Patent: March 21, 2023Assignee: Google LLCInventors: Yi Hung Chen, Chia-Kai Liang, Bartlomiej Maciej Wronski, Peyman Milanfar, Ignacio Garcia Dorado
-
Publication number: 20220345621Abstract: An example method includes outputting, for display, a frame having one or more images that are captured by at least one camera device; responsive to receiving an input to select the frame, storing the frame as a reference frame having one or more reference images, wherein the one or more reference images include the one or more images of the frame; determining, based on the reference images included in the reference frame, a reference pose of the at least one camera device; outputting, for display, a current frame having current images captured by the at least one camera device; determining, based on the current images, a current pose of the at least one camera device; and responsive to determining that the current pose is different from the reference pose, outputting an indication of instructions to manipulate the at least one camera device from the current into the reference pose.Type: ApplicationFiled: December 11, 2019Publication date: October 27, 2022Inventors: Fuhao Shi, Chia-Kai Liang, Youyou Wang
-
Publication number: 20220278186Abstract: An apparatus includes a display panel having a first pixel area having a first pixel density and a second pixel area having a second pixel density higher than the first, the panel configured to generate images viewable from a front side and a sensor positioned at the back side arranged to receive incident light transmitted from the front to the back through the first pixel area. The first pixel area has light emitting pixels and signal lines electrically connecting pixel circuits associated with the pixels, and the panel includes a layer having a light blocking material patterned to provide apertures to transmit the incident light between some of the light emitting pixels and the signal lines and to block the incident light from the pixel circuits and the signal lines. Apertures can have different dimensions.Type: ApplicationFiled: September 17, 2020Publication date: September 1, 2022Inventors: Sangmoo Choi, Jyothi Karri, Ion Bita, Chia-Kai Liang
-
Publication number: 20220075991Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: ApplicationFiled: November 18, 2021Publication date: March 10, 2022Applicant: Google LLCInventors: Chia-Kai Liang, Fuhao Shi
-
Publication number: 20220058808Abstract: Apparatus and methods related to image processing are provided. A computing device can determine a first image area of an image, such as an image captured by a camera. The computing device can determine a warping mesh for the image with a first portion of the warping mesh associated with the first image area. The computing device can determine a cost function for the warping mesh by: determining first costs associated with the first portion of the warping mesh that include costs associated with face-related transformations of the first image area to correct geometric distortions. The computing device can determine an optimized mesh based on optimizing the cost function. The computing device can modify the first image area based on the optimized mesh.Type: ApplicationFiled: August 30, 2021Publication date: February 24, 2022Inventors: Yichang Shih, Chia-Kai Liang, Wei-Sheng Lai, Ming-Hsuan Yang, Siargey Pisarchyk, Ryhor Karpiak
-
Publication number: 20220053133Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: ApplicationFiled: October 29, 2021Publication date: February 17, 2022Applicant: Google LLCInventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang
-
Patent number: 11227146Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: GrantFiled: August 10, 2020Date of Patent: January 18, 2022Assignee: Google LLCInventors: Chia-Kai Liang, Fuhao Shi
-
Publication number: 20210374909Abstract: The present disclosure describes systems and techniques directed to optical image stabilization movement to create a super-resolution image of a scene. The systems and techniques include a user device (102) introducing (502), through an optical image stabilization system (114), movement to one or more components of a camera system (112) of the user device (102). The user device (102) then captures (504) respective and multiple frames (306) of an image of a scene, where the respective and multiple frames (306) of the image of the scene have respective, sub-pixel offsets of the image of the scene across the multiple frames (306) as a result of the introduced movement to the one or more components of the camera system (112). The user device (102) performs (506), based on the respective, sub-pixel offsets of the image of the scene across the respective, multiple frames (306), super-resolution computations and creates (508) the super-resolution image of the scene based on the super-resolution computations.Type: ApplicationFiled: August 6, 2019Publication date: December 2, 2021Applicant: Google LLCInventors: Yi Hung Chen, Chia-Kai Liang, Bartlomiej Maciej Wronski, Peyman Milanfar, Ignacio Garcia Dorado
-
Patent number: 11190689Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: GrantFiled: July 29, 2020Date of Patent: November 30, 2021Assignee: Google LLCInventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang