Patents by Inventor Fuhao Shi
Fuhao Shi has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240107163Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: ApplicationFiled: November 28, 2023Publication date: March 28, 2024Inventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang
-
Publication number: 20240040250Abstract: Apparatus and methods related to stabilization of video content are provided. An example method includes receiving, by a mobile computing device, one or more image parameters associated with a video frame of a plurality of video frames. The method further includes receiving, from a motion sensor of the mobile computing device, motion data associated with the video frame. The method also includes predicting, by applying a neural network to the one or more image parameters and the motion data, a stabilized version of the video frame.Type: ApplicationFiled: December 10, 2020Publication date: February 1, 2024Inventors: Fuhao Shi, Zhenmei Shi, Wei-Sheng Lai
-
Patent number: 11856295Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: GrantFiled: October 29, 2021Date of Patent: December 26, 2023Assignee: Google LLCInventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang
-
Publication number: 20230336873Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: ApplicationFiled: June 16, 2023Publication date: October 19, 2023Inventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Patent number: 11683586Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: GrantFiled: June 1, 2021Date of Patent: June 20, 2023Assignee: Google LLCInventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Publication number: 20220345621Abstract: An example method includes outputting, for display, a frame having one or more images that are captured by at least one camera device; responsive to receiving an input to select the frame, storing the frame as a reference frame having one or more reference images, wherein the one or more reference images include the one or more images of the frame; determining, based on the reference images included in the reference frame, a reference pose of the at least one camera device; outputting, for display, a current frame having current images captured by the at least one camera device; determining, based on the current images, a current pose of the at least one camera device; and responsive to determining that the current pose is different from the reference pose, outputting an indication of instructions to manipulate the at least one camera device from the current into the reference pose.Type: ApplicationFiled: December 11, 2019Publication date: October 27, 2022Inventors: Fuhao Shi, Chia-Kai Liang, Youyou Wang
-
Publication number: 20220075991Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: ApplicationFiled: November 18, 2021Publication date: March 10, 2022Applicant: Google LLCInventors: Chia-Kai Liang, Fuhao Shi
-
Publication number: 20220053133Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: ApplicationFiled: October 29, 2021Publication date: February 17, 2022Applicant: Google LLCInventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang
-
Patent number: 11227146Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: GrantFiled: August 10, 2020Date of Patent: January 18, 2022Assignee: Google LLCInventors: Chia-Kai Liang, Fuhao Shi
-
Patent number: 11190689Abstract: Methods, systems, and apparatus, including computer programs encoded on computer-storage media, for multi-camera video stabilization. In some implementations, a video capture device has a first camera and a second camera. The video capture device provides a digital zoom capability permitting user-specified magnification changes within a digital zoom range during video recording. The video capture device is configured to use video data from different cameras over different portions of the digital zoom range. The video capture device can process image data captured using the second camera by applying a set of transformations including (i) a first transformation to a canonical reference space for the second camera, (ii) a second transformation to a canonical reference space for the first camera, and (iii) a third transformation to apply electronic image stabilization to image data in the canonical reference space for the first camera.Type: GrantFiled: July 29, 2020Date of Patent: November 30, 2021Assignee: Google LLCInventors: Youyou Wang, Fuhao Shi, Chia-Kai Liang
-
Publication number: 20210289139Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: ApplicationFiled: June 1, 2021Publication date: September 16, 2021Applicant: Google LLCInventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Patent number: 11064119Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: GrantFiled: September 12, 2019Date of Patent: July 13, 2021Assignee: Google LLCInventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Publication number: 20200374460Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: ApplicationFiled: August 10, 2020Publication date: November 26, 2020Applicant: Google LLCInventors: Chia-Kai Liang, Fuhao Shi
-
Patent number: 10812717Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: GrantFiled: December 28, 2018Date of Patent: October 20, 2020Assignee: Google LLCInventors: Chia-Kai Liang, Fuhao Shi
-
Publication number: 20200007770Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: ApplicationFiled: September 12, 2019Publication date: January 2, 2020Inventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Publication number: 20190342497Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: ApplicationFiled: December 28, 2018Publication date: November 7, 2019Inventors: Chia-Kai Liang, Fuhao Shi
-
Patent number: 10462370Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: GrantFiled: October 3, 2017Date of Patent: October 29, 2019Assignee: Google LLCInventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Publication number: 20190104255Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.Type: ApplicationFiled: October 3, 2017Publication date: April 4, 2019Inventors: Fuhao Shi, Chia-Kai Liang, Xue Tu
-
Patent number: 10171738Abstract: The subject matter described in this disclosure can be embodied in methods and systems for stabilizing video. A computing system determines a stabilized location of a facial feature in a frame of video accounting for its location in a previous frame. The computing system determines a physical camera pose in virtual space and maps the frame into virtual space. The computing system determines an optimized virtual camera pose using an optimization process that determines (1) a difference between the stabilized location of the facial feature and a location of the facial feature when viewed from a potential virtual camera pose, (2) a difference between the potential virtual camera pose and a previous virtual camera pose, and (3) a difference between the potential virtual camera pose and the physical camera pose. The computing system generates the stabilized view of the frame using the optimized virtual camera pose.Type: GrantFiled: May 4, 2018Date of Patent: January 1, 2019Assignee: Google LLCInventors: Chia-Kai Liang, Fuhao Shi