Patents by Inventor Cheng-Yi Liu

Cheng-Yi Liu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 12238412
    Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive sensor data from a drone that travels around a target object. The logic is further operable to generate, based on the sensor data, a first three-dimensional (3D) reconstruction of the target object. The logic is further operable to estimate a direction of sunlight and a direction of spectral reflection. The logic is further operable to plan a trajectory of sensor capturing positions for the drone to capture images of the target object that reduce an amount of sunlight and an amount of specular reflection.
    Type: Grant
    Filed: July 22, 2022
    Date of Patent: February 25, 2025
    Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICA
    Inventor: Cheng-Yi Liu
  • Patent number: 12167150
    Abstract: An electronic device is provided that determines initial three-dimensional (3D) coordinates of a lighting device. The electronic device controls an emission of light from the lighting device based on control signals. The emitted light includes at least one of a pattern of alternating light pulses or a continuous light pulse. The electronic device controls a plurality of imaging devices to capture a first plurality of images that include information about the emitted light. Based on the determined initial 3D coordinates and the information about the emitted light included in the first plurality of images, the electronic device estimates a plurality of rotation values and a plurality of translation values of each imaging device. Based on the plurality of rotation values and the plurality of translation values, the electronic device applies a simultaneous localization and mapping process for each imaging device, for spatial synchronization of the plurality of imaging devices.
    Type: Grant
    Filed: November 1, 2021
    Date of Patent: December 10, 2024
    Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICA
    Inventors: Brent Faust, Cheng-Yi Liu
  • Publication number: 20240387218
    Abstract: A cart for wafer transportation includes a cart body, a separator disposed between first and second wafer holders, an airtight lock configured to seal the cart body. A wafer transfer system includes a cart including a space for holding a wafer holder, a first workstation configured to load the wafer holder into the space and pressurize the space, and a second workstation configured to depressurize the space and unload the wafer holder from the space, wherein the cart is transportable between the first workstation and the second workstation. A method for transporting wafers includes docking a cart in a workstation; loading a wafer holder into a space of the cart; pressurizing the space to cause a pressure of the space to be greater than an atmospheric pressure; maintaining the pressure of the space at the pressure; and moving the cart carrying the wafer holder away from the workstation.
    Type: Application
    Filed: July 30, 2024
    Publication date: November 21, 2024
    Inventors: Ren-Hau Wu, Cheng-Kang Hu, Chieh-Chun Lin, Jia-Hong Liao, Cheng-Yi Liu
  • Patent number: 12125727
    Abstract: A cart for wafer transportation includes a cart body, a separator disposed between first and second wafer holders, an airtight lock configured to seal the cart body. A wafer transfer system includes a cart including a space for holding a wafer holder, a first workstation configured to load the wafer holder into the space and pressurize the space, and a second workstation configured to depressurize the space and unload the wafer holder from the space, wherein the cart is transportable between the first workstation and the second workstation. A method for transporting wafers includes docking a cart in a workstation; loading a wafer holder into a space of the cart; pressurizing the space to cause a pressure of the space to be greater than an atmospheric pressure; maintaining the pressure of the space at the pressure; and moving the cart carrying the wafer holder away from the workstation.
    Type: Grant
    Filed: August 24, 2022
    Date of Patent: October 22, 2024
    Assignee: TAIWAN SEMICONDUCTOR MANUFACTURING COMPANY, LTD.
    Inventors: Ren-Hau Wu, Cheng-Kang Hu, Chieh-Chun Lin, Jia-Hong Liao, Cheng-Yi Liu
  • Patent number: 12094197
    Abstract: A method for removing extraneous content in a first plurality of images, captured at a corresponding plurality of poses and a corresponding first plurality of times, by a first drone, of a scene in which a second drone is present includes the following steps, for each of the first plurality of captured images. The first drone predicts a 3D position of the second drone at a time of capture of that image. The first drone defines, in an image plane corresponding to that captured image, a region of interest (ROI) including a projection of the predicted 3D position of the second drone at a time of capture of that image. A drone mask for the second drone is generated, and then applied to the defined ROI, to generate an output image free of extraneous content contributed by the second drone.
    Type: Grant
    Filed: June 10, 2021
    Date of Patent: September 17, 2024
    Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICA
    Inventor: Cheng-Yi Liu
  • Patent number: 12026917
    Abstract: A method of 3D motion reconstruction of outdoor, highly flexible moving subjects with multiple cameras on drones is described herein. A multi-drone capturing system is used to remove the restriction of a dedicated virtual reality shooting place, “the hot seat,” so that the actor/target is able to perform agile or long-distance activities outdoor. The subject's 3D pose parameters are estimated by the captured multi-view images by the drones, and the sequence of poses becomes the motion in time to control the animation of a pre-built 3D model. The method is able to be directly integrated into an existing Virtual Reality/Augmented reality (VR/AR) production chain, and the subject is able to be extended to animals that are difficult to be contained in a VR camera mesh space.
    Type: Grant
    Filed: March 23, 2022
    Date of Patent: July 2, 2024
    Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICA
    Inventors: Alexander Berestov, Cheng-Yi Liu
  • Publication number: 20240031670
    Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive sensor data from a drone that travels around a target object. The logic is further operable to generate, based on the sensor data, a first three-dimensional (3D) reconstruction of the target object. The logic is further operable to estimate a direction of sunlight and a direction of spectral reflection. The logic is further operable to plan a trajectory of sensor capturing positions for the drone to capture images of the target object that reduce an amount of sunlight and an amount of specular reflection.
    Type: Application
    Filed: July 22, 2022
    Publication date: January 25, 2024
    Applicants: Sony Group Corporation, Sony Corporation of America
    Inventor: Cheng-Yi Liu
  • Publication number: 20230326776
    Abstract: A cart for wafer transportation includes a cart body, a separator disposed between first and second wafer holders, an airtight lock configured to seal the cart body. A wafer transfer system includes a cart including a space for holding a wafer holder, a first workstation configured to load the wafer holder into the space and pressurize the space, and a second workstation configured to depressurize the space and unload the wafer holder from the space, wherein the cart is transportable between the first workstation and the second workstation. A method for transporting wafers includes docking a cart in a workstation; loading a wafer holder into a space of the cart; pressurizing the space to cause a pressure of the space to be greater than an atmospheric pressure; maintaining the pressure of the space at the pressure; and moving the cart carrying the wafer holder away from the workstation.
    Type: Application
    Filed: August 24, 2022
    Publication date: October 12, 2023
    Inventors: Ren-Hau Wu, Cheng-Kang Hu, Chieh-Chun Lin, Jia-Hong Liao, Cheng-Yi Liu
  • Publication number: 20230306640
    Abstract: A method of 3D motion reconstruction of outdoor, highly flexible moving subjects with multiple cameras on drones is described herein. A multi-drone capturing system is used to remove the restriction of a dedicated virtual reality shooting place, “the hot seat,” so that the actor/target is able to perform agile or long-distance activities outdoor. The subject's 3D pose parameters are estimated by the captured multi-view images by the drones, and the sequence of poses becomes the motion in time to control the animation of a pre-built 3D model. The method is able to be directly integrated into an existing Virtual Reality/Augmented reality (VR/AR) production chain, and the subject is able to be extended to animals that are difficult to be contained in a VR camera mesh space.
    Type: Application
    Filed: March 23, 2022
    Publication date: September 28, 2023
    Inventors: Alexander Berestov, Cheng-Yi Liu
  • Patent number: 11743444
    Abstract: Provided is an electronic device for a temporal synchronization, which determines a set of parameters associated with each imaging device of a plurality of imaging devices. The set of parameters include frame rate of each imaging device. The electronic device generates a synchronization signal that includes a preamble pulse of a first time duration set based on the frame rate and a sequence of alternating ON and OFF pulses. Each pulse of the sequence of alternating ON and OFF pulses is of a second time duration set based on the set of parameters. Based on the synchronization signal, lighting devices may be controlled to generate a pattern of alternating light pulses that is captured by each imaging device. The electronic device further acquires a plurality of images that includes information about the pattern of alternating light pulses. The electronic device further synchronizes the plurality of images, based on the information.
    Type: Grant
    Filed: September 2, 2021
    Date of Patent: August 29, 2023
    Assignee: SONY GROUP CORPORATION
    Inventors: Brent Faust, Cheng-Yi Liu
  • Patent number: 11721030
    Abstract: A method for optimizing image capture of a scene by a swarm of drones including a root drone and first and second level-1 drones involves the root drone following a predetermined trajectory over the scene, capturing one or more root keyframe images, at a corresponding one or more root drone orientations and root drone-to-scene distances. For each root keyframe image: the root drone generates a ground mask image for that root keyframe image, and applies that ground mask image to the root keyframe image to generate a target image. The root drone then analyzes the target image to generate first and second scanning tasks for the first and second level-1 drones to capture a plurality of images of the scene at a level-1 drone-to-scene distance smaller than the root drone-to-scene distance; and the first and second level-1 drones carry out the first and second scanning tasks respectively.
    Type: Grant
    Filed: June 10, 2021
    Date of Patent: August 8, 2023
    Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICA
    Inventor: Cheng-Yi Liu
  • Publication number: 20230135230
    Abstract: An electronic device is provided that determines initial three-dimensional (3D) coordinates of a lighting device. The electronic device controls an emission of light from the lighting device based on control signals. The emitted light includes at least one of a pattern of alternating light pulses or a continuous light pulse. The electronic device controls a plurality of imaging devices to capture a first plurality of images that include information about the emitted light. Based on the determined initial 3D coordinates and the information about the emitted light included in the first plurality of images, the electronic device estimates a plurality of rotation values and a plurality of translation values of each imaging device. Based on the plurality of rotation values and the plurality of translation values, the electronic device applies a simultaneous localization and mapping process for each imaging device, for spatial synchronization of the plurality of imaging devices.
    Type: Application
    Filed: November 1, 2021
    Publication date: May 4, 2023
    Inventors: BRENT FAUST, CHENG-YI LIU
  • Publication number: 20230072555
    Abstract: Provided is an electronic device for a temporal synchronization, which determines a set of parameters associated with each imaging device of a plurality of imaging devices. The set of parameters include frame rate of each imaging device. The electronic device generates a synchronization signal that includes a preamble pulse of a first time duration set based on the frame rate and a sequence of alternating ON and OFF pulses. Each pulse of the sequence of alternating ON and OFF pulses is of a second time duration set based on the set of parameters. Based on the synchronization signal, lighting devices may be controlled to generate a pattern of alternating light pulses that is captured by each imaging device. The electronic device further acquires a plurality of images that includes information about the pattern of alternating light pulses. The electronic device further synchronizes the plurality of images, based on the information.
    Type: Application
    Filed: September 2, 2021
    Publication date: March 9, 2023
    Inventors: BRENT FAUST, CHENG-YI LIU
  • Publication number: 20220398756
    Abstract: A method for optimizing image capture of a scene by a swarm of drones including a root drone and first and second level-1 drones involves the root drone following a predetermined trajectory over the scene, capturing one or more root keyframe images, at a corresponding one or more root drone orientations and root drone-to-scene distances. For each root keyframe image: the root drone generates a ground mask image for that root keyframe image, and applies that ground mask image to the root keyframe image to generate a target image. The root drone then analyzes the target image to generate first and second scanning tasks for the first and second level-1 drones to capture a plurality of images of the scene at a level-1 drone-to-scene distance smaller than the root drone-to-scene distance; and the first and second level-1 drones carry out the first and second scanning tasks respectively.
    Type: Application
    Filed: June 10, 2021
    Publication date: December 15, 2022
    Applicants: Sony Group Corporation, Sony Corporation of America
    Inventor: Cheng-Yi Liu
  • Publication number: 20220398395
    Abstract: A method for removing extraneous content in a first plurality of images, captured at a corresponding plurality of poses and a corresponding first plurality of times, by a first drone, of a scene in which a second drone is present includes the following steps, for each of the first plurality of captured images. The first drone predicts a 3D position of the second drone at a time of capture of that image. The first drone defines, in an image plane corresponding to that captured image, a region of interest (ROI) including a projection of the predicted 3D position of the second drone at a time of capture of that image. A drone mask for the second drone is generated, and then applied to the defined ROI, to generate an output image free of extraneous content contributed by the second drone.
    Type: Application
    Filed: June 10, 2021
    Publication date: December 15, 2022
    Applicants: Sony Group Corporation, Sony Corporation of America
    Inventor: Cheng-Yi Liu
  • Patent number: 11460867
    Abstract: A system of multi-view imaging of an environment through which a target moves includes pluralities of drones, each drone having a drone camera. A first plurality of drones moves to track target movement, capturing a corresponding first plurality of images of the target's face, making real time determinations of the target's head pose and gaze from the captured images and transmitting the determinations to a second plurality of drones. The second plurality of drones moves to track target movement of the target, with drone camera poses determined at least in part by the head pose and gaze determinations received from the first plurality of drones, in order to capture a second plurality of images of portions of the environment in front of the target. Post-processing of the second plurality of images allows generation of a first-person view representative of a view of the environment seen by the target.
    Type: Grant
    Filed: June 30, 2020
    Date of Patent: October 4, 2022
    Assignee: Sony Group Corporation
    Inventor: Cheng-Yi Liu
  • Patent number: 11256257
    Abstract: A method of camera control for a camera capturing images of a target includes a sequence of at least four steps. The first step determines in real time a location of a drone and a pose of a camera on the drone. The second step, which may occur before or after the first step, determines in real time a location of a reference object, the location of the reference object having a fixed relationship to a location of the target. The third step uses the determined locations to calculate a distance, characterized by magnitude and direction, between the target and the drone. The fourth step uses the calculated distance to control the pose of the camera such that an image captured by the camera includes the target. Controlling the pose of the camera does not require any analysis of the captured image.
    Type: Grant
    Filed: June 30, 2020
    Date of Patent: February 22, 2022
    Assignee: Sony Group Corporation
    Inventors: Alexander Berestov, Cheng-Yi Liu, Jianfei Chen
  • Publication number: 20210405661
    Abstract: A system of multi-view imaging of an environment through which a target moves includes pluralities of drones, each drone having a drone camera. A first plurality of drones moves to track target movement, capturing a corresponding first plurality of images of the target's face, making real time determinations of the target's head pose and gaze from the captured images and transmitting the determinations to a second plurality of drones. The second plurality of drones moves to track target movement of the target, with drone camera poses determined at least in part by the head pose and gaze determinations received from the first plurality of drones, in order to capture a second plurality of images of portions of the environment in front of the target. Post-processing of the second plurality of images allows generation of a first-person view representative of a view of the environment seen by the target.
    Type: Application
    Filed: June 30, 2020
    Publication date: December 30, 2021
    Applicant: Sony Corporation
    Inventor: Cheng-Yi Liu
  • Publication number: 20210407302
    Abstract: A system of imaging a scene includes a plurality of drones, each drone moving along a corresponding flight path over the scene and having a drone camera capturing, at a corresponding first pose and first time, a corresponding first image of the scene; a fly controller that controls the flight path of each drone, in part by using estimates of the first pose of each drone camera provided by a camera controller, to create and maintain a desired pattern of drones with desired camera poses; and the camera controller, which receives, from the drones, a corresponding plurality of captured images, processing the received images to generate a 3D representation of the scene as a system output, and to provide the estimates of the first pose of each drone camera to the fly controller. The system is fully operational with as few as one human operator.
    Type: Application
    Filed: June 30, 2020
    Publication date: December 30, 2021
    Applicant: Sony Group Corporation
    Inventors: Cheng-Yi Liu, Alexander Berestov
  • Publication number: 20210405637
    Abstract: A method of camera control for a camera capturing images of a target includes a sequence of at least four steps. The first step determines in real time a location of a drone and a pose of a camera on the drone. The second step, which may occur before or after the first step, determines in real time a location of a reference object, the location of the reference object having a fixed relationship to a location of the target. The third step uses the determined locations to calculate a distance, characterized by magnitude and direction, between the target and the drone. The fourth step uses the calculated distance to control the pose of the camera such that an image captured by the camera includes the target. Controlling the pose of the camera does not require any analysis of the captured image.
    Type: Application
    Filed: June 30, 2020
    Publication date: December 30, 2021
    Applicant: Sony Group Corporation
    Inventors: Alexander Berestov, Cheng-Yi Liu, Jianfei Chen