Patents by Inventor Cheng-Yi Liu
Cheng-Yi Liu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 12238412Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive sensor data from a drone that travels around a target object. The logic is further operable to generate, based on the sensor data, a first three-dimensional (3D) reconstruction of the target object. The logic is further operable to estimate a direction of sunlight and a direction of spectral reflection. The logic is further operable to plan a trajectory of sensor capturing positions for the drone to capture images of the target object that reduce an amount of sunlight and an amount of specular reflection.Type: GrantFiled: July 22, 2022Date of Patent: February 25, 2025Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICAInventor: Cheng-Yi Liu
-
Patent number: 12167150Abstract: An electronic device is provided that determines initial three-dimensional (3D) coordinates of a lighting device. The electronic device controls an emission of light from the lighting device based on control signals. The emitted light includes at least one of a pattern of alternating light pulses or a continuous light pulse. The electronic device controls a plurality of imaging devices to capture a first plurality of images that include information about the emitted light. Based on the determined initial 3D coordinates and the information about the emitted light included in the first plurality of images, the electronic device estimates a plurality of rotation values and a plurality of translation values of each imaging device. Based on the plurality of rotation values and the plurality of translation values, the electronic device applies a simultaneous localization and mapping process for each imaging device, for spatial synchronization of the plurality of imaging devices.Type: GrantFiled: November 1, 2021Date of Patent: December 10, 2024Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICAInventors: Brent Faust, Cheng-Yi Liu
-
Publication number: 20240387218Abstract: A cart for wafer transportation includes a cart body, a separator disposed between first and second wafer holders, an airtight lock configured to seal the cart body. A wafer transfer system includes a cart including a space for holding a wafer holder, a first workstation configured to load the wafer holder into the space and pressurize the space, and a second workstation configured to depressurize the space and unload the wafer holder from the space, wherein the cart is transportable between the first workstation and the second workstation. A method for transporting wafers includes docking a cart in a workstation; loading a wafer holder into a space of the cart; pressurizing the space to cause a pressure of the space to be greater than an atmospheric pressure; maintaining the pressure of the space at the pressure; and moving the cart carrying the wafer holder away from the workstation.Type: ApplicationFiled: July 30, 2024Publication date: November 21, 2024Inventors: Ren-Hau Wu, Cheng-Kang Hu, Chieh-Chun Lin, Jia-Hong Liao, Cheng-Yi Liu
-
Patent number: 12125727Abstract: A cart for wafer transportation includes a cart body, a separator disposed between first and second wafer holders, an airtight lock configured to seal the cart body. A wafer transfer system includes a cart including a space for holding a wafer holder, a first workstation configured to load the wafer holder into the space and pressurize the space, and a second workstation configured to depressurize the space and unload the wafer holder from the space, wherein the cart is transportable between the first workstation and the second workstation. A method for transporting wafers includes docking a cart in a workstation; loading a wafer holder into a space of the cart; pressurizing the space to cause a pressure of the space to be greater than an atmospheric pressure; maintaining the pressure of the space at the pressure; and moving the cart carrying the wafer holder away from the workstation.Type: GrantFiled: August 24, 2022Date of Patent: October 22, 2024Assignee: TAIWAN SEMICONDUCTOR MANUFACTURING COMPANY, LTD.Inventors: Ren-Hau Wu, Cheng-Kang Hu, Chieh-Chun Lin, Jia-Hong Liao, Cheng-Yi Liu
-
Patent number: 12094197Abstract: A method for removing extraneous content in a first plurality of images, captured at a corresponding plurality of poses and a corresponding first plurality of times, by a first drone, of a scene in which a second drone is present includes the following steps, for each of the first plurality of captured images. The first drone predicts a 3D position of the second drone at a time of capture of that image. The first drone defines, in an image plane corresponding to that captured image, a region of interest (ROI) including a projection of the predicted 3D position of the second drone at a time of capture of that image. A drone mask for the second drone is generated, and then applied to the defined ROI, to generate an output image free of extraneous content contributed by the second drone.Type: GrantFiled: June 10, 2021Date of Patent: September 17, 2024Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICAInventor: Cheng-Yi Liu
-
Patent number: 12026917Abstract: A method of 3D motion reconstruction of outdoor, highly flexible moving subjects with multiple cameras on drones is described herein. A multi-drone capturing system is used to remove the restriction of a dedicated virtual reality shooting place, “the hot seat,” so that the actor/target is able to perform agile or long-distance activities outdoor. The subject's 3D pose parameters are estimated by the captured multi-view images by the drones, and the sequence of poses becomes the motion in time to control the animation of a pre-built 3D model. The method is able to be directly integrated into an existing Virtual Reality/Augmented reality (VR/AR) production chain, and the subject is able to be extended to animals that are difficult to be contained in a VR camera mesh space.Type: GrantFiled: March 23, 2022Date of Patent: July 2, 2024Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICAInventors: Alexander Berestov, Cheng-Yi Liu
-
Publication number: 20240031670Abstract: An apparatus includes one or more processors and logic encoded in one or more non-transitory media for execution by the one or more processors and when executed operable to receive sensor data from a drone that travels around a target object. The logic is further operable to generate, based on the sensor data, a first three-dimensional (3D) reconstruction of the target object. The logic is further operable to estimate a direction of sunlight and a direction of spectral reflection. The logic is further operable to plan a trajectory of sensor capturing positions for the drone to capture images of the target object that reduce an amount of sunlight and an amount of specular reflection.Type: ApplicationFiled: July 22, 2022Publication date: January 25, 2024Applicants: Sony Group Corporation, Sony Corporation of AmericaInventor: Cheng-Yi Liu
-
Publication number: 20230326776Abstract: A cart for wafer transportation includes a cart body, a separator disposed between first and second wafer holders, an airtight lock configured to seal the cart body. A wafer transfer system includes a cart including a space for holding a wafer holder, a first workstation configured to load the wafer holder into the space and pressurize the space, and a second workstation configured to depressurize the space and unload the wafer holder from the space, wherein the cart is transportable between the first workstation and the second workstation. A method for transporting wafers includes docking a cart in a workstation; loading a wafer holder into a space of the cart; pressurizing the space to cause a pressure of the space to be greater than an atmospheric pressure; maintaining the pressure of the space at the pressure; and moving the cart carrying the wafer holder away from the workstation.Type: ApplicationFiled: August 24, 2022Publication date: October 12, 2023Inventors: Ren-Hau Wu, Cheng-Kang Hu, Chieh-Chun Lin, Jia-Hong Liao, Cheng-Yi Liu
-
Publication number: 20230306640Abstract: A method of 3D motion reconstruction of outdoor, highly flexible moving subjects with multiple cameras on drones is described herein. A multi-drone capturing system is used to remove the restriction of a dedicated virtual reality shooting place, “the hot seat,” so that the actor/target is able to perform agile or long-distance activities outdoor. The subject's 3D pose parameters are estimated by the captured multi-view images by the drones, and the sequence of poses becomes the motion in time to control the animation of a pre-built 3D model. The method is able to be directly integrated into an existing Virtual Reality/Augmented reality (VR/AR) production chain, and the subject is able to be extended to animals that are difficult to be contained in a VR camera mesh space.Type: ApplicationFiled: March 23, 2022Publication date: September 28, 2023Inventors: Alexander Berestov, Cheng-Yi Liu
-
Patent number: 11743444Abstract: Provided is an electronic device for a temporal synchronization, which determines a set of parameters associated with each imaging device of a plurality of imaging devices. The set of parameters include frame rate of each imaging device. The electronic device generates a synchronization signal that includes a preamble pulse of a first time duration set based on the frame rate and a sequence of alternating ON and OFF pulses. Each pulse of the sequence of alternating ON and OFF pulses is of a second time duration set based on the set of parameters. Based on the synchronization signal, lighting devices may be controlled to generate a pattern of alternating light pulses that is captured by each imaging device. The electronic device further acquires a plurality of images that includes information about the pattern of alternating light pulses. The electronic device further synchronizes the plurality of images, based on the information.Type: GrantFiled: September 2, 2021Date of Patent: August 29, 2023Assignee: SONY GROUP CORPORATIONInventors: Brent Faust, Cheng-Yi Liu
-
Patent number: 11721030Abstract: A method for optimizing image capture of a scene by a swarm of drones including a root drone and first and second level-1 drones involves the root drone following a predetermined trajectory over the scene, capturing one or more root keyframe images, at a corresponding one or more root drone orientations and root drone-to-scene distances. For each root keyframe image: the root drone generates a ground mask image for that root keyframe image, and applies that ground mask image to the root keyframe image to generate a target image. The root drone then analyzes the target image to generate first and second scanning tasks for the first and second level-1 drones to capture a plurality of images of the scene at a level-1 drone-to-scene distance smaller than the root drone-to-scene distance; and the first and second level-1 drones carry out the first and second scanning tasks respectively.Type: GrantFiled: June 10, 2021Date of Patent: August 8, 2023Assignees: SONY GROUP CORPORATION, SONY CORPORATION OF AMERICAInventor: Cheng-Yi Liu
-
Publication number: 20230135230Abstract: An electronic device is provided that determines initial three-dimensional (3D) coordinates of a lighting device. The electronic device controls an emission of light from the lighting device based on control signals. The emitted light includes at least one of a pattern of alternating light pulses or a continuous light pulse. The electronic device controls a plurality of imaging devices to capture a first plurality of images that include information about the emitted light. Based on the determined initial 3D coordinates and the information about the emitted light included in the first plurality of images, the electronic device estimates a plurality of rotation values and a plurality of translation values of each imaging device. Based on the plurality of rotation values and the plurality of translation values, the electronic device applies a simultaneous localization and mapping process for each imaging device, for spatial synchronization of the plurality of imaging devices.Type: ApplicationFiled: November 1, 2021Publication date: May 4, 2023Inventors: BRENT FAUST, CHENG-YI LIU
-
Publication number: 20230072555Abstract: Provided is an electronic device for a temporal synchronization, which determines a set of parameters associated with each imaging device of a plurality of imaging devices. The set of parameters include frame rate of each imaging device. The electronic device generates a synchronization signal that includes a preamble pulse of a first time duration set based on the frame rate and a sequence of alternating ON and OFF pulses. Each pulse of the sequence of alternating ON and OFF pulses is of a second time duration set based on the set of parameters. Based on the synchronization signal, lighting devices may be controlled to generate a pattern of alternating light pulses that is captured by each imaging device. The electronic device further acquires a plurality of images that includes information about the pattern of alternating light pulses. The electronic device further synchronizes the plurality of images, based on the information.Type: ApplicationFiled: September 2, 2021Publication date: March 9, 2023Inventors: BRENT FAUST, CHENG-YI LIU
-
Publication number: 20220398756Abstract: A method for optimizing image capture of a scene by a swarm of drones including a root drone and first and second level-1 drones involves the root drone following a predetermined trajectory over the scene, capturing one or more root keyframe images, at a corresponding one or more root drone orientations and root drone-to-scene distances. For each root keyframe image: the root drone generates a ground mask image for that root keyframe image, and applies that ground mask image to the root keyframe image to generate a target image. The root drone then analyzes the target image to generate first and second scanning tasks for the first and second level-1 drones to capture a plurality of images of the scene at a level-1 drone-to-scene distance smaller than the root drone-to-scene distance; and the first and second level-1 drones carry out the first and second scanning tasks respectively.Type: ApplicationFiled: June 10, 2021Publication date: December 15, 2022Applicants: Sony Group Corporation, Sony Corporation of AmericaInventor: Cheng-Yi Liu
-
Publication number: 20220398395Abstract: A method for removing extraneous content in a first plurality of images, captured at a corresponding plurality of poses and a corresponding first plurality of times, by a first drone, of a scene in which a second drone is present includes the following steps, for each of the first plurality of captured images. The first drone predicts a 3D position of the second drone at a time of capture of that image. The first drone defines, in an image plane corresponding to that captured image, a region of interest (ROI) including a projection of the predicted 3D position of the second drone at a time of capture of that image. A drone mask for the second drone is generated, and then applied to the defined ROI, to generate an output image free of extraneous content contributed by the second drone.Type: ApplicationFiled: June 10, 2021Publication date: December 15, 2022Applicants: Sony Group Corporation, Sony Corporation of AmericaInventor: Cheng-Yi Liu
-
Patent number: 11460867Abstract: A system of multi-view imaging of an environment through which a target moves includes pluralities of drones, each drone having a drone camera. A first plurality of drones moves to track target movement, capturing a corresponding first plurality of images of the target's face, making real time determinations of the target's head pose and gaze from the captured images and transmitting the determinations to a second plurality of drones. The second plurality of drones moves to track target movement of the target, with drone camera poses determined at least in part by the head pose and gaze determinations received from the first plurality of drones, in order to capture a second plurality of images of portions of the environment in front of the target. Post-processing of the second plurality of images allows generation of a first-person view representative of a view of the environment seen by the target.Type: GrantFiled: June 30, 2020Date of Patent: October 4, 2022Assignee: Sony Group CorporationInventor: Cheng-Yi Liu
-
Patent number: 11256257Abstract: A method of camera control for a camera capturing images of a target includes a sequence of at least four steps. The first step determines in real time a location of a drone and a pose of a camera on the drone. The second step, which may occur before or after the first step, determines in real time a location of a reference object, the location of the reference object having a fixed relationship to a location of the target. The third step uses the determined locations to calculate a distance, characterized by magnitude and direction, between the target and the drone. The fourth step uses the calculated distance to control the pose of the camera such that an image captured by the camera includes the target. Controlling the pose of the camera does not require any analysis of the captured image.Type: GrantFiled: June 30, 2020Date of Patent: February 22, 2022Assignee: Sony Group CorporationInventors: Alexander Berestov, Cheng-Yi Liu, Jianfei Chen
-
Publication number: 20210405661Abstract: A system of multi-view imaging of an environment through which a target moves includes pluralities of drones, each drone having a drone camera. A first plurality of drones moves to track target movement, capturing a corresponding first plurality of images of the target's face, making real time determinations of the target's head pose and gaze from the captured images and transmitting the determinations to a second plurality of drones. The second plurality of drones moves to track target movement of the target, with drone camera poses determined at least in part by the head pose and gaze determinations received from the first plurality of drones, in order to capture a second plurality of images of portions of the environment in front of the target. Post-processing of the second plurality of images allows generation of a first-person view representative of a view of the environment seen by the target.Type: ApplicationFiled: June 30, 2020Publication date: December 30, 2021Applicant: Sony CorporationInventor: Cheng-Yi Liu
-
Publication number: 20210407302Abstract: A system of imaging a scene includes a plurality of drones, each drone moving along a corresponding flight path over the scene and having a drone camera capturing, at a corresponding first pose and first time, a corresponding first image of the scene; a fly controller that controls the flight path of each drone, in part by using estimates of the first pose of each drone camera provided by a camera controller, to create and maintain a desired pattern of drones with desired camera poses; and the camera controller, which receives, from the drones, a corresponding plurality of captured images, processing the received images to generate a 3D representation of the scene as a system output, and to provide the estimates of the first pose of each drone camera to the fly controller. The system is fully operational with as few as one human operator.Type: ApplicationFiled: June 30, 2020Publication date: December 30, 2021Applicant: Sony Group CorporationInventors: Cheng-Yi Liu, Alexander Berestov
-
Publication number: 20210405637Abstract: A method of camera control for a camera capturing images of a target includes a sequence of at least four steps. The first step determines in real time a location of a drone and a pose of a camera on the drone. The second step, which may occur before or after the first step, determines in real time a location of a reference object, the location of the reference object having a fixed relationship to a location of the target. The third step uses the determined locations to calculate a distance, characterized by magnitude and direction, between the target and the drone. The fourth step uses the calculated distance to control the pose of the camera such that an image captured by the camera includes the target. Controlling the pose of the camera does not require any analysis of the captured image.Type: ApplicationFiled: June 30, 2020Publication date: December 30, 2021Applicant: Sony Group CorporationInventors: Alexander Berestov, Cheng-Yi Liu, Jianfei Chen