Patents by Inventor Kai Zhenyu Wang

Kai Zhenyu Wang has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250201125
    Abstract: Techniques are discussed for determining prediction probabilities of an object based on a top-down representation of an environment. Data representing objects in an environment can be captured. Aspects of the environment can be represented as map data. A multi-channel image representing a top-down view of object(s) in the environment can be generated based on the data representing the objects and map data. The multi-channel image can be used to train a machine learned model by minimizing an error between predictions from the machine learned model and a captured trajectory associated with the object. Once trained, the machine learned model can be used to generate prediction probabilities of objects in an environment, and the vehicle can be controlled based on such prediction probabilities.
    Type: Application
    Filed: December 12, 2024
    Publication date: June 19, 2025
    Applicant: Zoox, Inc.
    Inventors: Xi Joey Hong, Benjamin John Sapp, James William Vaisey Philbin, Kai Zhenyu Wang
  • Patent number: 12291240
    Abstract: Techniques for determining a response of a simulated vehicle to a simulated object in a simulation are discussed herein. Log data captured by a physical vehicle in an environment can be received. Object data representing an object in the log data can be used to instantiate a simulated object in a simulation to determine a response of a simulated vehicle to the simulated object. Additionally, one or more trajectory segments in a trajectory library representing the log data can be determined and instantiated as a trajectory of the simulated object in order to increase the accuracy and realism of the simulation.
    Type: Grant
    Filed: November 18, 2021
    Date of Patent: May 6, 2025
    Assignee: Zoox, Inc.
    Inventors: Andres Guillermo Morales Morales, Samir Parikh, Kai Zhenyu Wang
  • Patent number: 12271204
    Abstract: Techniques are discussed for predicting an occupancy of visible region of an environment. For instance, a vehicle may generate sensor data representing an environment. The vehicle may then analyze the sensor data to determine an occluded region of the environment a visible region of the environment. Additionally, the vehicle may determine at least one prediction probability associated with occupancy of the visible region over a future period of time. In some instances, the vehicle determines the at least one prediction probability by inputting data representing at least the occluded region and the visible region into a machine learned model and receiving the at least one prediction probability from the machine learned model. Using the at least one prediction probability, the vehicle may then determine and perform one or more actions.
    Type: Grant
    Filed: October 27, 2020
    Date of Patent: April 8, 2025
    Assignee: Zoox, Inc.
    Inventors: Gowtham Garimella, Marin Kobilarov, Kai Zhenyu Wang
  • Patent number: 12269462
    Abstract: Techniques relating to determining regions based on intents of objects are described. In an example, a computing device onboard a first vehicle can receive sensor data associated with an environment of the first vehicle. The computing device can determine, based on the sensor data, a region associated with a second vehicle proximate the first vehicle that is to be occupied by the second vehicle while the vehicle performs a maneuver. Further, the computing device can determine an instruction for controlling the first vehicle based at least in part on the region.
    Type: Grant
    Filed: March 16, 2020
    Date of Patent: April 8, 2025
    Assignee: Zoox, Inc.
    Inventors: Gowtham Garimella, Marin Kobilarov, Kai Zhenyu Wang
  • Patent number: 12183204
    Abstract: Techniques are discussed for determining prediction probabilities of an object based on a top-down representation of an environment. Data representing objects in an environment can be captured. Aspects of the environment can be represented as map data. A multi-channel image representing a top-down view of object(s) in the environment can be generated based on the data representing the objects and map data. The multi-channel image can be used to train a machine learned model by minimizing an error between predictions from the machine learned model and a captured trajectory associated with the object. Once trained, the machine learned model can be used to generate prediction probabilities of objects in an environment, and the vehicle can be controlled based on such prediction probabilities.
    Type: Grant
    Filed: December 6, 2021
    Date of Patent: December 31, 2024
    Assignee: Zoox, Inc.
    Inventors: Xi Joey Hong, Benjamin John Sapp, James William Vaisey Philbin, Kai Zhenyu Wang
  • Patent number: 12103561
    Abstract: Techniques relating to monitoring map consistency are described. In an example, a monitoring component associated with a vehicle can receive sensor data associated with an environment in which the vehicle is positioned. The monitoring component can generate, based at least in part on the sensor data, an estimated map of the environment, wherein the estimated map is encoded with policy information for driving within the environment. The monitoring component can then compare first information associated with a stored map of the environment with second information associated with the estimated map to determine whether the estimated map and the stored map are consistent. Component(s) associated with the vehicle can then control the object based at least in part on results of the comparing.
    Type: Grant
    Filed: October 14, 2022
    Date of Patent: October 1, 2024
    Assignee: Zoox, Inc.
    Inventors: Pengfei Duan, James William Vaisey Philbin, Cooper Stokes Sloan, Sarah Tariq, Feng Tian, Chuang Wang, Kai Zhenyu Wang, Yi Xu
  • Patent number: 12084087
    Abstract: Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a distribution of predicted positions for the object in the future that meet a criterion, allowing for more efficient sampling. A predicted position of the object in the future may be determined by sampling from the distribution.
    Type: Grant
    Filed: November 24, 2021
    Date of Patent: September 10, 2024
    Assignee: Zoox, Inc.
    Inventors: Gowtham Garimella, Marin Kobilarov, Andres Guillermo Morales Morales, Ethan Miller Pronovost, Kai Zhenyu Wang, Xiaosi Zeng
  • Patent number: 12080044
    Abstract: Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a distribution of predicted positions for the object in the future. A predicted position of the object at a subsequent timestep may be determined by sampling from the distribution of predicted positions according to various sampling strategies. Alternatively, the predicted position of the object may be overwritten using a candidate position of the object.
    Type: Grant
    Filed: November 24, 2021
    Date of Patent: September 3, 2024
    Assignee: Zoox, Inc.
    Inventors: Gowtham Garimella, Marin Kobilarov, Andres Guillermo Morales Morales, Ethan Miller Pronovost, Kai Zhenyu Wang, Xiaosi Zeng
  • Patent number: 12065171
    Abstract: Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a predicted position of the object at a subsequent timestep. Further, a predicted trajectory of the object may be determined using predicted positions of the object at various timesteps.
    Type: Grant
    Filed: November 24, 2021
    Date of Patent: August 20, 2024
    Assignee: Zoox, Inc.
    Inventors: Gowtham Garimella, Marin Kobilarov, Andres Guillermo Morales Morales, Ethan Miller Pronovost, Kai Zhenyu Wang, Xiaosi Zeng
  • Patent number: 12013693
    Abstract: Techniques are disclosed for component verification for complex systems. The techniques may include receiving log data, obtaining ground truth data based on the log data and determining an outcome at least in part by simulating a prediction by a prediction component based on the log data and the ground truth data. The techniques may further include simulating a second prediction by the prediction component based on the ground truth data, determining whether the second prediction resulted in the negative outcome of the scenario and determining the disengagement event is attributable to a perception component of the autonomous operation system at least partly in response to determining the second prediction based on the ground truth data did not result in the negative outcome.
    Type: Grant
    Filed: December 30, 2019
    Date of Patent: June 18, 2024
    Assignee: Zoox, Inc.
    Inventors: Jonathan Philip Wai Wah Chan, Kai Zhenyu Wang
  • Patent number: 11966230
    Abstract: Techniques for determining a prediction probability associated with a disengagement event are discussed herein. A first prediction probability can include a probability that a safety driver associated with a vehicle (such as an autonomous vehicle) may assume control over the vehicle. A second prediction probability can include a probability that an object in an environment is associated the disengagement event. Sensor data can be captured and represented as a top-down representation of the environment. The top-down representation can be input to a machine learned model trained to output prediction probabilities associated with a disengagement event. The vehicle can be controlled based the prediction probability and/or the interacting object probability.
    Type: Grant
    Filed: December 17, 2020
    Date of Patent: April 23, 2024
    Assignee: ZOOX, INC.
    Inventors: Greg Woelki, Kai Zhenyu Wang, Bertrand Robert Douillard, Michael Haggblade, James William Vaisey Philbin
  • Patent number: 11858514
    Abstract: Techniques for top-down scene discrimination are discussed. A system receives scene data associated with an environment proximate a vehicle. The scene data is input to a convolutional neural network (CNN) discriminator trained using a generator and a classification of the output of the CNN discriminator. The CNN discriminator generates an indication of whether the scene data is a generated scene or a captured scene. If the scene data is data generated scene, the system generates a caution notification indicating that a current environmental situation is different from any previous situations. Additionally, the caution notification is communicated to at least one of a vehicle system or a remote vehicle monitoring system.
    Type: Grant
    Filed: March 30, 2021
    Date of Patent: January 2, 2024
    Assignee: ZOOX, INC.
    Inventors: Gerrit Bagschik, Andrew Scott Crego, Gowtham Garimella, Michael Haggblade, Andraz Kavalar, Kai Zhenyu Wang
  • Patent number: 11810225
    Abstract: Techniques for top-down scene generation are discussed. A generator component may receive multi-dimensional input data associated with an environment. The generator component may generate, based at least in part on the multi-dimensional input data, a generated top-down scene. A discriminator component receives the generated top-down scene and a real top-down scene. The discriminator component generates binary classification data indicating whether an individual scene in the scene data is classified as generated or classified as real. The binary classification data is provided as a loss to the generator component and the discriminator component.
    Type: Grant
    Filed: March 30, 2021
    Date of Patent: November 7, 2023
    Assignee: Zoox, Inc.
    Inventors: Gerrit Bagschik, Andrew Scott Crego, Gowtham Garimella, Michael Haggblade, Andraz Kavalar, Kai Zhenyu Wang
  • Patent number: 11810365
    Abstract: Techniques for modeling the probability distribution of errors in perception systems are discussed herein. For example, techniques may include modeling error distribution for attributes such as position, size, pose, and velocity of objects detected in an environment, and training a mixture model to output specific error probability distributions based on input features such as object classification, distance to the object, and occlusion. The output of the trained model may be used to control the operation of a vehicle in an environment, generate simulations, perform collision probability analyses, and to mine log data to detect collision risks.
    Type: Grant
    Filed: December 15, 2020
    Date of Patent: November 7, 2023
    Assignee: Zoox, Inc.
    Inventors: Andrew Scott Crego, Gowtham Garimella, Mahsa Ghafarianzadeh, Rasmus Fonseca, Muhammad Farooq Rama, Kai Zhenyu Wang
  • Patent number: 11734832
    Abstract: Techniques for determining predictions on a top-down representation of an environment based on object movement are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) may capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle, a pedestrian, a bicycle). A multi-channel image representing a top-down view of the object(s) and the environment may be generated based in part on the sensor data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) may also be encoded in the image. Multiple images may be generated representing the environment over time and input into a prediction system configured to output a trajectory template (e.g., general intent for future movement) and a predicted trajectory (e.g., more accurate predicted movement) associated with each object. The prediction system may include a machine learned model configured to output the trajectory template(s) and the predicted trajector(ies).
    Type: Grant
    Filed: February 2, 2022
    Date of Patent: August 22, 2023
    Assignee: Zoox, Inc.
    Inventors: Andres Guillermo Morales Morales, Marin Kobilarov, Gowtham Garimella, Kai Zhenyu Wang
  • Patent number: 11708093
    Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include determining a trajectory of the object, determining an intent of the trajectory, and sending the trajectory and the intent to a vehicle computing system to control an autonomous vehicle. The vehicle computing system may implement a machine learned model to process data such as sensor data and map data. The machine learned model can associate different intentions of an object in an environment with different trajectories. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on object's intentions and trajectories.
    Type: Grant
    Filed: May 8, 2020
    Date of Patent: July 25, 2023
    Assignee: Zoox, Inc.
    Inventors: Kenneth Michael Siebert, Gowtham Garimella, Benjamin Isaac Mattinson, Samir Parikh, Kai Zhenyu Wang
  • Publication number: 20230159059
    Abstract: Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a predicted position of the object at a subsequent timestep. Further, a predicted trajectory of the object may be determined using predicted positions of the object at various timesteps.
    Type: Application
    Filed: November 24, 2021
    Publication date: May 25, 2023
    Inventors: Gowtham Garimella, Marin Kobilarov, Andres Guillermo Morales Morales, Ethan Miller Pronovost, Kai Zhenyu Wang, Xiaosi Zeng
  • Publication number: 20230159060
    Abstract: Techniques for determining unified futures of objects in an environment are discussed herein. Techniques may include determining a first feature associated with an object in an environment and a second feature associated with the environment and based on a position of the object in the environment, updating a graph neural network (GNN) to encode the first feature and second feature into a graph node representing the object and encode relative positions of additional objects in the environment into one or more edges attached to the node. The GNN may be decoded to determine a distribution of predicted positions for the object in the future that meet a criterion, allowing for more efficient sampling. A predicted position of the object in the future may be determined by sampling from the distribution.
    Type: Application
    Filed: November 24, 2021
    Publication date: May 25, 2023
    Inventors: Gowtham Garimella, Marin Kobilarov, Andres Guillermo Morales Morales, Ethan Miller Pronovost, Kai Zhenyu Wang, Xiaosi Zeng
  • Publication number: 20230150549
    Abstract: Techniques for determining a response of a simulated vehicle to a simulated object in a simulation are discussed herein. Log data captured by a physical vehicle in an environment can be received. Object data representing an object in the log data can be used to instantiate a simulated object in a simulation to determine a response of a simulated vehicle to the simulated object. Additionally, one or more trajectory segments in a trajectory library representing the log data can be determined and instantiated as a trajectory of the simulated object in order to increase the accuracy and realism of the simulation.
    Type: Application
    Filed: November 18, 2021
    Publication date: May 18, 2023
    Inventors: Andres Guillermo Morales Morales, Samir Parikh, Kai Zhenyu Wang
  • Patent number: 11631200
    Abstract: Techniques for determining predictions on a top-down representation of an environment based on vehicle action(s) are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) can capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle or a pedestrian). A multi-channel image representing a top-down view of the object(s) and the environment can be generated based on the sensor data, map data, and/or action data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) can be encoded in the image. Action data can represent a target lane, trajectory, etc. of the first vehicle. Multiple images can be generated representing the environment over time and input into a prediction system configured to output prediction probabilities associated with possible locations of the object(s) in the future, which may be based on the actions of the autonomous vehicle.
    Type: Grant
    Filed: May 20, 2021
    Date of Patent: April 18, 2023
    Assignee: Zoox, Inc.
    Inventors: Gowtham Garimella, Marin Kobilarov, Andres Guillermo Morales Morales, Kai Zhenyu Wang