Patents by Inventor Congcong Li

Congcong Li has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20210192757
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for associating a new measurement of an object surrounding a vehicle with a maintained track. One of the methods includes receiving an object track for a particular object, receiving a new measurement characterizing a new object at a new time step, and determining whether the new object is the same as the particular object, comprising: generating a representation of the new object at the new and preceding time steps; generating a representation of the particular object at the new and preceding time steps; processing a first network input comprising the representations using a first neural network to generate an embedding of the first network input; and processing the embedding of the first network input using a second neural network to generate a predicted likelihood that the new object and the particular object are the same.
    Type: Application
    Filed: December 23, 2019
    Publication date: June 24, 2021
    Inventors: Ruichi Yu, Sachithra Madhawa Hemachandra, Ian James Mahon, Congcong Li
  • Publication number: 20210192238
    Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.
    Type: Application
    Filed: December 16, 2020
    Publication date: June 24, 2021
    Applicant: WAYMO LLC
    Inventors: Victoria Dean, Abhijit S. Ogale, Henrik Kretzschmar, David Harrison Silver, Carl Kershaw, Pankaj Chaudhari, Chen Wu, Congcong Li
  • Publication number: 20210150199
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using spatio-temporal-interactive networks.
    Type: Application
    Filed: November 16, 2020
    Publication date: May 20, 2021
    Inventors: Junhua Mao, Jiyang Gao, Yukai Liu, Congcong Li, Zhishuai Zhang, Dragomir Anguelov
  • Patent number: 10977501
    Abstract: Some aspects of the subject matter disclosed herein include a system implemented on one or more data processing apparatuses. The system can include an interface configured to obtain, from one or more sensor subsystems, sensor data describing an environment of a vehicle, and to generate, using the sensor data, (i) one or more first neural network inputs representing sensor measurements for a particular object in the environment and (ii) a second neural network input representing sensor measurements for at least a portion of the environment that encompasses the particular object and additional portions of the environment that are not represented by the one or more first neural network inputs; and a convolutional neural network configured to process the second neural network input to generate an output, the output including a plurality of feature vectors that each correspond to a different one a plurality of regions of the environment.
    Type: Grant
    Filed: December 21, 2018
    Date of Patent: April 13, 2021
    Assignee: Waymo LLC
    Inventors: Junhua Mao, Qian Yu, Congcong Li
  • Publication number: 20210103744
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a spatio-temporal embedding of a sequence of point clouds. One of the methods includes obtaining a temporal sequence comprising a respective point cloud input corresponding to each of a plurality of time points, each point cloud input comprising point cloud data generated from sensor data captured by one or more sensors of a vehicle at the respective time point; processing each point cloud input using a first neural network to generate a respective spatial embedding that characterizes the point cloud input; and processing the spatial embeddings of the point cloud inputs using a second neural network to generate a spatio-temporal embedding that characterizes the point cloud inputs in the temporal sequence.
    Type: Application
    Filed: October 5, 2020
    Publication date: April 8, 2021
    Inventors: Jiyang Gao, Zijian Guo, Congcong Li
  • Patent number: 10902272
    Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.
    Type: Grant
    Filed: May 20, 2020
    Date of Patent: January 26, 2021
    Assignee: WAYMO LLC
    Inventors: Victoria Dean, Abhijit S. Ogale, Henrik Kretzschmar, David Harrison Silver, Carl Kershaw, Pankaj Chaudhari, Chen Wu, Congcong Li
  • Patent number: 10867210
    Abstract: Aspects of the subject matter disclosed herein include methods, systems, and other techniques for training, in a first phase, an object classifier neural network with a first set of training data, the first set of training data including a first plurality of training examples, each training example in the first set of training data being labeled with a coarse-object classification; and training, in a second phase after completion of the first phase, the object classifier neural network with a second set of training data, the second set of training data including a second plurality of training examples, each training example in the second set of training data being labeled with a fine-object classification.
    Type: Grant
    Filed: December 21, 2018
    Date of Patent: December 15, 2020
    Assignee: Waymo LLC
    Inventors: Junhua Mao, Congcong Li, Yang Song
  • Publication number: 20200356794
    Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.
    Type: Application
    Filed: May 20, 2020
    Publication date: November 12, 2020
    Inventors: Victoria Dean, Abhijit S. Ogale, Henrik Kretzschmar, David Harrison Silver, Carl Kershaw, Pankaj Chaudhari, Chen Wu, Congcong Li
  • Patent number: 10699141
    Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.
    Type: Grant
    Filed: June 26, 2018
    Date of Patent: June 30, 2020
    Assignee: Waymo LLC
    Inventors: Victoria Dean, Abhijit S. Ogale, Henrik Kretzschmar, David Harrison Silver, Carl Kershaw, Pankaj Chaudhari, Chen Wu, Congcong Li
  • Publication number: 20200202145
    Abstract: Some aspects of the subject matter disclosed herein include a system implemented on one or more data processing apparatuses. The system can include an interface configured to obtain, from one or more sensor subsystems, sensor data describing an environment of a vehicle, and to generate, using the sensor data, (i) one or more first neural network inputs representing sensor measurements for a particular object in the environment and (ii) a second neural network input representing sensor measurements for at least a portion of the environment that encompasses the particular object and additional portions of the environment that are not represented by the one or more first neural network inputs; and a convolutional neural network configured to process the second neural network input to generate an output, the output including a plurality of feature vectors that each correspond to a different one a plurality of regions of the environment.
    Type: Application
    Filed: December 21, 2018
    Publication date: June 25, 2020
    Inventors: Junhua Mao, Qian Yu, Congcong Li
  • Publication number: 20200202209
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a classifier to detect open vehicle doors. One of the methods includes obtaining a plurality of initial training examples, each initial training example comprising (i) a sensor sample from a collection of sensor samples and (ii) data classifying the sensor sample as characterizing a vehicle that has an open door; generating a plurality of additional training examples, comprising, for each initial training example: identifying, from the collection of sensor samples, one or more additional sensor samples that were captured less than a threshold amount of time before the sensor sample in the initial training example was captured; and training the machine learning classifier on first training data that includes the initial training examples and the additional training examples to generate updated weights for the machine learning classifier.
    Type: Application
    Filed: December 21, 2018
    Publication date: June 25, 2020
    Inventors: Junhua Mao, Lo Po Tsui, Congcong Li, Edward Stephen Walker, JR.
  • Publication number: 20200202168
    Abstract: Aspects of the subject matter disclosed herein include methods, systems, and other techniques for training, in a first phase, an object classifier neural network with a first set of training data, the first set of training data including a first plurality of training examples, each training example in the first set of training data being labeled with a coarse-object classification; and training, in a second phase after completion of the first phase, the object classifier neural network with a second set of training data, the second set of training data including a second plurality of training examples, each training example in the second set of training data being labeled with a fine-object classification.
    Type: Application
    Filed: December 21, 2018
    Publication date: June 25, 2020
    Inventors: Junhua Mao, Congcong Li, Yang Song
  • Publication number: 20200202196
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for searching an autonomous vehicle sensor data repository. One of the methods includes maintaining a collection of sensor samples and, for each sensor sample, an embedding of the sensor sample; receiving a request specifying a query sensor sample, wherein the query sensor sample characterizes a query environment region; and identifying, from the collection of sensor samples, a plurality of relevant sensor samples that characterize similar environment regions to the query environment region, comprising: processing the query sensor sample through the embedding neural network to generate a query embedding; and identifying, from sensor samples in a subset of the sensor samples in the collection, a plurality of sensor samples that have embeddings that are closest to the query embedding.
    Type: Application
    Filed: December 23, 2019
    Publication date: June 25, 2020
    Inventors: Zijian Guo, Nichola Abdo, Junhua Mao, Congcong Li, Edward Stephen Walker, JR.
  • Publication number: 20200200905
    Abstract: Systems, methods, devices, and techniques for generating object-heading estimations. In one example, methods include actions of receiving sensor data representing measurements of an object that was detected within a proximity of a vehicle; processing the sensor data with one or more preliminary heading estimation subsystems to respectively generate one or more preliminary heading estimations for the object; processing two or more inputs with a second heading estimation subsystem to generate a refined heading estimation for the object, the two or more inputs including the one or more preliminary heading estimations for the object; and providing the refined heading estimation for the object to an external processing system.
    Type: Application
    Filed: December 21, 2018
    Publication date: June 25, 2020
    Inventors: David Lee, Xiaohan Jin, Congcong Li, Nichola Abdo
  • Publication number: 20200125112
    Abstract: Aspects of the disclosure relate to training and using a model for identifying actions of objects. For instance, LIDAR sensor data frames including an object bounding box corresponding to an object as well as an action label for the bounding box may be received. Each sensor frame is associated with a timestamp and is sequenced with respect to other sensor frames. Each given sensor data frame may be projected into a camera image of the object based on the timestamp associated with the given sensor data frame in order to provide fused data. The model may be trained using the fused data such that the model is configured to, in response to receiving fused data, the model outputs an action label for each object bounding box of the fused data. This output may then be used to control a vehicle in an autonomous driving mode.
    Type: Application
    Filed: October 22, 2018
    Publication date: April 23, 2020
    Inventors: Junhua Mao, Congcong Li, Alper Ayvaci, Chen Sun, Kevin Murphy, Ruichi Yu
  • Publication number: 20190392231
    Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.
    Type: Application
    Filed: June 26, 2018
    Publication date: December 26, 2019
    Inventors: Victoria Dean, Abhijit S. Ogale, Henrik Kretzschmar, David Harrison Silver, Carl Kershaw, Pankaj Chaudhari, Chen Wu, Congcong Li
  • Patent number: 10366502
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating vehicle heading predictions from point cloud data using a neural network. One of the methods includes receiving a plurality of different projections of point cloud data, wherein the point cloud data represents different sensor measurements of electromagnetic radiation reflected off a vehicle. Each of the plurality of projections of point cloud data is provided as input to a neural network subsystem trained to receive projections of point cloud data for a vehicle and to generate one or more vehicle heading classifications as an output. At the output of the neural network subsystem, one or more vehicle heading predictions is received.
    Type: Grant
    Filed: December 9, 2016
    Date of Patent: July 30, 2019
    Assignee: Waymo LLC
    Inventor: Congcong Li
  • Patent number: 9218366
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a query image model. In one aspect, a method includes receiving a set of images determined to be responsive to a query and ranked according to a first order; determining a positive image signature from a first subset of images selected from images ranked highest in the first order, determining a negative image signature from a second subset of images selected from images ranked lowest in the first order, determining a query image signature for the query based on a difference of the positive image signature and the negative image signature; and applying the query image signature to each image in the set of images to rank the images according to a second order that is different from the first order.
    Type: Grant
    Filed: December 31, 2013
    Date of Patent: December 22, 2015
    Assignee: Google Inc.
    Inventors: Congcong Li, Kunlong Gu, Charles J. Rosenberg