Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting occupancies of agents. One of the methods includes obtaining scene data characterizing a current scene in an environment; and processing a neural network input comprising the scene data using a neural network to generate a neural network output, wherein: the neural network output comprises respective occupancy outputs corresponding to a plurality of agent types at one or more future time points; the occupancy output for each agent type at a first future time point comprises respective occupancy probabilities for a plurality of locations in the environment; and in the occupancy output for each agent type at the first future time point, the respective occupancy probability for each location characterizes a likelihood that an agent of the agent type will occupy the location at the first future time point.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating realistic full-scene point clouds. One of the methods includes obtaining an initial scene point cloud characterizing an initial scene in an environment; obtaining, for each of one or more objects, an object point cloud that characterizes the object; and processing a first input comprising the initial scene point cloud and the one or more object point clouds using a first neural network that is configured to process the first input to generate a final scene point cloud that characterizes a transformed scene that has the one or more objects added to the initial scene.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generated simulated sensor data. One of the methods includes obtaining a surfel map generated from sensor observations of a real-world environment and generating, for each surfel in the surfel map, a respective grid having a plurality of grid cells, wherein each grid has an orientation matching an orientation of a corresponding surfel, and wherein each grid cell within each grid is assigned a respective color value. For a simulated location within a simulated representation of the real-world environment, a textured surfel rendering is generated, including combining color information from grid cells visible from the simulated location within the simulated representation of the real-world environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for multi object tracking using memory attention.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using spatio-temporal-interactive networks.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using dynamic voxelization. When deployed within an on-board system of a vehicle, processing the point cloud data using dynamic voxelization can be used to make autonomous driving decisions for the vehicle with enhanced accuracy, for example by combining representations of point cloud data characterizing a scene from multiple views of the scene.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using anchor trajectories.

Abstract: Embodiments described herein facilitate or enhance the implementation of image recognition processes which can perform recognition on images to identify objects and/or faces by class or by people.

Abstract: Embodiments described herein facilitate or enhance the implementation of image recognition processes which can perform recognition on images to identify objects and/or faces by class or by people.

Abstract: A similarity search may be performed on the image of a person, using visual characteristics and information that is known about the person. The search identifies images of other persons that are similar in appearance to the person in the image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to detect object in images. One of the methods includes receiving a training image and object location data for the training image; providing the training image to a neural network and obtaining bounding box data for the training image from the neural network, wherein the bounding box data comprises data defining a plurality of candidate bounding boxes in the training image and a respective confidence score for each candidate bounding box in the training image; determining an optimal set of assignments using the object location data for the training image and the bounding box data for the training image, wherein the optimal set of assignments assigns a respective candidate bounding box to each of the object locations; and training the neural network on the training image using the optimal set of assignments.

Abstract: An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to detect object in images. One of the methods includes receiving a training image and object location data for the training image; providing the training image to a neural network and obtaining bounding box data for the training image from the neural network, wherein the bounding box data comprises data defining a plurality of candidate bounding boxes in the training image and a respective confidence score for each candidate bounding box in the training image; determining an optimal set of assignments using the object location data for the training image and the bounding box data for the training image, wherein the optimal set of assignments assigns a respective candidate bounding box to each of the object locations; and training the neural network on the training image using the optimal set of assignments.

Abstract: An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Abstract: In one aspect, a system and method is provided that matches images that are associated with street addresses with images that are associated with locations that are stored with respect to another reference system, such as latitude/longitude. If the images match, the street address is associated with the location. In a further aspect, text contained in the images is extracted and associated with the street address as well.

Abstract: A system and method are presented for estimating the orientation of a panoramic camera mounted on a vehicle relative to the vehicle coordinate frame. An initial pose estimate of the vehicle is determined based on global positioning system data, inertial measurement unit data, and wheel odometry data of the vehicle. Image data from images captured by the camera is processed to obtain one or more tracks, each track including a sequence of matched feature points stemming from a same three-dimensional location. A correction parameter determined from the initial pose estimate and tracks can then be used to correct the orientations of the images captured by the camera. The correction parameter can be optimized by deriving a correction parameter for each of a multitude of distinct subsequences of one or more runs. Statistical analysis can be performed on the determined correction parameters to produce robust estimates.

Abstract: An image positioning system provides an interactive visualization that includes a representation of a geographic area and several camera pose indicators, each of which indicates a location within the geographic area at which a corresponding image was obtained. An operator may select one a pose indicators and adjust the position of the pose indicator relative to the representation of the geographic area. In response, the image positioning system may automatically generate a corrected location at which the image corresponding to the selected pose indicator was obtained. The corrected location then may be stored in a database and used for various applications that utilize image positioning data.

Abstract: A hierarchy of clusters is determined, where each leave of the hierarchy corresponds to one of the images in a group, and each cluster in the hierarchy identifies images in the group that are deemed similar to one another. The hierarchy identifies a similarity between each of the plurality of clusters.

Abstract: An embodiment provides for enabling retrieval of a collection of captured images that form at least a portion of a library of images. For each image in the collection, a captured image may be analyzed to recognize information from image data contained in the captured image, and an index may be generated, where the index data is based on the recognized information. Using the index, functionality such as search and retrieval is enabled. Various recognition techniques, including those that use the face, clothing, apparel, and combinations of characteristics may be utilized. Recognition may be performed on, among other things, persons and text carried on objects.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for object detection are disclosed. Methods can include, for each of a plurality of locations in one or more positive images, image filters are identified, each image filter representing visual features of a location in a positive image (e.g., an image that includes a particular object). Positive location feature scores and negative location feature scores are determined for locations within images. A positive location feature score is based on a similarity between the image filter and feature values for a positive image. A negative location feature score is determined based on a similarity between the image filter and feature values for a negative image. A distinctive location is identified based on the positive and negative location feature scores, and distinguishing feature values for identifying the particular object are identified for the distinctive location.