Abstract: The disclosed technology provides solutions for measuring sensor realism (or fidelity) and in particular, provides methods for quantitatively evaluating the fidelity of sensor data collected using a simulated (virtual) environment. In some aspects, a process of the disclosed technology includes steps for receiving simulation training data, providing the simulation training data to an encoder neural-network to generate a plurality of simulation embeddings, and generating a first cluster based on the plurality of simulation embeddings. The process can further include steps for receiving real-world training data, providing the real-world training data to the encoder neural-network to generate a plurality of real-world embeddings, and generating a second cluster based on the plurality of real-world embeddings. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for measuring sensor realism (or fidelity) and in particular, provides methods for quantitatively determining how various sensor modeling parameters contribute to an associated fidelity metric. In some aspects, a process of the disclosed technology includes steps for receiving a first set of sensor data, determining a first fidelity score for the first set of sensor data, and processing the first set of sensor data using a perturbation algorithm to generate a second set of sensor data. In some aspects, the process can further include steps for determining a second fidelity score for the second set of sensor data. Systems and machine-readable media are also provided.

Abstract: Aspects of the disclosed technology provide solutions for measuring divergence between recorded real-world scene data, and a simulated environment. A process of the disclosed technology can include steps for generating, based on real-world autonomous vehicle (AV) scene data, a computer-generated simulation of a real-world scenario, determining, an occlusion region in which at least one surrounding object obstructs a field of view of the AV, and determining, based on the occlusion region, a portion of the object of interest that is visible to the AV. In some aspects, the process can further include steps for determining a divergence value indicating a difference between the portion of the object of interest that is visible to the AV within the simulation of the real-world scenario to an actual portion of the object of interest that is visible to the AV within the real-world scenario. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for vehicle routing and in particular, for facilitating obstacle avoidance maneuvering by an autonomous vehicle (AV). In some implementations, a process of the disclosed technology can include steps for collecting environmental data about an environment around an autonomous vehicle, wherein the environmental data comprises data pertaining to a roadway navigated by the autonomous vehicle and one or more other vehicles navigating the roadway, processing the environmental data to generate an area grid comprising one or more grid sections, wherein each grid section corresponds with a different area of the roadway, and determining a risk-profile for the one or more grid sections based on the environmental data. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for improving passenger drop-off functions implemented by an autonomous vehicle (AV). In some implementations, a process of the disclosed technology can include steps for collecting environmental data about an environment around an autonomous vehicle, wherein the environmental data comprises data pertaining to a roadway navigated by the autonomous vehicle, processing the environmental data to generate an area grid comprising a plurality of grid sections, and associating, based on the environmental data, one or more features with each of the plurality of grid sections. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for generating synthetic 3D environments, In some aspects, the disclosed technology includes a process of collecting sensor data corresponding with a three-dimensional (3D) space; identifying objects in the sensor data; filtering objects for inclusion in the synthetic map based on (1) a corresponding influence score for each of the one or more of the objects and (2) a corresponding duration of presence of each of the objects in the sensor data; selecting one or more of the objects based on the filtering, wherein the corresponding influence score of each of the one or more of the objects is greater than a first threshold and the corresponding duration of presence of each of the one or more of the objects is greater than a second threshold; and generating the synthetic map including a 3D rendering of the one or more of the objects selected.

Abstract: System, methods, and computer-readable media for a training technique of an object trajectory prediction model that inputs semantic map data pertaining to an environment and trajectory data of an object into an autoencoder neural network, which outputs the original trajectory as a feature embedding vector. Each feature embedding vector serves as a unique identifier for each behavior. Cluster analysis is performed on feature embedding vectors to determine clusters, each associated with a particular behavior attribute.

Abstract: The disclosed technology provides solutions for generating synthetic scenes based on sensor data and in particular, for generating synthetic representations of entities using splines. A process of the disclosed technology can include steps for extracting trajectory data associated with movement of an entity in an environment, generating splines based on the trajectory data, and generating a synthetic scene based on the splines. Systems and machine-readable media are also provided.

Abstract: System, methods, and computer-readable media for processing real-world environments into feature vectors that can be randomly selected into a synthetic scene. Scene data is provided to a generative machine-learning network that processes road types, environment lighting conditions, object behaviors, and AV trajectories of an AV driving scene from the sensor data into feature embeddings that represent such features from the driving scenes. The feature embeddings may be stored as unique character profiles in a scene database to be randomly selected into the synthetic scene.

Abstract: The disclosed technology provides solutions for generating synthetic objects and in particular, for generating three-dimensional objects using known geometries that can be queried from a database using feature vector information. A process of the disclosed technology can include steps for receiving sensor data including a detected object, wherein the sensor data represents one or more three-dimensional (3D) attributes of the detected object, processing the sensor data to identify at least one intrinsic attribute associated with the detected object, and querying an object database, based on the at least one intrinsic attribute, to identify a stored object that is similar to the detected object. In some aspects, the process can further include steps for modifying one or more attributes of the stored object based on the one or more 3D attributes of the detected object to generate a synthetic object. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for improving object tracking. In some aspects, a process of the disclosed technology can include steps for receiving Light Detection and Ranging (LiDAR) point cloud data corresponding with an object, providing the LiDAR point cloud data to an autoencoder neural network to generate a set of feature embeddings, and tracking the object based on the feature embeddings. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for vehicle routing and in particular, for facilitating obstacle avoidance maneuvering by an autonomous vehicle (AV). In some implementations, a process of the disclosed technology can include steps for collecting environmental data about an environment around an autonomous vehicle, wherein the environmental data comprises data pertaining to a roadway navigated by the autonomous vehicle and one or more other vehicles navigating the roadway, processing the environmental data to generate an area grid comprising one or more grid sections, wherein each grid section corresponds with a different area of the roadway, and determining a risk-profile for the one or more grid sections based on the environmental data. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for improving passenger drop-off functions implemented by an autonomous vehicle (AV). In some implementations, a process of the disclosed technology can include steps for collecting environmental data about an environment around an autonomous vehicle, wherein the environmental data comprises data pertaining to a roadway navigated by the autonomous vehicle, processing the environmental data to generate an area grid comprising a plurality of grid sections, and associating, based on the environmental data, one or more features with each of the plurality of grid sections. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions provides solutions for improving sensor data accuracy and in particular, for improving radar data by de-noising radar elevation measurements using a height-map. In some aspects, a process of the disclosed technology can include steps for receiving camera data corresponding with a first location, receiving radar data comprising a plurality of radar points, and processing the radar data to generate height-corrected radar data. In some aspects, the process can further include steps for projecting the height-corrected radar data into an image space to generate radar-image data. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for improving object detection system based on height map data. A process of the disclosed technology can include steps for receiving image data, receiving height map data, the height map data corresponding with a location of the image data, projecting the height map data onto the image data to generate composite image data, and training an object detection model based on the composite image data. In some aspects, the process can further include steps for localizing one or more objects represented by the image data using the object detection model. Systems and machine-readable media are also provided.

Abstract: The disclosed technology provides solutions for improving object classification using machine-learning techniques. In particular, solutions for improving detection/classification in rare-event scenarios are provided. In some approaches, a process of the invention can include steps for: receiving road data, wherein the road data comprises sensor data associated with a driving scene, receiving object data from an object database, and inserting a virtual object, at a first location, within the driving scene, wherein the virtual object is based on the object data. In some aspects, the process can further include steps for performing an object identification process to classify the virtual object at the first location in the driving scene. Systems and machine-readable media are also provided.