Abstract: Driving simulations may be generated based on driving log data and used to validate autonomous vehicle safety systems. A driving simulation system may receive driving log data from autonomous vehicles and determine events which caused a vehicle to diverge from a planned trajectory. Driving simulations may be generated based on the driving log data, and executed using a simulated vehicle that follows the initial planned trajectory of the autonomous vehicle. The driving simulations may be analyzed to detect potential collisions and near misses, and to determine success conditions for the behaviors of vehicle safety systems. Driving simulations and associated success conditions may be aggregated and used in validation suites for vehicle controllers and/or vehicle safety systems, thereby providing more comprehensive and robust autonomous vehicle validation based on real-world driving scenarios.

Abstract: Simulating output of a perception system may comprise receiving scenario data indicating a position associated with a simulated sensor and a position and/or identifier of an object, and instantiating a three-dimensional representation of an environment and the object (i.e., a simulated environment). The system may generate depth data indicating distances and/or positions of surfaces in the simulated environment relative to the simulated sensor position and determine a three-dimensional region of interest based at least in part on the depth data associated with at least a portion of the object. In some examples, the three-dimensional region of interest may be smaller than a size of the object, due to an occlusion by topology of the simulated environment and/or another object in the simulated environment.

Abstract: Procedural world generation using tertiary data is described. In an example, a computing device can receive road network data associated with the real environment and a road mesh associated with a real environment. The computing device can associate the road network data with the road mesh to generate a simulated environment. Additionally, the computing device can associate supplemental data with the road network data and the road mesh to enhance the simulated environment (e.g., supplementing information otherwise unavailable to the sensor data due to an occlusion). The computing device can output the simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous robotic computing device for at least one of navigating, planning, or decision making.

Abstract: Techniques associated with improving performance and realism of simulation instances associated with simulation testing of autonomous vehicles. In some cases, a simulation system may be configured to run a pre-simulation test to identify and store occlusion data to improve the performance of subsequent simulations associated with a shared scene or route.

Abstract: A simulation system may generate radar data for synthetic simulations of autonomous vehicles, by using a data store of object radar data and attributes determined from sensor data captured in real-world physical environments. The radar data store may include radar point clouds representing real-world objects and associated object attributes, as well as radar background data captured for a number of physical environments. The simulation system may construct radar data for use in a simulation based on radar object data and/or radar background data, including using various probabilities within various overlay regions to determine subsets of object and background radar points to be rendered. During a simulation, the generated radar data may be provided to a simulated radar sensor of a simulated vehicle configured to execute trained perception models based on radar data input.

Abstract: A simulation system may generate radar data for synthetic simulations of autonomous vehicles, by using a data store of object radar data and attributes determined from sensor data captured in real-world physical environments. The radar data store may include radar point clouds representing real-world objects and associated object attributes, as well as radar background data captured for a number of physical environments. The simulation system may construct radar data for use in a simulation based on radar object data and/or radar background data, including using various probabilities within various overlay regions to determine subsets of object and background radar points to be rendered. During a simulation, the generated radar data may be provided to a simulated radar sensor of a simulated vehicle configured to execute trained perception models based on radar data input.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: An object model for a real-world object is constructed using a parameter-based object generator, where the parameters for the object model are determined using sensor data such as photographic images, radar images, or LIDAR images obtained from the real-world object. Conversion of the sensor data to appropriate object parameters is accomplished using a machine learning system. The machine learning system is calibrated by generating a plurality of object parameters, generating a corresponding plurality of learning objects from the plurality of object parameters, generating a plurality of simulated sensor return data for the learning objects, and then providing the simulated sensor return data and object parameters to the machine learning system.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: A system for converting global shutter image data into simulated rolling shutter image data may include a global-to-rolling shutter image converter. The converter may be configured to receive global shutter image data associated with a plurality of global shutter images. The global shutter image data may include dynamic global shutter image data representative of a moving object in the plurality of global shutter images. The converter may also be configured to determine an optical flow field based at least in part on two or more of the plurality of global shutter images and generate the simulated rolling shutter image based at least in part on the optical flow field. The converter may also be configured to generate high dynamic range (HDR) images based at least in part on the optical flow field and the global shutter image data.

Abstract: A sensor simulation system may generate sensor data for use in simulations by rendering two-dimensional views of a three-dimensional simulated environment. In various examples, the sensor simulation system uses sensor dependency data to determine specific views to be re-rendered at different times during the simulation. The sensor simulation system also may generate unified views with multi-sensor data at each region (e.g., pixel) of the two-dimensional view for consumption by different sensor types. A hybrid technique may be used in some implementations in which rasterization is used to generate a view, after which ray tracing is used to align the view with a particular sensor. Spatial and temporal upsampling techniques also may be used, including depth-aware and velocity-aware analyses for simulated objects, to improve view resolution and reduce the frequency of re-rendering views.

Abstract: A driving simulation system may perform simulations using one or more of a virtual driving simulator or a log-based driving simulator. A log-based simulator may replay data from logs to test whether a control algorithm is successfully capable of navigating a scenario. However, when such log-based simulations are invalid (e.g., based on the results of the simulations or evaluations of conditions of the simulation), a virtual simulation may be generated for the vehicle control system during which new log data may be captured. Such a virtual simulation may comprise sensor simulations and more sophisticated object control for ensuring convergence to a scenario to be tested. The new log data may be used for additional log-based simulations, thereby improving the durability and flexibility for testing and/or validation, while reducing the computational overhead of driving simulation scenarios.

Abstract: A system for generating simulated LiDAR data may include a depth offset generator and an intensity value generator. The depth offset generator may be configured to receive environment data including depth information, e.g., a depth map, the environment. The depth offset generator may determine an optical flow field from the depth information and estimate depths for positions to simulate LiDAR sensor data. field. The depth offset generator can also generate timestamp information based on attributes of the simulated LiDAR sensor, and determine the estimated depths using the timestamp information. The intensity value generator may be configured to determine an intensity for pixels based on physical attributes associated with those pixels. The simulated LiDAR data may be used in simulations run on autonomous vehicle control systems.

Abstract: Procedural world generation using tertiary data is described. In an example, a computing device can receive road network data associated with the real environment and a road mesh associated with a real environment. The computing device can associate the road network data with the road mesh to generate a simulated environment. Additionally, the computing device can associate supplemental data with the road network data and the road mesh to enhance the simulated environment (e.g., supplementing information otherwise unavailable to the sensor data due to an occlusion). The computing device can output the simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous robotic computing device for at least one of navigating, planning, or decision making.

Abstract: Procedural world generation using tertiary data is described. In an example, a computing device can receive road network data associated with the real environment and a road mesh associated with a real environment. The computing device can associate the road network data with the road mesh to generate a simulated environment. Additionally, the computing device can associate supplemental data with the road network data and the road mesh to enhance the simulated environment (e.g., supplementing information otherwise unavailable to the sensor data due to an occlusion). The computing device can output the simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous robotic computing device for at least one of navigating, planning, or decision making.

Abstract: Procedural generation of simulated environments is described herein. In an example, a computing device can access road network data associated with a real environment. Further, the computing device can generate, based on the sensor data, a road mesh associated with the real environment. The computing device can associate the road network data with the road mesh to generate a simulated environment and can procedurally render at least one object into the simulated environment based at least in part on at least one rule. The computing device can output the simulated environment for at least one of testing, validating, or training an algorithm used by an autonomous vehicle for at least one of navigating, planning, or decision making.