Abstract: Techniques for improving synthetic training data generation by models used to generate training data for training object detection models are disclosed. Synthetic data may be generated by a synthetic training data generation model and provided to a crit model. The critic network may determine, based on real-world data associated with similar scenarios represented by the generated synthetic data, whether the generated synthetic data is distinguishable from real-world data. If so, the system may adjust the parameters of the synthetic training data generation model and again execute the model to generate synthetic data. This subsequent synthetic data is then critiqued by the critic network. This process may be iteratively performed until the synthetic data generated by the synthetic training data generation model is indistinguishable from real-world data. The synthetic training data generation model may then be used to generate data that may be used to train other models.

Abstract: Sensor data captured by multiple vehicle sensors may be analyzed and compared to detect sensor attacks and/or other sensor anomalies. Vehicle systems described herein may determine overlapping regions in the fields of view of multiple sensors of a vehicle, including sensors of the same or different modalities. The sensor outputs from the overlapping regions may be compared to determine when a sensor output is unexpected based on the outputs of overlapping sensors. When an unexpected sensor output is detected, the vehicle systems may analyze the sensor data to determine whether the cause of the unexpected sensor output is benign or a potential malicious attack on a sensor or group of sensors. The vehicle systems also may determine and perform responses of the vehicle to various sensor anomalies, including controlling the vehicle to avoid potential threats and/or capturing additional sensor data relating to the causes of the sensor anomalies.

Abstract: Techniques for generating high resolution images based on ToF sensor data are discussed herein. A vehicle may generate a composite image by combining multiple images, such as a first image and a second image. Since the images may be captured at different times and/or positions in the environment, to accurately combine the images, the ToF device may compensate for the motion of the vehicle. Thus, the ToF device may unproject the pixels of the first image into a three-dimensional point cloud. Further, the ToF device may update the position of the points in the point cloud. Based on the updated point cloud, the ToF device can determine a motion vector which may be applied to the first image to generate an updated image. The ToF device can determine a composite image by combining the updated image with the second image. The vehicle can be controlled based on the composite image.

Abstract: This disclosure describes methods, apparatuses, and systems for determining energy budget data based at least in part on route data and/or environmental data and controlling one or more settings associated with a vehicle based at least in part on the energy budget data. For example, a system can receive route data associated with a route for a vehicle to travel from a first location to a second location. The system can also receive environmental data representing an environment associated with the route data. The system can determine, based at least in part on the route data or the environmental data, energy budget data. The system can further control at least one of a propulsion system of the vehicle, a computing system of the vehicle, or a thermal control system of the vehicle based at least in part on the energy budget data.

Abstract: Techniques for determining location data usable to control a vehicle in an environment are discussed herein. A computing device can determine a position and/or pose of an autonomous vehicle based on an inter-vehicle communication with another autonomous vehicle from a fleet of autonomous vehicles. The autonomous vehicle can receive pose information and drive time data associated with the other autonomous vehicle for determining a current pose. The techniques can also include adjusting a predicted pose of the autonomous vehicle based on the pose information and the drive time data.

Abstract: Techniques for evaluating lane changing candidate trajectories are described herein. A vehicle may generate one or more candidate trajectories to follow within an environment. One or more of the candidate trajectories may be lane changing trajectories. As such, the vehicle may determine a lane changing cost associated with following the lane changing candidate trajectories to evaluate the degree to which the candidate trajectory respects the right-of-way of the object(s) located in the target lane. Accordingly, when generating the lane changing cost, the vehicle may determine that a relevant object is located within the target lane. Based on the object being longitudinally behind the vehicle, the vehicle may determine one or more sub-costs that may be combined into the lane changing cost. The vehicle may associate the lane change cost to the corresponding candidate trajectory and select (or determine) to follow the candidate trajectory based on the cost value.

Abstract: Techniques for generating and/or updating a timekeeping mechanism for an ECU are described herein. Monotonic and/or global time sources may be used to generate and/or update a real time clock used by component(s) of an autonomous vehicle. In some examples, a clock manager may receive a monotonic time value from a monotonic clock and a global time value from a global clock. The clock manager may use the monotonic time value and the global time value to generate an adjustment factor. In some instances, the clock manager may store the adjustment factor in a read-only shared memory location. In such instances, vehicle component may determine a predicted current time based on reading the adjustment factor from the shared memory and the monotonic time value. Upon determining the predicted current time, the vehicle component may synchronize a real time clock of the vehicle component according to the predicted current time.

Abstract: Techniques for determining an optimal trajectory for a vehicle are discussed herein. In some cases, the described techniques include receiving, from a first computing system and by a second computing system (e.g., a computing system associated with a trajectory validation system of the vehicle), a primary trajectory and an alternative trajectory for the vehicle. In some cases, the second computing system is configured to select or otherwise determine whether to select the primary trajectory, the alternative trajectory, or neither trajectory for the vehicle. In some cases, the alternative trajectory is determined by applying a lateral swerve to the primary trajectory.

Abstract: Techniques are described for self-supervised training of machine-learned models for detecting road features based on images of a driving environment. The model may be implemented as a neural network having two output heads: a first semantic segmentation head trained to output a likelihood of a particular road feature at various locations in the image, and a second geometric shape trained to output the parameters of a predicted road feature at the various locations. During training, the segmentation head of the model may be trained using labeled data associated with road features (e.g., driveways, lane geometry, etc.). The geometric shape head may be trained based on training data masked by the output of the segmentation head. In some examples, a Voronoi diagram may be generated to expand the labeled training data, and the segmentation output mask may be applied to the Voronoi diagram as the masked training data for the geometric shape head.

Abstract: There are provided methods, systems, and computer-readable media for maintain distance during vehicle interactions. The method comprises determining a first path for a vehicle traversing an environment, determining a second path for an object traversing the environment, determining a region of potential collision between the vehicle and the object, determining that one of the vehicle or the object is in or will be in the region of potential collision first, determining a stopping distance for at least the other of the vehicle or the object from a location in the region of potential collision, determining a current distance of at least said other of the vehicle or the object from the location in the region of potential collision, calculating a cost value based at least in part on the stopping distance and the current distance, and controlling the vehicle based at least in part on the cost value.

Abstract: Techniques for determining labels for potentially non-impeding objects in an environment are disclosed. Unlabeled lidar segments may be evaluated to determine whether they are located in a drivable road in an environment and to determine a lidar intensity value for the segments. Based on the intensity and whether the segment is in a drivable region, the system determines a corresponding range of lidar values associated with a label or no label. The system assigns the label (or no label) associated with the corresponding range. The label and associated segment data may then be used to classify an object associated with the segment, control a vehicle, and train a machine-learned model.

Abstract: Techniques for estimating a ground plane based on lidar data and/or attributes of the ground plane are discussed herein. A vehicle captures radar data, e.g., 4D radar data including height information, as it traverses an environment. The radar data can include direct returns from an object and reflected or multipath returns, e.g., that reflect off a ground surface and the object. A position of the ground plane can be estimated based at least in part on a distance between direct returns and the reflected returns. Attributes of the ground plane may be determined from differences between the direct returns and the reflected returns.

Abstract: This disclosure relates to methods, systems, and techniques for visualizing external audio captured by a microphone of the vehicle. Using the techniques described herein, external audio signals may be interpreted and converted into clear visual indicium that can be understood by individuals associated with the vehicle, such as passengers inside the vehicle or individuals awaiting pickup.

Abstract: Techniques are described herein for analyzing vehicle log data and determining initial movement times for stationary objects represented in the log data. Various techniques may include receiving log data including a sequence of bounding shapes representing an object detected in a physical environment, and analyzing the sequence of bounding shapes to distinguish actual object movements from noise within the perceived object data and/or errors caused by partially occluded objects. Additional techniques described herein may include executing log-based driving simulations during which simulated agents are controlled based on the initial movement times of corresponding objects in the physical environment. For instance, a simulated agent may be controlled using a trajectory of a corresponding real-world object and/or may be converted to a programmatic agent at a time during the simulation after the initial movement time of the corresponding real-world object.

Abstract: Techniques for analyzing a parameter space are discussed. Techniques may include receiving policy data for evaluating a vehicle controller. The techniques may further include determining, using a Bayesian optimization and based at least in part on the vehicle controller, parameter sets associated with adverse events. The adverse events may be associated with a violation of the policy data. The techniques may associate, based on exposure data, parameter bounds of the adverse events and probabilities of the adverse events in a driving environment. A safety metric may be determined based on the Bayesian optimization. The techniques may also include weighting an impact of an adverse event based on the safety metric.

Abstract: Techniques associated with generating and maintaining sparse geographic and map data. In some cases, the system may maintain a factor graph comprising a plurality of nodes. In some cases, the nodes may comprise pose data and sensor data associated with an autonomous vehicle at the geographic position represented by the node. The nodes may be linked based on shared trajectories and shared sensor data.

Abstract: Determining a first joint probability distribution over states of a sensor platform, the first joint probability distribution associated with a first sequence of time steps. Receiving sensor data indicating measurements at time steps of a second sequence of time steps. Integrating out variables from the first joint probability distribution to determine a second joint probability distribution over poses of the sensor platform, the second joint probability distribution associated with the second sequence of time steps. Determining, using the second joint probability distribution, parameters of a third joint probability distribution, the third joint probability distribution being characterized by unary factors for time steps of the second sequence of time steps and binary factors for pairs of neighboring time steps of the second sequence of time steps. Estimating, using the sensor data and the third joint probability distribution, poses of the sensor platform associated with the second sequence of time steps.

Abstract: Techniques for generating a local graph are described herein. A vehicle may receive a destination and generate a preferred path thereto. The vehicle may generate a local graph based on the preferred path which may define a subregion of the environment within which the vehicle may perform dynamic routing operations. For example, the vehicle may determine a local boundary which may define the subregion. Further, the vehicle may identify one or more junction(s) located within the local boundary and identify candidate (or alternative) driving lane(s) exiting such junction(s). The vehicle may generate a lane reference (e.g., optimized trajectory for the vehicle to follow to the destination) for each candidate driving lane. In such cases, the local boundary, the candidate driving lanes, and/or lane reference(s) may be data represented in the local graph. The vehicle may be controlled based on the local graph.

Abstract: Trajectories, such as for autonomous vehicles, are provided. The trajectories may be defined by a geometric profile and a temporal profile. The geometric profile defines a path through an environment independent of time. The temporal profile defines desired states of the vehicle along the path at defined points in time. In some examples, one of the profiles is modified to provide a modified profile. The vehicle is controlled at least in part on the modified profile on the other of the geometric profile and the temporal profile.

Abstract: A thermal normalization system for battery cells is described. The thermal normalization system normalizes temperatures across battery cells to reduce temperature gradients and hot zones within battery cells to improve performance and lifespan. The thermal normalization system includes parallel heat transfer paths for heat from cells, including a first path through an end of the cells in contact with a cooling element, such as a cold plate, and a second path through a heat bridge that couples a portion of the cell remote from the cooling element to the cooling element along a path thermally isolated from the first end of the cells.