Abstract: Techniques for determining labels and occupancy data for voxels and pixels representing object protrusions are disclosed. The occupancy status of voxels surrounding an occupied voxel is determined and used to determine the occupancy density of the occupied voxel. The loss for the occupied voxel is adjusted inversely proportionately to the occupancy density. The adjusted-loss voxel is used to train a machine-learned model to detect objects in an environment and, specifically, to more accurately detect objects having protrusions that may otherwise not be associated with the object. This model may be used to provide data used to control a vehicle.

Abstract: Techniques for determining behavior profiles and unique identities for objects observed in an environment surrounding an autonomous vehicle are described. The behavior profiles may be generated and associated with unique identifiers that are stored and used by a fleet of vehicles for planning and navigating in traffic environments. The behavior profiles may be used to control operation of the vehicle and increase safety of the vehicle by identifying potentially risky or erratic objects and navigating through the environment accordingly. The behavior profiles may be stored at a central database and propagated across a fleet to increase observations used to build profiles and thereby increase profile confidence.

Abstract: Techniques for estimating a ground profile, including a ground height (e.g., elevation) and ground roll, for an environment associated with a vehicle are described herein. Sensor data associated with a vehicle may be used to determine sensor data points associated with a path of the vehicle. Based on an upper bound slope and lower bound slope, such as a maximum and/or minimum rate of ground roll change, a region (e.g., search window) may be determined. Based on the region, a subset of sensor data points included in the region may be determined. The subset of sensor data points may be used to determine a representative height and representative roll. In some instances, the representative height and representative roll may be determined using a Hough transform and the subset of sensor data points. A ground profile may be determined based on the representative height and representative roll.

Abstract: Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded regions in an environment. A system may receive a path associated with an autonomous vehicle traversing an environment. A system may detect a road segment associated with an intersection of the path and a path of a potential object. A system may determine, based at least in part on a termination condition, a length along the road segment. A system may determine, based on environment data and sensor data associated with the autonomous vehicle, an occluded region along the road segment within the length. A system may determine whether the occluded region is relevant. A system may determine, based on a minimum time to collision and determining that the occluded region is relevant, a heatmap of cost values associated with the occluded region. A system may control the autonomous vehicle based on the heatmap.

Abstract: A unitary seal comprising a plurality of attenuated portions may be cut from a sheet of material in a folded configuration, thereby maximizing an amount of the sheet that can be made into seals and minimizing wasted material of the sheet. In examples, the unitary seal may comprise four corners, a first side having three attenuated portions, and a second side opposite the first side having three attenuated portions. The plurality of attenuated portions may be associated with thinned regions of the seal that provide flexibility and behave as joints within the seal such that the seal may be unfolded after cutting and prior to installation. In some examples, individual attenuated portions may have sealing surfaces with a minimum thickness equal to about one third the thickness of a non-attenuated portion of the seal. In some instances, a minimum thickness of an attenuated portion is at least about 2.5 mm.

Abstract: A plurality of samples associated with a simulated environment in a vicinity of a depth sensor may be obtained. A first one of the plurality of samples may be within a bounded region associated with a projection from the depth sensor, and a second one of the plurality of samples may be within the bounded region associated with the projection from the depth sensor and is different from the first one of the plurality of samples. Based at least in part on the first one of the plurality of samples and the second one of the plurality of samples, a return of the depth sensor may be simulated.

Abstract: A sensor assembly includes one or more sensors configured to collect data from an environment. A sensor window is disposed in a field of view of the sensor, such that the sensor senses the environment through the sensor window to generate sensor data. The sensor window is rotated relative to the sensor to cause obstructions on the sensor window, like water or debris, to disperse.

Abstract: Techniques for training a machine learning model are described herein. For example, the techniques may include determining a gradient variable to store in memory to determine gradient values for two or more operations of a neural network. The gradient variable may represent an input or an output of an operation and can be accessed from the memory to determine multiple different gradient values during backward propagation. Such techniques may be used to train a neural network that can reduce a number of gradient variables stored in memory during forward propagation based on determining a gradient variable that is usable as an input to multiple gradient functions associated with a sequence of operations.

Abstract: Techniques for generating a dataset with clustered scenarios are provided. A first log data associated with a first vehicle is determined based on first criteria. A second log data is determined based on second criteria different from the first criteria. An instruction to generate a dataset is received. The first log data and the second log data are parameterized. A similarity between the first log data and the second log data based is determined at least in part on the parameterizing the first log data and the second log data. Based at least in part on the similarity between the first log data and the second log data, at least one of the first log data or the second log data from a dataset associated with a second vehicle is excluded.

Abstract: Techniques for filtering candidate collisions with objects in an environment are discussed herein. A vehicle may receive sensor data representative of the environment. The vehicle can evaluate such sensor data and generate instance segmentation maps that represent the environment at a future time. The vehicle can determine instance prediction maps based on the instance segmentation maps and the velocity associated with the instances included therein. The vehicle can analyze such instance prediction maps to determine whether the vehicle is predicted to collide with any of the instances. Upon identifying a potential interaction, the vehicle can determine whether the instance is located, at a future instance prediction map that is prior to the map in which the potential interaction occurs, substantially behind the vehicle. Based on the instance being located behind the vehicle, the vehicle may filter out the potential interaction from further processing and control the vehicle accordingly.

Abstract: Techniques for supervised execution of software applications on a vehicle computing device. In some cases, an example method may include receiving a first request to operate a vehicle in a first operational mode. The method may further include determining that the first operational mode is associated with a first set of software applications comprising a first software application. The method may further include based at least in part on receiving the first request, initiating execution of the first software application. The method may further include, based at least in part on receiving the first request and determining that a second software application is outside the first set, at least one of stopping execution of the second software application or refraining from initiating execution of the second software application.

Abstract: Techniques for identifying relevant objects and/or trajectories are described herein. A vehicle may identify laterally adjacent lane reference(s) and determine whether it is kinematically feasible for the vehicle to navigate to the lane reference(s). If the vehicle is unable to navigate to the lane reference, the vehicle may exclude the objects located in the associated driving lane from being included in further relevancy filtering operations. In contrast, if the vehicle can navigate to the lane reference, the vehicle may identify an object located in the driving lane of the lane reference and determine whether the object and the vehicle are predicted to overlap spatially and/or temporally. Based on identifying an overlap, the vehicle can determine a relevance score for the object trajectory and use the relevance score to determine whether the object is a relevant object and/or to determine whether the object trajectory is a relevant trajectory.

Abstract: Techniques for notifying candidate passengers of potential hazards are discussed herein. A vehicle may navigate to a location within an environment to pick up a passenger for transportation. Upon arriving at the pickup location, the vehicle may determine whether the passenger is located at the pickup location. Based on the passenger not being located at the pickup location, the vehicle may capture sensor data from one or more sensor devices on the vehicle. The vehicle may analyze the sensor data to identify potential hazard(s) proximate the pickup location. Based on identifying potential hazard(s), the vehicle may determine a recommended path for the passenger to follow from their current location to the vehicle. The vehicle may transmit a message that includes the potential hazard(s) and/or the recommended path to a user device of the passenger.

Abstract: Four-wheel steering of a vehicle, e.g., in which leading wheels and trailing wheels are steered independently of each other, can provide improved maneuverability and stability. Steering angle constraints may be used to limit or prevent saturation of steering by only one set of wheels as well as to reduce sideslip. The steering angle constraints are vehicle independent and applicable across a fleet of different vehicles based on vehicle speed. For instance, the steering angle constraints establish angle limits that dynamically adjust based on vehicle speed to ensure vehicle velocity and acceleration remain within predefined limits as established by autonomous vehicle system design.

Abstract: Techniques are discussed herein for converting neural network models and validating the conversions to maintain accuracy and avoid performance regressions. A model conversion pipeline may execute inference on original and converted neural network models to identify potential accuracy regressions. A layer-by-layer conversion and analysis can be used to determine the errors generated by specific layers or operations within the model. In some examples, conversion and validation processes may be initiated on pre-trained neural networks. Latency data also may be collected for individual layers and compared between the original and converted neural network models. An automated pipeline incorporating various conversion and validation techniques may maintain sets of conversion options for different models, to perform automatic conversion, execution, validation, and deployment of updated neural network models to the cloud or other target systems.

Abstract: Techniques for performing prediction based on relative lateral and/or longitudinal position data are described herein. A vehicle may detect an object in an environment. The vehicle may generate an embedding associated with the object and input the embedding into a machine-learned model. The machine-learned model may generate a rotation matrix based on the lateral or longitudinal of object(s), and apply the rotation matrix to the embedding. Based on modifying the embedding (or generating a modified embedding), the attention layer of the machine-learned model may perform attention on the modified embedding and respond by outputting output data. The vehicle may rotate the output data (or generate rotated output data) which may be used by one or more machine-learned models to predict object behavior, generate vehicle actions, etc.

Abstract: A reflector unit may behave as a retroreflector to reflect light of a selected color. The reflector unit may comprise a plurality of reflector elements having a first side configured to reflect light of a first color and a second side configured to reflect light of a second color. In response to receiving a signal (e.g., indicating a bi-directional vehicle has changed a direction of travel), the reflector unit may cause the reflector elements to rotate and expose a different side of the reflector elements depending on the direction of travel.

Abstract: Techniques are provided comprising receiving map data and determining a first polygon representing a first traversable surface and a second polygon representing a second traversable surface. The first polygon may include a first portion that overlaps with a second portion of the second polygon. The techniques may determine, based at least in part on elevation information associated with the first traversable surface and the second traversable surface, respective elevation layers associated with the first polygon and the second polygon. It may be determined whether to merge the first polygon and the second polygon based at least in part on whether the first polygon is associated with a same elevation layer as the second polygon. Based at least in part on determining whether to merge the first and second polygons, an updated map may be determined.

Abstract: Techniques are discussed herein for training and executing machine learning (ML) prediction models used to control autonomous vehicles in driving environments. In various examples, ML prediction models configured to output joint trajectory predictions for multiple objects in an environment may be trained by evaluating the interactions between the objects represented by the predicted trajectories. A training component may train an ML prediction model using a standard loss function based on the accuracy of the predicted trajectories relative to the ground truth trajectories, and based on an auxiliary loss determined by the agent-to-agent interactions represented by the predicted trajectories.