Abstract: Techniques for segmenting sensor data are discussed herein. Data can be represented in individual levels in a multi-resolution voxel space. A first level can correspond to a first region of an environment and a second level can correspond to a second region of an environment that is a subset of the first region. In some examples, the levels can comprise a same number of voxels, such that the first level covers a large, low-resolution region, while the second level covers a smaller, higher-resolution region, though more levels are contemplated. Operations may include analyzing sensor data represented in the voxel space from a perspective, such as a top-down perspective. From this perspective, techniques may generate masks that represent objects in the voxel space. Additionally, techniques may generate segmentation data to verify and/or generate the masks, or otherwise cluster the sensor data.

Abstract: Techniques for determining predictions on a top-down representation of an environment based on vehicle action(s) are discussed herein. Sensors of a first vehicle (such as an autonomous vehicle) can capture sensor data of an environment, which may include object(s) separate from the first vehicle (e.g., a vehicle or a pedestrian). A multi-channel image representing a top-down view of the object(s) and the environment can be generated based on the sensor data, map data, and/or action data. Environmental data (object extents, velocities, lane positions, crosswalks, etc.) can be encoded in the image. Action data can represent a target lane, trajectory, etc. of the first vehicle. Multiple images can be generated representing the environment over time and input into a prediction system configured to output prediction probabilities associated with possible locations of the object(s) in the future, which may be based on the actions of the autonomous vehicle.

Abstract: Ground truth data may be too sparse to supervise training of a machine-learned (ML) model enough to achieve an ML model with sufficient accuracy/recall. For example, in some cases, ground truth data may only be available for every third, tenth, or hundredth frame of raw data. Training an ML model to detect a velocity of an object when ground truth data for training is sparse may comprise training the ML model to predict a future position of the object based at least in part on image, radar, and/or lidar data (e.g., for which no ground truth may be available). The ML model may be altered based at least in part on a difference between ground truth data associated with a future time and the future position.

Abstract: Techniques are disclosed for executing a data processing pipeline. The techniques may include receiving a job at a data pipeline queue, setting up one or more distributed processing environments, and allocating the job to one of the distributed processing environments. The techniques may further include receiving the allocated job at a job queue within the distributed processing environment, increasing a priority level of the job, and executing the job within the distributed processing environment. The techniques can further include providing a retry pipeline at the data processing pipeline, and re-executing the job at a stage following a failure of at least one of its components. The techniques may decrement the retry budget as the job is re-executed.

Abstract: Techniques associated with generating simulation scenarios for simulating a vehicle controller are discussed herein. Log data may include sensor data captured by sensors of a vehicle. The log data may represent objects in an environment. Objects may be associated with a region of a discretized representation of the environment relative to the vehicle. Specific states of objects (relative position in a region type, velocity, classification, size, etc.) may represent an instance of an occupation. Log data can be aggregated based on similar region type and/or object state. A statistical model over object states can be determined for each region type and can later be sampled to determine simulation parameters. A simulation scenario can be generated based on the simulation parameters, and a vehicle controller can be evaluated based on the simulation scenario.

Abstract: Techniques are disclosed for a combined machine learned (ML) model that may generate a track confidence metric associated with a track and/or a classification of an object. Techniques may include obtaining a track. The track, which may include object detections from one or more sensor data types and/or pipelines, may be input into a machine-learning (ML) model. The model may output a track confidence metric and a classification. In some examples, if the track confidence metric does not satisfy a threshold, the ML model may cause the suppression of the output of the track to a planning component of an autonomous vehicle.

Abstract: A seat for a vehicle may include a seatback including one or more inflatable bladders that may be configured to provide adjustable lumbar support and to deploy in response to a triggering event signal indicative of a change of velocity, collision, or predicted collision event. One or more inflatable bladders may be configured to be independently controllable and configured to provide adjustable lumbar support at a region of the seatback during normal vehicle operation. Moreover, when at least a portion of the back of an occupant of the seat pushes against a front surface of the seatback due to the event, the one or more inflatable bladders may be configured to compress at least partially a comfort foam or comfort material in the seatback and expedite engagement of the energy absorbing material with the occupant's back.

Abstract: Techniques for determining a safety metric associated with a vehicle controller are discussed herein. To determine whether a complex system (which may be uninspectable) is able to operate safely, various operating regimes (scenarios) can be identified based on operating data and associated with a scenario parameter to be adjusted. To validate safe operation of such a system, a scenario may be identified for inspection. Error metrics of a subsystem of the system can be quantified. The error metrics, in addition to stochastic errors of other systems/subsystems can be introduced to the scenario. The scenario parameter may also be perturbed. Any multitude of such perturbations can be instantiated in a simulation to test, for example, a vehicle controller. A safety metric associated with the vehicle controller can be determined based on the simulation, as well as causes for any failures.

Abstract: A door interface system for a vehicle door includes a sensor and a visual indicator. The sensor may be a proximity sensor and may be positioned proximate to the visual indicator such that it will detect an object proximate to the proximity sensor. The visual indicator may convey the position of the sensor and/or a status of the vehicle door. The door interface system is configured to control the vehicle door based at least in part on detecting an object proximate the visual indicator.

Abstract: Techniques for controlling a vehicle based on height data and/or classification data being determined utilizing multi-channel image data are discussed herein. The vehicle can capture lidar data as it traverses an environment. The lidar data can be associated with a voxel space as three-dimensional data. Semantic information can be determined and associated with the lidar data and/or the three-dimensional voxel space. A multi-channel input image can be determined based on the three-dimensional voxel space and input into a machine learned (ML) model. The ML model can output data to determine height data and/or classification data associated with a ground surface of the environment. The height data and/or classification data can be utilized to determine a mesh associated with the ground surface. The mesh can be used to control the vehicle and/or determine additional objects proximate the vehicle.

Abstract: A sensor pod system includes one or more sensor pods with a plurality of sensors configured to collect data from an environment. A sensor pod may have an effective field of view created by individual sensors with overlapping fields of view. The sensor pod system may include sensors of different types and modalities. Sensor pods of the sensor pod system may be modularly disposed on a vehicle, for example, an autonomous vehicle to collect and provide data of the environment during operation of the vehicle. The sensor pods may be disposed at elevated locations around the vehicle to reduce obstacles within the sensor pods fields of view.

Abstract: Techniques for diagnosing and disambiguating a bias in one or more axels of a bidirectional four-wheel drive vehicle. In some examples, the diagnostics system may diagnose and disambiguate the bias in the two axles by collecting data related to the movement of the vehicle while the vehicle traverses a zero curvature path in a forward direction. The diagnostics system may then determine a bias in an axle acting as the rear axle based on the collected data. The bias in the axle acting as the front axle may then be determined based on the collected data and the bias in the rear axle.

Abstract: An occupant protection system for a vehicle may include an expandable curtain and/or an expandable bladder configured to be expanded from a stowed state to a deployed state. The system may also include a transverse ceiling trim panel and one or more side ceiling trim panels configured to be coupled to a ceiling of the vehicle, with the transverse ceiling trim panel extending substantially transversely with respect to the one or more side ceiling trim panels. Portions of the ceiling trim panels may be configured to deflect and allow expansion of the expandable curtain and/or the expandable bladder to the deployed state. The system may also include a deployment controller and one or more inflators configured to cause deployment of the expandable curtain at a first time and deployment of the expandable bladder at a second time after the first time.

Abstract: Motion can be induced at a vehicle, e.g., by actuating components of an active suspension system, and first sensor data and second sensor data representing an environment of the vehicle can be captured at a first position and a second position, respectively, resulting from the induced motion. A second sensor can determine motion information associated with the first position and the second position. Calibration information about the sensor, the first sensor data, and the motion information can be used to determine an expectation of sensor data at the second position. A calibration error can be the difference between the second sensor data and the expected sensor data.

Abstract: Techniques for determining information associated with sounds detected in an environment based on audio data and map data or perception data are discussed herein. A vehicle can use map data and/or perception data to distinguish between multiple audio signals or sounds. A direct source of sound can be distinguished from a reflected source of sound by determining a direction of arrival of sounds and which objects the directions of arrival are associated with in the environment. A reflected sound can be received without receiving a direct sound. Based on the reflected sound and map data or perception data, characteristics of sound in an occluded region of the environment may be determined and used to control the vehicle.

Abstract: A vehicle can include various sensors to detect objects in an environment. Sensor data can be captured by a perception system in a vehicle and represented in a voxel space. Operations may include analyzing the data from a top-down perspective. From this perspective, techniques can associate and generate masks that represent objects in the voxel space. Through manipulation of the regions of the masks, the sensor data and/or voxels associated with the masks can be clustered or otherwise grouped to segment data associated with the objects.

Abstract: Techniques relating to multi-passenger interaction via display(s) of a vehicle are described. In an example, sensor data is received from a sensor associated with an interior of a vehicle. Based at least partly on the sensor data, a first occupancy state associated with a first seat of the vehicle and a second occupancy state associated with a second seat of the vehicle can be determined. Content to present via displays associated with the first seat and/or the second seat can be determined based at least in part on the first occupancy state and the second occupancy state and can be presented via each respective display.

Abstract: A lidar photosensor amplification circuit may include light sensors; amplifiers corresponding respectively to the light sensors, in a powered-on state, to amplify output signals of the respective light sensors as amplified outputs; and switches corresponding respectively to the amplifiers, where individual switches may be controlled to pass a respective amplified output in a closed state or disconnect the amplified output in an open state. The lidar photosensor amplification circuit may be controlled by a controller according to timing rules that conserve power supplied to photosensor amplification circuit, reduce heat produced by the light sensor board, and do not aggravate cross-talk between sensors. The timing rules include staging an amplifier for a staging time before the corresponding light sensor is to be read, closing a switch after the staging time has passed, and powering down the amplifier and opening the switch after a reading time passes.

Abstract: Techniques described herein are directed to comparing, using a machine-trained model, neural network activations associated with data representing a simulated environment and activations associated with data representing real environment to determine whether the simulated environment is causes similar responses by the neural network, e.g., a detector. If the simulated environment and the real environment do not activate the same way (e.g., the variation between neural network activations of real and simulated data meets or exceeds a threshold), techniques described herein are directed to modifying parameters of the simulated environment to generate a modified simulated environment that more closely resembles the real environment.