Abstract: Techniques are described for cleaning sensors of a vehicle. An attribute of the surrounding environment (e.g., type of precipitation, intensity of precipitation, etc.) is determined along with orientations and/or attributes of the sensors (e.g. fields-of-view, focal lengths, spectral ranges, etc). Cleaning frequencies are then be determined based, at least in part, on the attribute of the environment together with the orientations and/or attributes of the sensors. Cleaning elements are then activated at the determined frequencies to clean the sensors. In some examples, the cleaning frequencies may be determined based additionally on an attribute of travel of the vehicle (e.g. direction of travel of the vehicle, speed of the vehicle, etc.).

Abstract: Techniques for determining a condition of a component of a vehicle and/or adjusting a configuration of a vehicle based on a condition of a component of the vehicle are discussed herein. The vehicle can receive vehicle component data and use the vehicle component data to determine a condition of one or more vehicle components. The vehicle can use a sensor, such as an image sensor, to capture or otherwise determine the vehicle component data. The condition of the component(s) of the vehicle may be determined using a machine learned model trained to determine condition of the component from image data. The vehicle can determine vehicle configuration data based on the condition. The vehicle configuration data may include an instruction to navigate to a maintenance facility, to remove the vehicle from service, constraining operation of the vehicle, etc.

Abstract: Techniques for managing data associated with different versions of components over time by a computing system are discussed herein. The computing system can generate, store, and/or delete data associated with different versions of components having a hierarchical relationship such that data can be fed from one component to another component over time). The computing system can receive log data associated with one or more components and determine regeneration data for different components of a vehicle computing device over time. The computing system can include a user interface for receiving file generation requests to test, validate, or verify functionality of a particular component of the vehicle computing device at different instances of time.

Abstract: A vehicle computing system may implement techniques to determine whether two objects in an environment are related as an articulated object. The techniques may include applying heuristics and algorithms to object representations (e.g., bounding boxes) to determine whether two objects are related as a single object with two portions that articulate relative to each other. The techniques may include predicting future states of the articulated object in the environment. One or more model(s) may be used to determine presence of the articulated object and/or predict motion of the articulated object in the future. Based on the presence and/or motion of the articulated object, the vehicle computing system may control operation of the vehicle.

Abstract: Automating reinforcement learning for autonomous vehicles may include assigning a probability with a scenario and varying that probability based at least in part on changes in performance by the autonomous vehicle associated with that scenario. The amount of time and computational bandwidth required to train a machine-learned component of an autonomous vehicle and the accuracy of the machine-learned component may be improved by determining a reward for performance of the autonomous vehicle in a scenario based at least in part on an severity metric. The impact severity metric may be determined based at least in part on a velocity, angle, and/or interaction area associated with the impact.

Abstract: Techniques for collision avoidance using an object contour are discussed. A trajectory associated with a vehicle may be received. Sensor data can be received from a sensor associated with the vehicle. A bounding contour may be determined and associated with an object represented in the sensor data. Based on the trajectory, a simulated position of the vehicle can be determined. Additionally, a predicted position of the bounding contour can be determined. Based on the simulated position of the vehicle and the predicted position of the bounding contour, a distance between the vehicle and the object may be determined. An action can be performed based on the distance between the vehicle and the object.

Abstract: Techniques for a perception system of a vehicle that can detect and track objects in an environment are described herein. The perception system may include a machine-learned model that includes one or more different portions, such as different components, subprocesses, or the like. In some instances, the techniques may include training the machine-learned model end-to-end such that outputs of a first portion of the machine-learned model are tailored for use as inputs to another portion of the machine-learned model. Additionally, or alternatively, the perception system described herein may utilize temporal data to track objects in the environment of the vehicle and associate tracking data with specific objects in the environment detected by the machine-learned model. That is, the architecture of the machine-learned model may include both a detection portion and a tracking portion in the same loop.

Abstract: An active suspension control system for a vehicle includes a mathematical model based on a modal expansion of the vehicle. Model parameters of the vehicle can be extracted from the modal expansion using sensor data generated on the vehicle, e.g., on demand and/or in real time. The model parameters and the modal expansion can be used to determine a vehicle state, predict future vehicle states, and control aspects of an active suspension system based on the predicted future vehicle states. The model parameters may also be used to update the mathematical model, e.g., to account for component wear over time, and/or to detect anomalies or defects in the active suspension system.

Abstract: Techniques for generating more accurate determinations of object proximity by using vectors in data structures based on vehicle sensor data are disclosed. Vectors reflecting a distance and direction to a nearest object edge from a reference point in a data structure are used to determine a distance and direction from a point of interest in an environment to a nearest surface. In some examples, a weighted average query point response vector is determined using the determined distance vectors of cells neighboring the cell in which the point of interest is located and nearest to the same object as the query point, providing a more accurate estimate of the distance to the nearest object from the point of interest.

Abstract: Techniques are discussed herein for controlling autonomous vehicles within a driving environment, including generating and using bounding contours associated with objects detected in the environment. Image data may be captured and analyzed to identify and/or classify objects within the environment. Image-based and/or lidar-based techniques may be used to determine depth data associated with the objects, and a bounding contour may be determined based on the object boundaries and associated depth data. An autonomous vehicle may use the bounding contours of objects within the environment to classify the objects, predict the positions, poses, and trajectories of the objects, and determine trajectories and perform other vehicle control actions while safely navigating the environment.

Abstract: Techniques associated with detecting non-deterministic behavior with a component and/or subcomponents of an autonomous vehicle are discussed herein. In some cases, a simulation system may be configured to simulate operations of the autonomous vehicle and to detect changes in behavior between instances and with respect to log data or expected results. The simulation system may flag or otherwise identify components and/or subcomponents responsive to detecting potentially non-deterministic behavior.

Abstract: Techniques for determining gaps for performing lane change operations are described. A first region in an environment of a vehicle can be determined. The first region can be associated with a first time period through which the vehicle is unable to travel and can correspond to a constraint space. A second region of the environment can be determined. The second region can be associated with a second time period and can correspond to a configuration space. A gap in the environment can be determined based on a portion of the configuration space that is exclusive of the constraint space. A trajectory can be determined based on the gap. The trajectory can be associated with performing a lane change operation and can be associated with a cost. The vehicle can be controlled to perform the lane change operation based at least in part on the trajectory and the cost.

Abstract: Techniques and systems for testing, evaluating and/or otherwise assessing the operation of vehicles are discussed herein. Aspects relate to a system and method comprising receiving video data comprising a video stream derived from a camera during a vehicle run and being in a first format; storing part of the video stream in the first format; transcoding part of the video stream from the first format into a second format; storing, in a cache, the part of the video stream as video segments in the second format; receiving a request for video data associated with a point in time, wherein: if video segments in the second format do not correspond to the point in time, transcoding part of the video stream in the first format to the second format to create further second video segments in the second format; transmitting and storing, in the cache, the further second video segments.

Abstract: Techniques related to generating map data explicitly indicating a total drivable surface, which may include multiple types of drivable surfaces, are described herein. For instance, a given portion of a map may include map data indicating a combination of various drivable surfaces, such as road segments, lane properties, intersections, parking areas, shoulders, driveways, etc. Examples include joining these different types of drivable surfaces into combined map data that explicitly indicates a total drivable surface, such as a perimeter boundary indicating or representing a transition from a drivable surface to a non-drivable surface. The map data indicating the total drivable surface may be searched to determine information related to a drivable surface boundary, such as location and type. This boundary information may be used in various contexts, such as when planning a trajectory or remotely controlling a vehicle.

Abstract: A vehicle computing system may implement techniques to control a vehicle to avoid collisions between the vehicle and an object in an environment in which a vehicle path and an object path merge. The techniques may include determining an initial merge location associated with the vehicle path and the object path and a final merge location. The final merge location may represent a location at which the vehicle is proximate to and ahead of the object. The vehicle computing system may determine whether the vehicle may proceed to the final merge location and merge with the object without the occurrence of a collision. The vehicle computing system may determine to maintain a vehicle trajectory or modify the vehicle to trajectory to yield to the object based on a determination of whether the vehicle may proceed without the occurrence of the collision.

Abstract: Techniques for enabling operation of a vehicle after detecting a fault associated with a component of the vehicle are described herein. A vehicle computing system can monitor components of the vehicle and identify a fault associated with a component. Based on the fault, the vehicle computing system can cause the vehicle to cease (e.g., stop) or limit operation in an environment. The vehicle computing system can receive a request to override the fault and continue operation of the vehicle in a recovery mode with limited parameters (e.g., limited speed, acceleration, etc.). Based on a determination that the request to override the fault is valid, the vehicle computing system can cause the vehicle to be controlled, by an operator or an autonomous controller, according to parameters of the recovery mode.

Abstract: A door interface system for a vehicle door includes a sensor and a visual indicator. The sensor may be a proximity sensor and may be positioned proximate to the visual indicator such that it will detect an object proximate to the proximity sensor. The visual indicator may convey the position of the sensor and/or a status of the vehicle door. The door interface system is configured to control the vehicle door based at least in part on detecting an object proximate the visual indicator.

Abstract: Techniques are disclosed for component verification for complex systems. The techniques may include receiving log data, obtaining ground truth data based on the log data and determining an outcome at least in part by simulating a prediction by a prediction component based on the log data and the ground truth data. The techniques may further include simulating a second prediction by the prediction component based on the ground truth data, determining whether the second prediction resulted in the negative outcome of the scenario and determining the disengagement event is attributable to a perception component of the autonomous operation system at least partly in response to determining the second prediction based on the ground truth data did not result in the negative outcome.

Abstract: Techniques for aggregating costs associated with one or more heat maps to control a vehicle in an environment are discussed herein. A vehicle computing device can implement a model to determine heat maps and respective cost information for different features of the environment based on sensor data. The vehicle computing device can output a planned trajectory for the vehicle based on combining the heat maps. The techniques can also include determining a rationalization or root cause detailing reasons why the planned trajectory was determined.

Abstract: A computer implemented method. Includes: obtaining, from a sensor system onboard a vehicle, a plurality of hypotheses for an orientation of an object in a vicinity of the vehicle, relative to a coordinate system of the vehicle; estimating, based on the plurality of hypotheses, values of a probability distribution for the orientation of the object at a plurality of candidate orientations; and estimating, based at least in part on the estimated values of the probability distribution at the plurality of candidate orientations, the orientation of the object relative to the coordinate system.