Abstract: Systems and techniques for determining adverse traction conditions currently and/or likely to be being experienced by a vehicle and mitigating such conditions are described. Using sensor data, a vehicle computing device may determine an adverse traction conditions and various attributes of the condition. The vehicle computing device may adjust individual wheel configurations to address this condition, for example, by increasing or decreasing cross-corner wheels, rapidly and repeatedly increasing and decreasing particular wheels, and/or steering wheel individual to increase traction overall and/or reduce the speed of the vehicle (e.g., without using brakes). When normal traction conditions are recognized, the vehicle computing system may return the wheel suspension and steering configurations back to normal.

Abstract: Predicting a future trajectory of an object may comprise using a machine-learned model architecture to iteratively predict a series of predicted poses (positions and orientations) of the object over time. The machine-learned model may condition these predictions on a lane orientation of nearest target pose to a predicted pose or last predicted pose of the object.

Abstract: A vehicle trajectory may be evaluated based on a relative probability distribution associated with a vehicle and an object in the vehicle environment and/or based on velocities associated with the vehicle and/or the object. The relative probability distribution may be determined based on a vehicle trajectory and/or a predicted object trajectory. A collision probability and/or a collision severity may be determined based on the relative probability distribution and/or the velocities. The collision probability and/or severity may be used to determine whether the vehicle trajectory is safe and/or to inform the driving behavior of the vehicle.

Abstract: Techniques are described herein for safely transitioning a vehicle from a first trajectory to a second trajectory. In response to a safety event, for example, a vehicle may switch from being controlled in accordance with a nominal planner output (a planned trajectory) to a safety, or stopping, trajectory. In some examples, in response to the vehicle overcoming the safety event, a safe and smooth return to a nominal planning system output trajectory (an updated planned trajectory) may be determined and used to cause the vehicle to continue along to a previously determined destination.

Abstract: Techniques for estimating a ground profile for an environment associated with a vehicle are described herein. Sensor data associated with a vehicle may be used to determine sensor data points associated with a path of the vehicle. Using a first metric and a second metric, such as a maximum slope and minimum slope based on the position of the vehicle and/or a previously-determined ground point, a region (e.g., search window) may be determined, and based on the region, a subset of sensor data points included in the region. The subset of sensor data points may be used to determine a ground point. In some instances, the ground point may be determined based on an average height of the sensor data points. A ground profile may be determined that at least partially includes the ground point.

Abstract: Techniques for generating a vehicle trajectory usable for for controlling an autonomous vehicle in an environment are described herein. For example, the techniques may include a computing device implementing a machine learned model and a statistical model in parallel in a tree search of a tree structure to predict a trajectory of the autonomous vehicle in the environment. The tree structure can process a latent representation and a non-latent representation in parallel to determine potential actions and/or trajectories for the autonomous vehicle. The tree structure of the computing device can output a trajectory for controlling the autonomous vehicle in the environment. Depending on examples, the environment is a real-world environment or a simulated environment.

Abstract: The techniques described herein relate to controlling and/or influencing the routes driven by vehicles, autonomous or otherwise, such as vehicles in a fleet of vehicles managed by a fleet management system. In some cases, the techniques described herein relate to centrally generating road weights using a remote system such as a fleet management system and providing those pre-calculated weights to vehicles to simplify onboard route planning. Rather than transmitting large amounts of raw traffic, road condition, and other data to each vehicle, the remote system pre-processes the data into condensed road weights optimized for route planning. This architecture provides various technical advantages such as reduced data transmission, decreased computational load on vehicles, and decentralized control of fleet-wide routing.

Abstract: A vehicle includes a sensor assembly and a cleaning mechanism for cleaning the sensor assembly. The sensor assembly has a sensor window and a sensor residing within or behind the sensor window. The sensor has a field of view (FoV) for sensing an environment of the vehicle. The FoV includes a first portion directed towards the vehicle and a second portion directed away from the vehicle for sensing an environment of the vehicle. The cleaning mechanism has a rail coupled to the sensor assembly at a location inside the first portion of the FoV, a carriage operably coupled to the rail, an actuator coupled to the carriage, and a circular-shaped wiper coupled to the carriage. The circular-shaped wiper encircles the sensor window.

Abstract: Techniques for capturing videos of content displayed and/or generated by a web-based application are described herein. The videos may be captured by automating screenshots of the content within a headless browser at a fixed frame interval. The headless browser may be used to automate control of the web-based application to generate the content and capture screenshots of the content as the content would appear if the web-based application was being accessed through a traditional web browser graphical user interface. The screenshots may be captured at specific frame intervals while the content is being generated. Additionally, the headless browser or a server executing headless browser may wait for the web-based application to load individual frames of the content before capturing screenshots. The captured screenshots may then be combined to generate a video of the content.

Abstract: Techniques for generating a driving surface cost landscape for determining costs for vehicle positions in an environment are described herein. A planning component within a vehicle may determine a non-preferred surface associated with a type of non-preferred area of an environment along a route of a vehicle, determine a preferred surface associated with a preferred area of the environment, determine an adjusted non-preferred surface by removing an overlapping area of the non-preferred surface that overlaps the preferred surface and determine a cost associated with a vehicle position based at least on the preferred surface and the adjusted non-preferred surface. The planning component may then determine a control trajectory for the autonomous vehicle based at least in part on the cost associated with the vehicle position.

Abstract: Techniques for object detection based on sensor data are discussed herein. In some cases, the techniques described herein include receiving lidar data and radar data associated with an environment of a vehicle, wherein the lidar data comprises a first set of lidar points associated with a first portion of the environment. The techniques further include determining, based on the radar data, a first subset of the first set that is associated with a dynamic movement pattern; determining a first ratio based on the first subset and the first set. Further, the techniques include, based at least in part on determining that the first ratio exceeds a threshold, determining that the first portion is associated with a dynamic object in the environment. In some examples, the vehicle can be controlled based on the dynamic object.

Abstract: Systems and techniques for whether to associate predicted object trajectories of objects detected in an environment with candidate vehicle trajectories are described. A vehicle computing system may determine a vehicle occupancy state of a vehicle along a path of travel, determine, based at least in part on the predicted object trajectory, an object occupancy state of the object along the predicted object trajectory, determine a state interaction score for the vehicle occupancy state and the object occupancy state, determine, based at least in part on the state interaction score, a relevancy score representing a relevance of the predicted object trajectory to a candidate trajectory, and control the vehicle based at least in part on the relevancy score.

Abstract: Techniques are discussed herein for generating trajectories for controlling motion and/or other behaviors of vehicles in complex driving environments. In certain examples, a search algorithm may be used to determine and evaluate a set of possible candidate actions for a vehicle, including candidate actions based on a predetermined exploration policy and additional candidate actions based on machine learned models that output predicted behaviors for the vehicle based on the current driving environment. Costs associated the various candidate actions may be evaluated based on state transition costs and/or future state predictions of the driving environment. Certain examples may include a tree search using a combination of predetermined heuristic candidate actions and adaptive-learning candidate actions at various nodes within a tree structure representing a driving route from a current vehicle state to an intended end state.

Abstract: Vehicle computing systems may receive and fuse long-wave infrared data with radar data to detect and track objects in low-visibility driving environments. In some examples, radar data including position and velocity data may be projected over infrared image data to detect, classify, and track infrared-emitting objects. Machine-learned transformer models with attention also may be trained to output object detections based on combined infrared and radar data. The fusion of infrared and radar data may be used individual and/or may be synchronized with other sensor modalities. In some examples, the fusion and analysis of infrared and radar data may be used in specific low-visibility driving environments, using low-light, fog, and shadowed areas of the environment, to detect and track infrared-emitting and/or moving objects such as pedestrians and animals that may be obscured from detection by other sensor modalities.

Abstract: Techniques for detecting and determining a density for a non-impeding object based on correlation between radar and lidar returns are described herein. The techniques provide for receiving lidar data and radar data from a vehicle system operating in an environment. Portions of the lidar data and radar data are determined based on being associated with moving objects in the environment. The portions are then correlated to determine a similarity between the radar and lidar data. The similarity may then be used to determine an indication and density of the non-impeding object in the environment and cause the vehicle to operate within the environment accordingly.

Abstract: There is provided a the system configured to receive a pretrained model configured to output a plurality of representations based at least in part on an input comprising a graph representing an environment through which a vehicle is traversing, a node of the graph comprising, as an embedding, a vector encoding; receive a task comprising: a function to append to the pretrained model, a dataset, a loss function; and update, based at least in part on the task, the pretrained model to generate an updated machine learned model; and deploy the updated machine learned model to a vehicle computing system associated with a vehicle configured to be controlled based at least in part on an output of the updated machine learned model.

Abstract: Techniques for verifying operation of testing mechanisms are discussed herein. For example, a computing device can implement a testing component to test performance of a vehicle controller configured to control operation of a vehicle. The testing component can apply one or more heuristics and/or machine learned models to initiate, perform, and analyze tests that measure whether an instruction update improves performance of the vehicle controller and/or whether the instruction update causes performance of another component to degrade. The testing component can be used to validate the instruction update in a testing environment prior to transmitting the instruction update to a vehicle controller coupled to a vehicle in a real-world environment.

Abstract: A machine-learned architecture may predict a set of spatially-diverse paths that an object may take in the future. The paths generated by this architecture may be time-invariant (e.g., not identifying a time at which the object may occupy a position along one of these paths) but can be used by a second machine-learned model to predict progress in time along these paths. This segregation of the spatial paths and progress in time along the paths improves the accuracy of the ultimate prediction and better captures rare object behavior.

Abstract: A transformer-based machine-learned model may predict object behavior by using cross-attention between dynamically-sized patches of a top-down representation of the environment and an object location and/or previous behavior. A patch's size may be based at least in part on map data and/or sensor data and a portion of the top-down representation associated with an area of the environment that is outside a roadway may be excluded from patch generation.

Abstract: Techniques for interacting with a remote vehicle using tracked gestures are described herein. A vehicle may receive sensor data, which may be combined to generate a representation of the vehicle traversing the environment. The representation may include features of the environment, such as people and/or objects. The representation may be displayed at a user interface. In some instances, the user interface may be associated with a wearable computing device and may be associated with a user, such as a remote operator. The wearable computing device may receive natural user input data of a user. The computing device may then determine an action associated with the vehicle based on the natural user input and transmit information to the vehicle to perform the action.