Abstract: A vehicle computing system may implement techniques to determine an action for a vehicle to perform based on a cost associated therewith. The cost may be based on a detected object (e.g., another vehicle, bicyclist, pedestrian, etc.) operating in the environment and/or a possible object associated with an occluded region (e.g., a blocked area in which an object may be located). The vehicle computing system may determine two or more actions the vehicle could take with respect to the detected object and/or the occluded region and may generate a simulation associated with each action. The vehicle computing system may run the simulation associated with each action to determine a safety cost, a progress cost, a comfort cost, an operational rules cost, and/or an occlusion cost associated with each action. The vehicle computing system may select the action for the vehicle to perform based on an optimal cost being associated therewith.

Abstract: Techniques are discussed for controlling a vehicle, such as an autonomous vehicle, based on occluded areas in an environment. An occluded area can represent areas where sensors of the vehicle are unable to sense portions of the environment due to obstruction by another object. An occlusion grid representing the occluded area can be stored as map data or can be dynamically generated. An occlusion grid can include occlusion fields, which represent discrete two- or three-dimensional areas of driveable environment. An occlusion field can indicate an occlusion state and an occupancy state, determined using LIDAR data and/or image data captured by the vehicle. An occupancy state of an occlusion field can be determined by ray casting LIDAR data or by projecting an occlusion field into segmented image data. The vehicle can be controlled to traverse the environment when a sufficient portion of the occlusion grid is visible and unoccupied.

Abstract: There is provided a method comprising: determining an occupancy status of a first region of a vehicle; determining, based at least in part on the occupancy status, a first climate control setting for the first region; controlling a climate control system of the vehicle to adjust a climate of the first region according to the first climate control setting; determining that a second region of the vehicle is unoccupied, wherein the second region is fluidly connected to the first region; determining a second climate control setting, wherein the second climate control setting is based at least in part on the occupancy status of the first region and characteristic data associated with a predicted potential change in occupancy status for the second region; and controlling the climate control system to adjust a climate of the second region according to the second climate control setting.

Abstract: Techniques associated with generating and maintaining sparse geographic and map data. In some cases, the system may maintain a factor graph comprising a plurality of nodes. In some cases, the nodes may comprise pose data and sensor data associated with an autonomous vehicle at the geographic position represented by the node. The nodes may be linked based on shared trajectories and shared sensor data.

Abstract: Techniques for maintaining and synchronizing data is a processing pipeline data between multiple processing units to improve a system latency are described herein. For example, the techniques may include determining, in response to an invocation of vision processing on first vision data stored in a first memory range in a first memory associated with a central processing unit (CPU), that second vision data stored in a second memory range in a second memory associated with a graphic processing unit (GPU) is a modified copy of the first vision data. The second vision data may be obtained using a non-blocking operation from the second memory range. The first vision data stored in the first memory range may be replaced with the second vision data obtained from the second memory range. The vision processing may then be performed using the second vision data stored in the first memory.

Abstract: Techniques for predicting and avoiding collisions with objects detected in an environment based on sensor data are discussed herein. Sensors of a vehicle may detect one or more objects in the environment. A model may determine intersection values indicative of probabilities that the object will follow different paths that intersect with a planned path of the vehicle. A vehicle may receive one or more intersection values from a model usable by a computing device to control the vehicle.

Abstract: A system may include one or more processors configured to receive a plurality of images representing an environment. The images may include image data generated by an image capture device. The processors may also be configured to transmit the image data to an image segmentation network configured to segment the images. The processors may also be configured to receive sensor data associated with the environment including sensor data generated by a sensor of a type different than an image capture device. The processors may be configured to associate the sensor data with segmented images to create a training dataset. The processors may be configured to transmit the training dataset to a machine learning network configured to run a sensor data segmentation model, and train the sensor data segmentation model using the training dataset, such that the sensor data segmentation model is configured to segment sensor data.

Abstract: Techniques for generating trajectories and drivable areas for navigating a vehicle in an environment are discussed herein. The techniques can include receiving a trajectory representing an initial trajectory for a vehicle, such as an autonomous vehicle, to traverse the environment in a drivable area. A location can be determined along the trajectory. A cost associated with the location can determined and can be evaluated with respect to a cost threshold. Further, the techniques can include determining, based at least in part on the cost meeting or exceeding the cost threshold, an action associated with the location, and controlling the autonomous vehicle to traverse the environment based at least in part on the action.

Abstract: Techniques are described for using variables associated with vehicle wheels (e.g., linear velocity at a wheel and orientation of the wheel) to estimate velocity of a vehicle during a turn maneuver. In examples of the disclosure, in association with one or more wheels, a wheel orientation during the maneuver and a linear speed during the maneuver may be determined, and well as a yaw rate (e.g., from an inertial measurement unit, gyroscope, etc.) of the vehicle. Examples of the present disclosure include, based on the variables associated with the wheel(s) and the yaw rate associated with the turn maneuver, estimating a vehicle velocity, which may be used by various downstream components, such as to determine or update a pose of a vehicle as part of a localization operation.

Abstract: Techniques for determining a warped occupancy grid fit to a vehicle trajectory are discussed herein. In some examples, a portion of memory may be allocated to an occupancy grid. Further, a warped occupancy grid can be warped and associated with an environment that an autonomous vehicle is traversing according to a trajectory and/or throughway. A transformation maybe be determined between the warped occupancy grid and the memory allocated to the occupancy grid. Sensor data can be received from a sensor associated with the autonomous vehicle and may be associated with the warped occupancy grid and stored in the occupancy grid. The autonomous vehicle may be controlled according to the warped occupancy grid by identifying sensor data returns in cells of the warped occupancy grid that may indicate a detection of an object in a path of travel of the vehicle.

Abstract: Techniques for using ball joint sensor data to determine conditions relevant to a vehicle are described in this disclosure. For example, in one example, the ball joint sensor data may be used to determine estimated steering data. The estimated steering data may be directly used to navigate through an environment, such as by the vehicle relying on the estimated steering data when planning, tracking, or executing a driving maneuver. Also, the estimated steering data may be used to verify the reliability of other steering sensor data used to navigate through the environment.

Abstract: A location of an occupant within a vehicle and/or an activity engaged in by the occupant may be determined. Based on the location and/or the activity, a point of interest associated with the occupant and/or the vehicle may be determined. One or more systems of the vehicle, such as the steering and/or suspension, may be controlled to minimize acceleration associated with the point of interest, thereby increasing a comfort of the occupant. In instances where the vehicle includes more than one occupant, the vehicle may be adjusted to accommodate the multiple occupants.

Abstract: Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. In particular, a method may include monitoring a fleet of vehicles, at least one of which is configured to autonomously transit from a first geographic region to a second geographic region, detecting data indicating an event associated with the vehicle having a calculated confidence level, receiving data representing a subset of candidate trajectories responsive to detecting the event, which is associated with a planned path for the vehicle, identifying guidance data to select from one or more of the candidate trajectories as a guided trajectory, receiving data representing a selection of a candidate trajectory, and transmitting the selection of the candidate trajectory as of the guided trajectory to the vehicle.

Abstract: A vehicle computing system may implement techniques to determine attributes (or intent) of an object detected by a vehicle operating in the environment. The techniques may include determining a set of features with respect to a detected object by a first model and determining, by a second model and based on the set of features, one or more attributes of the object. The first model and the second model may be configured to process at least one image frame to determine the one or more attributes of the object. A model may receive sensor data as an input, and output features and/or an attribute for the detected object. Based on the attribute(s) of the object, a vehicle computing system may control operation of the vehicle.

Abstract: Techniques for using ball joint sensor data to determine conditions relevant to a vehicle are described in this disclosure. For example, in one example, the ball joint sensor data may be used to determine a ride height at a portion of the vehicle, which may be used to determine roll data and/or pitch data. The ride height, roll data, and/or pitch data may be directly used to navigate through an environment, such as by the vehicle relying on the data when interpreting sensor data or planning driving operations. Also, the ride height, roll data, and/or pitch data may be used to verify the reliability of other sensor data used to navigate through the environment.

Abstract: A teleoperations system that collaboratively works with an autonomous vehicle guidance system to generate a path for controlling the autonomous vehicle may comprise generating one or more trajectories at the teleoperations system based at least in part on environment data received from the autonomous vehicle and presenting the one or more trajectories to a teleoperator (e.g., a human user, machine-learned model, or artificial intelligence component). A selection of one of the trajectories may be received at the teleoperations system and transmitted to the autonomous vehicle. The one or more trajectories may be generated at the teleoperations system and/or received from the autonomous vehicle. Regardless, the autonomous vehicle may generate a control trajectory based on the trajectory received from teleoperations, instead of merely implementing the trajectory from the teleoperations system.

Abstract: Training a model for detecting pedestrian objects is described. Annotated data can be received by a training component. In an example, the annotated data can include a first indication of a first object (e.g., a pedestrian) and a second indication of a second object (e.g., a pedestrian object), wherein the first object and the second object comprise a portion of a compound object (e.g., a pedestrian/pedestrian object system). In an example, a training component can train a model to determine an output associated with a second object based at least in part on the annotated data and the model can be transmitted to a vehicle configured to be controlled by output(s) of the model.

Abstract: The techniques discussed herein include modifying a Kalman filter to additionally include a loss component that dampens the effect measurements with large errors (or measurements indicating states that are rather different than the predicted state) have on the Kalman filter and, in particular, the updated uncertainty and/or updated prediction. In some examples, the techniques include scaling a Kalman gain based at least in part on a loss function that is based on the innovation determined by the Kalman filter. The techniques additionally or alternatively include a reformulation of a Kalman filter that ensures that the uncertainties determined by the Kalman filter remain symmetric and positive definite.

Abstract: A computer-implemented method. Includes obtaining pointwise data indicating, for a plurality of time steps, a pointwise measurement of a state of an object detected by an object detection system. Includes obtaining, from a runtime model, runtime data indicating, for the plurality of time steps, a runtime estimate of the state of the object. Includes processing, by a benchmark model, the pointwise data to determine, for the plurality of time steps, a benchmark estimate of the state of the object. Includes evaluating a metric measuring, for the plurality of time steps, a deviation between the runtime estimate and the benchmark estimate of the state of the object. Includes updating, based on the on the evaluation of the metric, the runtime model.

Abstract: Techniques associated with predicting behaviors and states of objects in a physical environment using thermal data.. In some cases, the system may be configured to determine heat signatures of individual features of an object and based on a combination of heat signatures determine a predicted behavior and/or a state of the object. The system may also utilize the thermal data to determine a confidence associated with predicted behavior and/or states of the object.