Abstract: Techniques for training a model for detecting objects in an environment are discussed herein. For example, techniques can include determining losses associated with spatial features of candidate bounding boxes output by a machine-learned (ML) model and utilizing the losses to train the ML model. Techniques may include determining candidate bounding box(es) associated with an object detected in an environment using the ML model and receiving a ground truth bounding box associated with the detected object. A yaw error loss may be determined by comparing yaw features of the candidate bounding box to the ground truth bounding box. The candidate bounding box may be axis aligned with respect to the ground truth bounding box and an intersection over union (IoU) loss may be determined based on an IoU between the axis aligned candidate bounding box and the ground truth bounding box. The ML model may be trained based on the losses.

Abstract: Techniques for evaluating and validating progress of training machine-learned models are described herein. The techniques may include storing, in a database, metric data associated with outputs from machine-learned models based on sensor data inputs. For instance, the metric data may include first metric data associated with a first bounding box output by a machine-learned model and second metric data associated with a second bounding box output by an updated version of the machine-learned model. The techniques also include a graphical user interface (GUI) for presenting visualizations of the metric data that improve the ability to evaluate the performance of a machine-learned model. In some examples, an indication of a request to evaluate the updated version of the machine-learned model may be received via the GUI. Based on the indication, the GUI may cause presentation of visualization(s) of difference(s) between first metric(s) of the first metric data and second metric(s) of the second metric data.

Abstract: The present disclosure is related to generating map data explicitly indicating a total drivable surface, which may include multiple types of drivable surfaces. For instance, a given portion of a map may include map data indicating a combination of various drivable surfaces, such as road segments, lane properties, intersections, parking areas, shoulders, driveways, etc. Examples of the present disclosure join these different types of drivable surfaces into combined map data that explicitly indicates a total drivable surface, such as a perimeter boundary indicating or representing a transition from a drivable surface to a non-drivable surface. The map data indicating the total drivable surface may be searched to determine information related to a drivable surface boundary, such as location and type. This boundary information may be used in various contexts, such as when planning a trajectory or remotely controlling a vehicle.

Abstract: Techniques for controlling a vehicle to select parking spaces are discussed herein. The vehicle may receive a request to travel to a destination and obtain information about a first set and a second set of parking spaces within a threshold distance of the destination. The vehicle may determine a first likelihood of availability associated with the first set of parking spaces and a second likelihood of availability associated with the second set of parking spaces. The vehicle may determine a first cost associated with a first route to the first set of parking spaces and a second cost associated with a second route to the second set of parking spaces by the cost function. Upon determining that the first cost is equal to or less than the second cost, the vehicle may navigate based on the first route and park within one of the first set of parking spaces.

Abstract: A vehicle computing system may implement techniques to determine relevance of objects detected in an environment to a vehicle operating in the environment. The techniques may include determining locations and trajectories associated with a detected object and generating simulations of movement (e.g., estimated states) of the detected object and the vehicle over a period of time. The vehicle computing system may compare locations, distances, and/or trajectories of the detected object and the vehicle in an estimated state to determine whether the detected object is relevant to the vehicle performing the action (e.g., impacts the vehicle's ability to safely perform the action). The relevance determination may be based in part on a semantic classification of the detected object.

Abstract: Vehicles may be associated with a variety of different types of events, including events associated with vehicle operations and/or events associated with vehicle passengers. The present disclosure is related to, when an event is detected, exchanging information with a vehicle passenger, such as via the passenger's mobile device and/or wearable device. In some instances, an event may be detected, and examples of the present disclosure may provide an application notification presenting various information. The notification may, in some examples, be configured to help the passenger locate the passenger device, to alert the passenger to the event, to provide instructions, and/or to provide a control interface for controlling a vehicle operation. In some examples, the notification may supersede operations of the passenger device, such as by being presented even if the device is in a locked state or has an unrelated application open and in the foreground.

Abstract: Tracking component decisions may comprise generating a data structure in association with an output determined by a component. This data structure, along with one or more data structures generated in association with other outputs generated by the same or different components of the vehicle, may be used to determine a trace that identifies component(s) that determined outputs that affected a particular component's generation of an output. The data structures and/or traces may be used to determine whether a component is the source of an error, a portion of the component that is the source of the error, unintended impacts to unmodified portions of components, among additional or alternate uses discussed herein.

Abstract: Techniques to provide guidance to a vehicle operating in an environment are discussed herein. For example, such techniques may include sending a request for assistance, receiving a reference trajectory, and causing the vehicle to determine a trajectory based on the reference trajectory. Data such as sensor data and vehicle state data may be sent from the vehicle to a remote computing device. The computing device outputs a user interface using the data and determines the reference trajectory based on receiving an input in the user interface. The techniques can send an indication of the reference trajectory to the vehicle for use in planning a trajectory for the vehicle. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the trajectory.

Abstract: This disclosure describes techniques for latency mitigation during inter-process communication based on pre-allocating and managing shared memory. During the startup phase (e.g., prior to run-time), processes of vehicle computing systems may request to engage in communication with other processes within the vehicle computing system. Based on the request, a memory manager may determine an amount of shared memory to pre-allocate. In such examples, the amount of shared memory may be determined based on a maximum number of messages that may be “in-flight” at any single instance during run-time, and a maximum size of a message. The memory manager may pre-allocate shared memory consistent with the determined amount, and transmit a key to the requesting process. In such examples, the key may enable the process to access the pre-allocated shared memory. After the startup phase has ended, the process may read and/or write messages to and from the shared memory.

Abstract: Wirelessly charging a vehicle battery may be performed using a plurality of charging coils. The offboard coils and the onboard coils may be spaced apart by different amounts, such that, when the vehicle is positioned in a vehicle charging space, at least one pair of an onboard coil and an offboard coil may still be substantially aligned for wireless charging, even when other coils are misaligned. The offboard coils may include sizes that are conducive to charge smaller capacity systems in some arrangements and are combinable to charge larger capacity systems in other arrangements. In addition, the use of relatively small coils may allow for flexibility in thermal management.

Abstract: Techniques and systems for testing, evaluating and/or otherwise assessing the operation of vehicles are discussed herein. Aspects relate to a system and method comprising receiving vehicle data derived from a vehicle data logging system, the vehicle data being associated with a vehicle run and comprising a plurality of data topics associated with a corresponding plurality of vehicle components and displaying the vehicle data with a timeline component of a user interface during playback of the vehicle run, the timeline component comprising a portion in the user interface and the portion being configurable with a start time and an end time. The vehicle data may comprise timing data for at least one of the data topics and the vehicle data may be displayed at points in time along the portion in the user interface corresponding to the timing data.

Abstract: Techniques for estimating vehicle poses are described herein. A pose may refer to the inertial pose or position of a vehicle. The pose of the vehicle may be used to determine a position of the vehicle in an environment, for determining a trajectory of the vehicle through the environment, or the like. In examples, a pose determination component of an autonomous vehicle can determine a pose at a frequency that is different from other systems on the vehicle and/or that is not synchronized with those systems. The techniques described herein allow for estimating poses at any measurement time, e.g., to determine a pose at a measurement time at which other data is measured, generated, or the like.

Abstract: Techniques are disclosed for implementing a convolutional neural network that determines an offset field for deforming a kernel to be used in a convolution. The offset field is temporally-based, at least in part, on data generated at an earlier time. Furthermore, techniques are disclosed for using sensor data to train a neural network to learn shapes or configurations of such deformed kernels. The temporal-based deformable convolutions may be used for object identification, object matching, object classification, segmentation, and/or object tracking, in various examples.

Abstract: The described techniques relate to modifying a trajectory of a vehicle, such as an autonomous vehicle, based on an overlap area associated with an object in the environment. In examples, map data may be used, in part, to generate an initial trajectory for an autonomous vehicle to follow through an environment. In some cases, a yield trajectory may be generated based on detection of the object, and the autonomous vehicle may evaluate a cost function to determine whether to execute the yield or follow the initial trajectory. In a similar manner, the autonomous vehicle may determine a merge location of two lanes of a junction, and use the merge location to update extents of an overlap area to prevent the autonomous vehicle from blocking the junction and/or provide sufficient space to yield to the oncoming vehicle while merging.

Abstract: A transportation system controls a fleet of autonomous vehicles to implement passenger transportation and coordinate delivery of baggage or other associated items using separate vehicles. The transportation system receives passenger data and associated item data via a user interface, and determines the number and type of autonomous vehicles to transport the passengers and items from selected pick-up locations to a destination. In various implementations, the transportation system may support different pick-up locations, pick-up times and/or delivery times for the passengers and associated items. The transportation system also may determine delayed item delivery options for different delivery times and modes of transportation.

Abstract: An imitation learning-based machine-learned (ML) model to augment or replace the prediction and/or planner components of an autonomous vehicle may be trained using a two stage and multi-discipline approach. A first stage of training may include training the ML component to output a predicted action associated with a target vehicle and modifying the ML component to reduce a difference between the predicted action and the observed action taken by the target vehicle. A second stage may use reinforcement learning to further tun the ML component. The resultant model may be used on its own, with enough training data, or to rank or weight candidate trajectories generated by a planning component of the vehicle. The ML component may provide embeddings of environment features to first transformer(s) that output to a long short-term memory that outputs to second transformer(s) to determine the predicted action.

Abstract: Techniques for determining errors or drifts between maps used for updating maps and/or controlling a system which uses the map. In some examples, a first, global, map may be received or determined. Sensor data may then be used to localize a system with respect to the first map and to generate a first trajectory relative to the first map. The sensor data may be used to create a second map and a second trajectory for navigating the system relative to the second map. Differences between the first and second trajectories (or portions thereof), when compared in a common reference frame, may be used as an indication of drift between various processes or errors in the maps and, subsequently, be used for updating the first map and/or controlling the system.

Abstract: Techniques for determining an optimal trajectory for a vehicle are discussed herein. In some cases, the described techniques include receiving, from a first computing system and by a second computing system (e.g., a computing system associated with a trajectory validation system of the vehicle), a primary trajectory and an alternative trajectory for the vehicle. In some cases, the second computing system is configured to select or otherwise determine whether to select the primary trajectory, the alternative trajectory, or neither trajectory for the vehicle. In some cases, the alternative trajectory is determined by applying a lateral swerve to the primary trajectory.