Abstract: A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.

Abstract: This disclosure relates to systems and techniques for identifying collisions, such as relatively low energy impact collisions involving an autonomous vehicle. Sensor data from a first sensor modality in a first array may be used to determine a first estimated location of impact and second sensor data from a second sensor modality in a second array may be used to determine a second estimated location of impact. A low energy impact event may be configured when the first estimated location of impact corresponds to the second estimated location of impact.

Abstract: Navigation systems can identify objects in an environment and generate representations of those objects. A representation of an articulated vehicle can include two segments rotated relative to each other about a pivot, with a first segment corresponding to a first portion of the articulated vehicle and the second segment corresponding to a second portion of the articulated vehicle. The articulated object can be tracked in the environment by generating estimated updated states of the articulated agent based on previous states and/or measured states of the object using differing motion model updates for the differing portions. The estimated updated states may be determined using one or more filtering algorithms, which may be constrained using pseudo-observables.

Abstract: Techniques to use a trained model to determine a yaw of an object are described. For example, a system may implement various techniques to generate multiple representations for an object in an environment. Each representation vary based on the technique and data used. An estimation component may estimate a representation from the multiple representations. The model may be implemented to output a yaw for the object using the multiple representations, the estimated representation, and/or additional information. The output yaw may be used to track an object, generate a trajectory, or otherwise control a vehicle.

Abstract: Techniques for updating data operations in a perception system are discussed herein. A vehicle may use a perception system to capture data about an environment proximate to the vehicle. The perception system may receive image data, lidar data, and/or radar data to determine information about an object in the environment. As different sensors may be associated with different time periods for capturing and/or processing operations, the techniques include updating object data with data from sensors associated with a shorter time period to generate intermediate object data.

Abstract: A vehicle computing system may implement techniques to determine whether two objects in an environment are related as an articulated object. The techniques may include applying heuristics and algorithms to object representations (e.g., bounding boxes) to determine whether two objects are related as a single object with two portions that articulate relative to each other. The techniques may include predicting future states of the articulated object in the environment. One or more model(s) may be used to determine presence of the articulated object and/or predict motion of the articulated object in the future. Based on the presence and/or motion of the articulated object, the vehicle computing system may control operation of the vehicle.

Abstract: Techniques for determining error models for use in simulations are discussed herein. Ground truth perception data and vehicle perception data can be determined from vehicle log data. Further, objects in the log data can be identified as relevant objects by signals output by a planner system or based on the object being located in a driving corridor. Differences between the ground truth perception data and the vehicle perception data can be determined and used to generate error models for the relevant objects. The error models can be applied to objects during simulation to increase realism and test vehicle components.

Abstract: Techniques for determining an output from a plurality of sensor modalities are discussed herein. Features from a radar sensor, a lidar sensor, and an image sensor may be input into respective models to determine respective intermediate outputs associated with a tracks associated with an object and associated confidence levels. The Intermediate outputs from a radar model, a lidar model, and an vision model may be input into a fused model to determine a fused confidence level and fused output associated with the track. The fused confidence level and the individual confidence levels are compared to a threshold to generate the track to transmit to a planning system or prediction system of an autonomous vehicle. Additionally, a vehicle controller can control the autonomous vehicle based on the track and/or on the confidence level(s).

Abstract: Training a model for detecting pedestrian objects is described. Annotated data can be received by a training component. In an example, the annotated data can include a first indication of a first object (e.g., a pedestrian) and a second indication of a second object (e.g., a pedestrian object), wherein the first object and the second object comprise a portion of a compound object (e.g., a pedestrian/pedestrian object system). In an example, a training component can train a model to determine an output associated with a second object based at least in part on the annotated data and the model can be transmitted to a vehicle configured to be controlled by output(s) of the model.

Abstract: The techniques discussed herein include modifying a Kalman filter to additionally include a loss component that dampens the effect measurements with large errors (or measurements indicating states that are rather different than the predicted state) have on the Kalman filter and, in particular, the updated uncertainty and/or updated prediction. In some examples, the techniques include scaling a Kalman gain based at least in part on a loss function that is based on the innovation determined by the Kalman filter. The techniques additionally or alternatively include a reformulation of a Kalman filter that ensures that the uncertainties determined by the Kalman filter remain symmetric and positive definite.

Abstract: A computer-implemented method. Includes obtaining pointwise data indicating, for a plurality of time steps, a pointwise measurement of a state of an object detected by an object detection system. Includes obtaining, from a runtime model, runtime data indicating, for the plurality of time steps, a runtime estimate of the state of the object. Includes processing, by a benchmark model, the pointwise data to determine, for the plurality of time steps, a benchmark estimate of the state of the object. Includes evaluating a metric measuring, for the plurality of time steps, a deviation between the runtime estimate and the benchmark estimate of the state of the object. Includes updating, based on the on the evaluation of the metric, the runtime model.

Abstract: Fast simulation of a scenario (e.g., simulating the scenario once as opposed to multiple times) to determine performance metric(s) of a configuration of one or more components of an autonomous vehicle may include training a perception error model based at least in part on a difference between a prediction output by a perception component associated with a future time and a perception output associated with that future time once that future time has arrived. A contour or heat map output by the perception error model may be used to determine one or more performance metric(s) associated with a component of the autonomous vehicle and identify which component may cause a degradation of a performance metric.

Abstract: A computer-implemented method. Includes obtaining pointwise data indicating, for a plurality of time steps, a pointwise measurement of a state of an object detected by an object detection system. Includes obtaining, from a runtime model, runtime data indicating, for the plurality of time steps, a runtime estimate of the state of the object. Includes processing, by a benchmark model, the pointwise data to determine, for the plurality of time steps, a benchmark estimate of the state of the object. Includes evaluating a metric measuring, for the plurality of time steps, a deviation between the runtime estimate and the benchmark estimate of the state of the object. Includes updating, based on the on the evaluation of the metric, the runtime model.

Abstract: Systems, methods, and apparatuses described herein are directed to performing segmentation on voxels representing three-dimensional data to identify static and dynamic objects. LIDAR data may be captured by a perception system for an autonomous vehicle and represented in a voxel space. Operations may include determining a drivable surface by parsing individual voxels to determine an orientation of a surface normal of a planar approximation of the voxelized data relative to a reference direction. Clustering techniques can be used to grow a ground plane including a plurality of locally flat voxels. Ground plane data can be set aside from the voxel space, and the remaining voxels can be clustered to determine objects. Voxel data can be analyzed over time to determine dynamic objects. Segmentation information associated with ground voxels, static object, and dynamic objects can be provided to a tracker and/or planner in conjunction with operating the autonomous vehicle.

Abstract: Techniques for a perception system of a vehicle that can detect and track objects in an environment are described herein. The perception system may include a machine-learned model that includes one or more different portions, such as different components, subprocesses, or the like. In some instances, the techniques may include training the machine-learned model end-to-end such that outputs of a first portion of the machine-learned model are tailored for use as inputs to another portion of the machine-learned model. Additionally, or alternatively, the perception system described herein may utilize temporal data to track objects in the environment of the vehicle and associate tracking data with specific objects in the environment detected by the machine-learned model. That is, the architecture of the machine-learned model may include both a detection portion and a tracking portion in the same loop.

Abstract: Techniques for determining an output from a plurality of sensor modalities are discussed herein. Features from a radar sensor, a lidar sensor, and an image sensor may be input into respective models to determine respective intermediate outputs associated with a tracks associated with an object and associated confidence levels. The Intermediate outputs from a radar model, a lidar model, and an vision model may be input into a fused model to determine a fused confidence level and fused output associated with the track. The fused confidence level and the individual confidence levels are compared to a threshold to generate the track to transmit to a planning system or prediction system of an autonomous vehicle. Additionally, a vehicle controller can control the autonomous vehicle based on the track and/or on the confidence level(s).

Abstract: Techniques for a perception system of a vehicle that can detect and track objects in an environment are described herein. The perception system may include a machine-learned model that includes one or more different portions, such as different components, subprocesses, or the like. In some instances, the techniques may include training the machine-learned model end-to-end such that outputs of a first portion of the machine-learned model are tailored for use as inputs to another portion of the machine-learned model. Additionally, or alternatively, the perception system described herein may utilize temporal data to track objects in the environment of the vehicle and associate tracking data with specific objects in the environment detected by the machine-learned model. That is, the architecture of the machine-learned model may include both a detection portion and a tracking portion in the same loop.

Abstract: Techniques for a perception system of a vehicle that can detect and track objects in an environment are described herein. The perception system may include a machine-learned model that includes one or more different portions, such as different components, subprocesses, or the like. In some instances, the techniques may include training the machine-learned model end-to-end such that outputs of a first portion of the machine-learned model are tailored for use as inputs to another portion of the machine-learned model. Additionally, or alternatively, the perception system described herein may utilize temporal data to track objects in the environment of the vehicle and associate tracking data with specific objects in the environment detected by the machine-learned model. That is, the architecture of the machine-learned model may include both a detection portion and a tracking portion in the same loop.

Abstract: Tracking a current and/or previous position, velocity, acceleration, and/or heading of an object using sensor data may comprise determining whether to associate a current object detection generated from recently received (e.g., current) sensor data with a previous object detection generated from formerly received sensor data. In other words, a track may identify that an object detected in former sensor data is the same object detected in current sensor data. However, multiple types of sensor data may be used to detect objects and some objects may not be detected by different sensor types or may be detected differently, which may confound attempts to track an object. An ML model may be trained to receive outputs associated with different sensor types and/or a track associated with an object, and determine a data structure comprising a region of interest, object classification, and/or a pose associated with the object.

Abstract: A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.