Abstract: A sensor data processing system includes various elements, including a perception unit that collects data representing positions of sensors on a vehicle and obtains environmental information around the vehicle via the sensors. The sensor data processing system also includes a feature fusion unit that combines the first environmental information from the sensors into first fused feature data representing first positions of objects around the vehicle, provides the first fused feature data to the object tracking unit, receives feedback for the first fused feature data from the object tracking unit, and combines second environmental information from the sensors using the feedback into second fused feature data representing second positions of objects around the vehicle. The sensor data processing system may then at least partially control operation of the vehicle using the second fused feature data.

Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving a plurality of image frames representative of a scene; receiving point cloud data representative of the scene; determining, using a NeRF model, a three-dimensional reconstruction of the scene based on the plurality of image frames; and outputting fused data that combines first BEV features of the three-dimensional reconstruction of the scene and second BEV features of the point cloud data. Other aspects and features are also claimed and described.

Abstract: Systems and techniques are described herein for training an object-detection model. For instance, a method for training an object-detection model is provided. The method may include obtaining a light detection and ranging (LIDAR) capture; obtaining a first LIDAR-based representation of an object as captured from a first distance; obtaining a second LIDAR-based representation of the object as captured from a second distance; augmenting the LIDAR capture using the first LIDAR-based representation of the object and the second LIDAR-based representation of the object to generate an augmented LIDAR capture; and training a machine-learning object-detection model using the augmented LIDAR capture.

Abstract: Systems and methods related to controlling an autonomous vehicle (“AV”) are described herein. Implementations can process actor(s) from a past episode of locomotion of a vehicle, and stream(s) in an environment of the vehicle during the past episode to generate predicted output(s). The actor(s) may each be associated with a corresponding object in the environment of the vehicle, and the stream(s) may each represent candidate navigation paths in the environment of the vehicle. Further, implementations can process the predicted output(s) to generate further predicted output(s), and can compare the predicted output(s) to associated reference label(s). The processing can be performed utilizing layer(s) or distinct, additional layer(s) of machine learning (“ML”) model(s). Implementations can update the layer(s) or the additional layer(s) based on the comparing, and subsequently use the ML model(s) in controlling the AV.

Abstract: Implementations process, using machine learning (ML) layer(s) of ML model(s), actor(s) from a past episode of locomotion of a vehicle and stream(s) in an environment of the vehicle during the past episode to forecast associated trajectories, for the vehicle and for each of the actor(s), with respect to a respective associated stream of the stream(s). Further, implementations process, using a stream connection function, the associated trajectories to forecast a plurality of associated trajectories, for the vehicle and each of the actor(s), with respect to each of the stream(s). Moreover, implementations iterate between using the ML layer(s) and the stream connection function to update the associated trajectories for the vehicle and each of the actor(s). Implementations subsequently use the ML layer(s) in controlling an AV.

Abstract: Example methods for multistage autonomous vehicle motion planning include obtaining sensor data descriptive of an environment of the autonomous vehicle; identifying one or more objects in the environment based on the sensor data; generating a plurality of candidate strategies, wherein each candidate strategy of the plurality of candidate strategies comprises a set of discrete decisions respecting the one or more objects, wherein generating the plurality of candidate strategies includes: determining that at least two strategies satisfy an equivalence criterion, such that the plurality of candidate strategies include at least one candidate strategy corresponding to an equivalence class representative of a plurality of different strategies that are based on different discrete decisions; determining candidate trajectories respectively for the plurality of candidate strategies; and initiating control of the autonomous vehicle based on a selected candidate trajectory.

Abstract: Implementations process, using machine learning (ML) layer(s) of ML model(s), actor(s) from a past episode of locomotion of a vehicle and stream(s) in an environment of the vehicle during the past episode to forecast associated trajectories, for the vehicle and for each of the actor(s), with respect to a respective associated stream of the stream(s). Further, implementations process, using a stream connection function, the associated trajectories to forecast a plurality of associated trajectories, for the vehicle and each of the actor(s), with respect to each of the stream(s). Moreover, implementations iterate between using the ML layer(s) and the stream connection function to update the associated trajectories for the vehicle and each of the actor(s). Implementations subsequently use the ML layer(s) in controlling an AV.

Abstract: Systems and methods related to controlling an autonomous vehicle (“AV”) are described herein. Implementations can obtain a plurality of instances that each include input and output. The input can include actor(s) from a given time instance of a past episode of locomotion of a vehicle, and stream(s) in an environment of the vehicle during the past episode. The actor(s) may be associated with an object in the environment of the vehicle at the given time instance, and the stream(s) may each represent candidate navigation paths in the environment of the vehicle. The output may include ground truth label(s) (or reference label(s)). Implementations can train a machine learning (“ML”) model based on the plurality of instances, and subsequently use the ML model in controlling the AV. In training the ML model, the actor(s) and stream(s) can be processed in parallel.

Abstract: Example methods for multistage autonomous vehicle motion planning include obtaining sensor data descriptive of an environment of the autonomous vehicle; identifying one or more objects in the environment based on the sensor data; generating a plurality of candidate strategies, wherein each candidate strategy of the plurality of candidate strategies comprises a set of discrete decisions respecting the one or more objects, wherein generating the plurality of candidate strategies includes: determining that at least two strategies satisfy an equivalence criterion, such that the plurality of candidate strategies include at least one candidate strategy corresponding to an equivalence class representative of a plurality of different strategies that are based on different discrete decisions; determining candidate trajectories respectively for the plurality of candidate strategies; and initiating control of the autonomous vehicle based on a selected candidate trajectory.

Abstract: Implementations process, using machine learning (ML) layer(s) of ML model(s), actor(s) from a past episode of locomotion of a vehicle and stream(s) in an environment of the vehicle during the past episode to forecast associated trajectories, for the vehicle and for each of the actor(s), with respect to a respective associated stream of the stream(s). Further, implementations process, using a stream connection function, the associated trajectories to forecast a plurality of associated trajectories, for the vehicle and each of the actor(s), with respect to each of the stream(s). Moreover, implementations iterate between using the ML layer(s) and the stream connection function to update the associated trajectories for the vehicle and each of the actor(s). Implementations subsequently use the ML layer(s) in controlling an AV.