Abstract: Systems and methods for trajectory prediction are provided. A method can include obtaining LIDAR data, radar data, and map data; inputting the LIDAR data, the radar data, and the map data into a network model; transforming, by the network model, the radar data into a coordinate frame associated with a most recent radar sweep in the radar data; generating, by the network model, one or more features for each of the LIDAR data, the transformed radar data, and the map data; combining, by the network model, the one or more generated features to generate fused feature data; generating, by the network model, prediction data based at least in part on the fused feature data; and receiving, as an output of the network model, the prediction data. The prediction data can include a respective predicted trajectory for a future time period for one or more detected objects.

Abstract: Systems, devices, and methods for trajectory association and tracking are provided. A method can include method can include obtaining input data indicative of a respective trajectory for each of one or more first objects for a first time step and input data indicative of a respective trajectory for each of one or more second objects for a second time step subsequent to the first time step. The method can include generating, using a machine-learned model, a temporally-consistent trajectory for at least one of the one or more first objects or the one or more second objects based at least in part on the input data and determining a third predicted trajectory for the at least one of the one or more first objects or the one or more second objects for at least the second time step based at least in part on the temporally-consistent trajectory.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices for detecting objects are provided. For example, the disclosed technology can obtain a representation of sensor data associated with an environment surrounding a vehicle. Further, the sensor data can include sensor data points. A point classification and point property estimation can be determined for each of the sensor data points and a portion of the sensor data points can be clustered into an object instance based on the point classification and point property estimation for each of the sensor data points. A collection of point classifications and point property estimations can be determined for the portion of the sensor data points clustered into the object instance. Furthermore, object instance property estimations for the object instance can be determined based on the collection of point classifications and point property estimations for the portion of the sensor data points clustered into the object instance.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices for associating objects are provided. For example, the disclosed technology can receive sensor data associated with the detection of objects over time. An association dataset can be generated and can include information associated with object detections of the objects at a most recent time interval and object tracks of the objects at time intervals in the past. A subset of the association dataset including the object detections that satisfy some association subset criteria can be determined. Association scores for the object detections in the subset of the association dataset can be determined. Further, the object detections can be associated with the object tracks based on the association scores for each of the object detections in the subset of the association dataset that satisfy some association criteria.

Abstract: Systems and methods for detecting objects and predicting their motion are provided. In particular, a computing system can obtain a plurality of sensor sweeps. The computing system can determine movement data associated with movement of the autonomous vehicle. For each sensor sweep, the computing system can generate an image associated with the sensor sweep. The computing system can extract, using the respective image as input to one or more machine-learned models, feature data from the respective image. The computing system can transform the feature data into a coordinate frame associated with a next time step. The computing system can generate a fused image. The computing system can generate a final fused image. The computing system can predict, based, at least in part, on the final fused representation of the plurality of sensors sweeps from the plurality of sensor sweeps, movement associated with the feature data at one or more time steps in the future.

Abstract: An autonomous vehicle computing system can include a primary perception system configured to receive a plurality of sensor data points as input generate primary perception data representing a plurality of classifiable objects and a plurality of paths representing tracked motion of the plurality of classifiable objects. The autonomous vehicle computing system can include a secondary perception system configured to receive the plurality of sensor data points as input, cluster a subset of the plurality of sensor data points of the sensor data to generate one or more sensor data point clusters representing one or more unclassifiable objects that are not classifiable by the primary perception system, and generate secondary path data representing tracked motion of the one or more unclassifiable objects. The autonomous vehicle computing system can generate fused perception data based on the primary perception data and the one or more unclassifiable objects.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices associated with object association and tracking are provided. Input data can be obtained. The input data can be indicative of a detected object within a surrounding environment of an autonomous vehicle and an initial object classification of the detected object at an initial time interval and object tracks at time intervals preceding the initial time interval. Association data can be generated based on the input data and a machine-learned model. The association data can indicate whether the detected object is associated with at least one of the object tracks. An object classification probability distribution can be determined based on the association data. The object classification probability distribution can indicate a probability that the detected object is associated with each respective object classification. The association data and the object classification probability distribution for the detected object can be outputted.

Abstract: Systems and methods for detecting and classifying objects proximate to an autonomous vehicle can include a sensor system and a vehicle computing system. The sensor system includes at least one LIDAR system configured to transmit ranging signals relative to the autonomous vehicle and to generate LIDAR data. The vehicle computing system receives the LIDAR data from the sensor system. The vehicle computing system also determines at least a range-view representation of the LIDAR data and a top-view representation of the LIDAR data, wherein the range-view representation contains a fewer number of total data points than the top-view representation. The vehicle computing system further detects objects of interest in the range-view representation of the LIDAR data and generates a bounding shape for each of the detected objects of interest in the top-view representation of the LIDAR data.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices associated with sensor output segmentation are provided. For example, sensor data can be accessed. The sensor data can include sensor data returns representative of an environment detected by a sensor across the sensor's field of view. Each sensor data return can be associated with a respective bin of a plurality of bins corresponding to the field of view of the sensor. Each bin can correspond to a different portion of the sensor's field of view. Channels can be generated for each of the plurality of bins and can include data indicative of a range and an azimuth associated with a sensor data return associated with each bin. Furthermore, a semantic segment of a portion of the sensor data can be generated by inputting the channels for each bin into a machine-learned segmentation model trained to generate an output including the semantic segment.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices for autonomous vehicle operation are provided. For example, a computing system can receive object data that includes portions of sensor data. The computing system can determine, in a first stage of a multiple stage classification using hardware components, one or more first stage characteristics of the portions of sensor data based on a first machine-learned model. In a second stage of the multiple stage classification, the computing system can determine second stage characteristics of the portions of sensor data based on a second machine-learned model. The computing system can generate an object output based on the first stage characteristics and the second stage characteristics. The object output can include indications associated with detection of objects in the portions of sensor data.

Abstract: Image processing systems can include one or more cameras configured to obtain image data, one or more memory devices configured to store a classification model that classifies image features within the image data as including or not including detected objects, and a field programmable gate array (FPGA) device coupled to the one or more cameras. The FPGA device is configured to implement one or more image processing pipelines for image transformation and object detection. The one or more image processing pipelines can generate a multi-scale image pyramid of multiple image samples having different scaling factors, identify and aggregate features within one or more of the multiple image samples having different scaling factors, access the classification model, provide the features as input to the classification model, and receive an output indicative of objects detected within the image data.

Abstract: Object detection systems and methods can include identifying an object of interest within an image obtained from a camera, obtaining a first supplemental portion of data associated with the object of interest determining an estimated location of the object of interest within three-dimensional space based at least in part on the first supplemental portion of data and a known relative location of the camera, determining a portion of the LIDAR point data corresponding to the object of interest based at least in part on the estimated location of the object of interest within three-dimensional space, and providing one or more of at least a portion of the image corresponding to the object of interest and the portion of LIDAR point data corresponding to the object of interest as an output.

Abstract: Systems and methods for detecting and classifying objects proximate to an autonomous vehicle can include a sensor system and a vehicle computing system. The sensor system includes at least one LIDAR system configured to transmit ranging signals relative to the autonomous vehicle and to generate LIDAR data. The vehicle computing system receives the LIDAR data from the sensor system. The vehicle computing system also determines at least a range-view representation of the LIDAR data and a top-view representation of the LIDAR data, wherein the range-view representation contains a fewer number of total data points than the top-view representation. The vehicle computing system further detects objects of interest in the range-view representation of the LIDAR data and generates a bounding shape for each of the detected objects of interest in the top-view representation of the LIDAR data.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices for autonomous vehicle operation are provided. For example, a computing system can receive object data that includes portions of sensor data. The computing system can determine, in a first stage of a multiple stage classification using hardware components, one or more first stage characteristics of the portions of sensor data based on a first machine-learned model. In a second stage of the multiple stage classification, the computing system can determine second stage characteristics of the portions of sensor data based on a second machine-learned model. The computing system can generate an object output based on the first stage characteristics and the second stage characteristics. The object output can include indications associated with detection of objects in the portions of sensor data.

Abstract: A method and non-transitory computer-readable medium capture an image of bulk grain and apply a feature extractor to the image to determine a feature of the bulk grain in the image. For each of a plurality of different sampling locations in the image, based upon the feature of the bulk grain at the sampling location, a determination is made regarding a classification score for the presence of a classification of material at the sampling location. A quality of the bulk grain of the image is determined based upon an aggregation of the classification scores for the presence of the classification of material at the sampling locations.

Abstract: An object detection system for an autonomous vehicle processes sensor data, including one or more images, obtained for a road segment on which the autonomous vehicle is being driven. The object detection system compares the images to three-dimensional (3D) environment data for the road segment to determine pixels in the images that correspond to objects not previously identified in the 3D environment data. The object detection system then analyzes the pixels to classify the objects not previously identified in the 3D environment data.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices for detecting objects are provided. For example, the disclosed technology can obtain a representation of sensor data associated with an environment surrounding a vehicle. Further, the sensor data can include sensor data points. A point classification and point property estimation can be determined for each of the sensor data points and a portion of the sensor data points can be clustered into an object instance based on the point classification and point property estimation for each of the sensor data points. A collection of point classifications and point property estimations can be determined for the portion of the sensor data points clustered into the object instance. Furthermore, object instance property estimations for the object instance can be determined based on the collection of point classifications and point property estimations for the portion of the sensor data points clustered into the object instance.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices for operating an autonomous vehicle are provided. For example, the disclosed technology can include receiving sensor data and map data. The sensor data can include information associated with an environment detected by sensors of a vehicle. The map data can include information associated with traffic signals in the environment. Further, an input representation can be generated based on the sensor data and the map data. The input representation can include regions of interest associated with images of the traffic signals. States of the traffic signals in the environment can be determined, based on the input representation and a machine-learned model. Traffic signal state data that includes a determinative state of the traffic signals can be generated based on the states of the traffic signals.

Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices for associating objects are provided. For example, the disclosed technology can receive sensor data associated with the detection of objects over time. An association dataset can be generated and can include information associated with object detections of the objects at a most recent time interval and object tracks of the objects at time intervals in the past. A subset of the association dataset including the object detections that satisfy some association subset criteria can be determined. Association scores for the object detections in the subset of the association dataset can be determined. Further, the object detections can be associated with the object tracks based on the association scores for each of the object detections in the subset of the association dataset that satisfy some association criteria.

Abstract: Generally, the present disclosure is directed to systems and methods for detecting objects of interest and determining location information and motion information for the detected objects based at least in part on sensor data (e.g., LIDAR data) provided from one or more sensor systems (e.g., LIDAR systems) included in the autonomous vehicle. The perception system can include a machine-learned detector model that is configured to receive multiple time frames of sensor data and implement curve-fitting of sensor data points over the multiple time frames of sensor data. The machine-learned model can be trained to determine, in response to the multiple time frames of sensor data provided as input, location information descriptive of a location of one or more objects of interest detected within the environment at a given time and motion information descriptive of the motion of each object of interest.