Abstract: Techniques for generating a database containing predicted driving scenarios are discussed herein. A database management component may receive sensor data captured by sensors of vehicle(s) (e.g., driving log data) based on previous driving trips within various physical driving environments. The database management component may cluster driving scenarios observed from the sensor data based on the driving scenario's feature similarity. In some examples, the database management component may determine prediction information (e.g., actual trajectory from log data or trajectory generated by machine-learning model) to associate with each cluster. Accordingly, each cluster may include an encoded representation (e.g., key) as well as corresponding prediction information (e.g., value). In some examples, the database management component may store some or all key-value pairs in a database accessible by one or more vehicles while such vehicles navigate an environment.

Abstract: Examples of the present disclosure describe systems and methods for using AI to identify regions of interest (ROI) in medical images. In aspects, medical reports and images may be provided to a second environment. The second environment may use the medical report data/medical images to train a natural language processing (NLP)-based algorithm to identify the location in images of ROI described in the medical report data. The output of the NLP-based algorithm may be stored in an ROI repository in the second environment. After the NLP-based algorithm has been trained, a request to train a user-specific model may be received in a first environment. Data objects for the requested user-specific model may be provided to the second environment, which uses the ROI repository to train the model. The trained model may be provided to the first environment, where the trained user-specific model/algorithm may be tested and stored.

Abstract: A machine-learned architecture may predict a set of spatially-diverse paths that an object may take in the future. The paths generated by this architecture may be time-invariant (e.g., not identifying a time at which the object may occupy a position along one of these paths) but can be used by a second machine-learned model to predict progress in time along these paths. This segregation of the spatial paths and progress in time along the paths improves the accuracy of the ultimate prediction and better captures rare object behavior.

Abstract: Predicting a future state, such as a future position and/or orientation (i.e., pose), of an object may comprise classifying, by a first machine-learned model, a lane the object may occupy and classifying, by a second machine-learned model, a target pose the object may occupy. A third machine-learned model may determine an offset from the target pose that may be used to determine a predicted (future) pose of the object by applying the offset to the target pose.

Abstract: A machine-learned architecture may predict a set of spatially-diverse paths that an object may take in the future. The paths generated by this architecture may be time-invariant (e.g., not identifying a time at which the object may occupy a position along one of these paths) but can be used by a second machine-learned model to predict progress in time along these paths. This segregation of the spatial paths and progress in time along the paths improves the accuracy of the ultimate prediction and better captures rare object behavior.

Abstract: Examples of the present disclosure describe systems and methods for using AI to identify regions of interest (ROI) in medical images. In aspects, medical reports and images may be provided to a second environment. The second environment may use the medical report data/medical images to train a natural language processing (NLP)-based algorithm to identify the location in images of ROI described in the medical report data. The output of the NLP-based algorithm may be stored in an ROI repository in the second environment. After the NLP-based algorithm has been trained, a request to train a user-specific model may be received in a first environment. Data objects for the requested user-specific model may be provided to the second environment, which uses the ROI repository to train the model. The trained model may be provided to the first environment, where the trained user-specific model/algorithm may be tested and stored.

Abstract: A machine-learned architecture for determining whether an object is relevant to a vehicle's action planning may comprise a convolutional neural network, graph neural network, and/or multi-layer perceptron that may determine a relevance score associated with an object that indicates indicating whether an object is likely to impact operation(s) of a vehicle. In some examples, the machine-learned architecture may use scene information and/or an object track to determine the relevance score.

Abstract: A machine-learned architecture may predict a set of spatially-diverse paths that an object may take in the future. The paths generated by this architecture may be time-invariant (e.g., not identifying a time at which the object may occupy a position along one of these paths) but can be used by a second machine-learned model to predict progress in time along these paths. This segregation of the spatial paths and progress in time along the paths improves the accuracy of the ultimate prediction and better captures rare object behavior.

Abstract: Techniques for determining a response of a simulated vehicle to a simulated object in a simulation are discussed herein. Log data captured by a physical vehicle in an environment can be received. Object data representing an object in the log data can be used to instantiate a simulated object in a simulation to determine a response of a simulated vehicle to the simulated object. Additionally, one or more trajectory segments in a trajectory library representing the log data can be determined and instantiated as a trajectory of the simulated object in order to increase the accuracy and realism of the simulation.

Abstract: Techniques for determining a response of a simulated vehicle to a simulated object in a simulation are discussed herein. Log data captured by a physical vehicle in an environment can be received. Object data representing an object in the log data can be used to instantiate a simulated object in a simulation to determine a response of a simulated vehicle to the simulated object. Additionally, one or more trajectory segments in a trajectory library representing the log data can be determined and instantiated as a trajectory of the simulated object in order to increase the accuracy and realism of the simulation.

Abstract: Techniques for determining a classification probability of an object in an environment are discussed herein. Techniques may include analyzing sensor data associated with an environment from a perspective, such as a top-down perspective, using multi-channel data. From this perspective, techniques may determine channels of multi-channel input data and additional feature data. Channels corresponding to spatial features may be included in the multi-channel input data and data corresponding to non-spatial features may be included in the additional feature data. The multi-channel input data may be input to a first portion of a machine-learned (ML) model, and the additional feature data may be concatenated with intermediate output data from the first portion of the ML model, and input into a second portion of the ML model for subsequent processing and to determine the classification probabilities. Additionally, techniques may be performed on a multi-resolution voxel space representing the environment.

Abstract: Techniques for determining a classification probability of an object in an environment are discussed herein. Techniques may include analyzing sensor data associated with an environment from a perspective, such as a top-down perspective, using multi-channel data. From this perspective, techniques may determine channels of multi-channel input data and additional feature data. Channels corresponding to spatial features may be included in the multi-channel input data and data corresponding to non-spatial features may be included in the additional feature data. The multi-channel input data may be input to a first portion of a machine-learned (ML) model, and the additional feature data may be concatenated with intermediate output data from the first portion of the ML model, and input into a second portion of the ML model for subsequent processing and to determine the classification probabilities. Additionally, techniques may be performed on a multi-resolution voxel space representing the environment.

Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include determining a trajectory of the object, determining an intent of the trajectory, and sending the trajectory and the intent to a vehicle computing system to control an autonomous vehicle. The vehicle computing system may implement a machine learned model to process data such as sensor data and map data. The machine learned model can associate different intentions of an object in an environment with different trajectories. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on object's intentions and trajectories.

Abstract: Techniques for determining a response of a simulated vehicle to a simulated object in a simulation are discussed herein. Log data captured by a physical vehicle in an environment can be received. Object data representing an object in the log data can be used to instantiate a simulated object in a simulation to determine a response of a simulated vehicle to the simulated object. Additionally, one or more trajectory segments in a trajectory library representing the log data can be determined and instantiated as a trajectory of the simulated object in order to increase the accuracy and realism of the simulation.

Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include inputting data into a model and receiving an output from the model representing a discretized representation. The discretized representation may be associated with a probability of an object reaching a location in the environment at a future time. A vehicle computing system may determine a trajectory and a weight associated with the trajectory using the discretized representation and the probability. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the trajectory and the weight output by the vehicle computing system.

Abstract: Techniques for determining a classification probability of an object in an environment are discussed herein. Techniques may include analyzing sensor data associated with an environment from a perspective, such as a top-down perspective, using multi-channel data. From this perspective, techniques may determine channels of multi-channel input data and additional feature data. Channels corresponding to spatial features may be included in the multi-channel input data and data corresponding to non-spatial features may be included in the additional feature data. The multi-channel input data may be input to a first portion of a machine-learned (ML) model, and the additional feature data may be concatenated with intermediate output data from the first portion of the ML model, and input into a second portion of the ML model for subsequent processing and to determine the classification probabilities. Additionally, techniques may be performed on a multi-resolution voxel space representing the environment.

Abstract: Techniques for determining a classification probability of an object in an environment are discussed herein. Techniques may include analyzing sensor data associated with an environment from a perspective, such as a top-down perspective, using multi-channel data. From this perspective, techniques may determine channels of multi-channel input data and additional feature data. Channels corresponding to spatial features may be included in the multi-channel input data and data corresponding to non-spatial features may be included in the additional feature data. The multi-channel input data may be input to a first portion of a machine-learned (ML) model, and the additional feature data may be concatenated with intermediate output data from the first portion of the ML model, and input into a second portion of the ML model for subsequent processing and to determine the classification probabilities. Additionally, techniques may be performed on a multi-resolution voxel space representing the environment.

Abstract: Examples of the present disclosure describe systems and methods for using AI to identify regions of interest (ROI) in medical images. In aspects, medical reports and images may be provided to a first environment. The first environment may use the medical report data/medical images to train a natural language processing (NLP)-based algorithm to identify the location in images of ROI described in the medical report data. The output of the NLP-based algorithm may be stored in an ROI repository in the first environment. After the NLP-based algorithm has been trained, a request to train a user-specific model may be received in a second environment. Data objects for the requested user-specific model may be provided to the first environment, which uses the ROI repository to train the model. The trained model may be provided to the second environment, where the trained user-specific model/algorithm may be tested and stored.

Abstract: Examples of the present disclosure describe systems and methods for using AI to identify regions of interest (ROI) in medical images. In aspects, medical reports and images may be provided to a second environment. The second environment may use the medical report data/medical images to train a natural language processing (NLP)-based algorithm to identify the location in images of ROI described in the medical report data. The output of the NLP-based algorithm may be stored in an ROI repository in the second environment. After the NLP-based algorithm has been trained, a request to train a user-specific model may be received in a first environment. Data objects for the requested user-specific model may be provided to the second environment, which uses the ROI repository to train the model. The trained model may be provided to the first environment, where the trained user-specific model/algorithm may be tested and stored.

Abstract: Techniques to predict object behavior in an environment are discussed herein. For example, such techniques may include inputting data into a model and receiving an output from the model representing a discretized representation. The discretized representation may be associated with a probability of an object reaching a location in the environment at a future time. A vehicle computing system may determine a trajectory and a weight associated with the trajectory using the discretized representation and the probability. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the trajectory and the weight output by the vehicle computing system.