Abstract: A method for inferring a location of an object includes extracting features from sensor data obtained from a number of sensors of an autonomous vehicle and encoding the features to a number of sensor space representations. The method also reshapes the number of sensor space representations to a feature space representation corresponding to a feature space of a spatial area. The method further identifies the object based on a mapping of the features to the feature space representation. The method still further projects a representation of the identified object to a location of the feature space and controls an action of the autonomous vehicle based on projecting the representation.

Abstract: System, methods, and other embodiments described herein relate to improving querying of a visual dataset of images through implementing system-aware cascades. In one embodiment, a method includes enumerating a set of cascade classifiers that are each separately comprised of transformation modules and machine learning modules arranged in multiple pairs. Classifiers of the set of cascade classifiers are configured to extract content from an image according to a query. The method includes selecting a query classifier from the set of cascade classifiers based, at least in part, on system costs that characterize computational resources consumed by the classifiers of the set of cascade classifiers. The computational resources include at least data handling costs. The method includes identifying content within the image using the query classifier.

Abstract: A method for inferring a location of an object includes extracting features from sensor data obtained from a number of sensors of an autonomous vehicle and encoding the features to a number of sensor space representations. The method also reshapes the number of sensor space representations to a feature space representation corresponding to a feature space of a spatial area. The method further identifies the object based on a mapping of the features to the feature space representation. The method still further projects a representation of the identified object to a location of the feature space and controls an action of the autonomous vehicle based on projecting the representation.

Abstract: System, methods, and other embodiments described herein relate to conditionally generating custom images by sampling latent space of a generator. In one embodiment, a method includes, in response to receiving a request to generate a custom image, generating a component instruction by translating a description about requested characteristics for the object instance into a vector that identifies a portion of a latent space within a respective generator. The method includes computing the object instance by controlling the respective one of the generators according to the component instruction to produce the object instance. The respective one of the generators being configured to generate objects within a semantic object class. The method includes generating the custom image from at least the object instance to produce the custom image from the description as a photorealistic image approximating a real image corresponding to the description.

Abstract: A method for training a machine learning model includes receiving real data comprising a real element in a real environment. The training also includes annotating the real element with a first annotation based on predicted attributes of the real element. The first annotation having a first format. The training further includes converting the first format of the first annotation to a second format corresponding to a ground truth annotation of the real element. The training still further includes adjusting parameters of the machine learning model to minimize a difference between values of the ground truth annotation of the real element and the converted first annotation.

Abstract: A method for inferring a location of a three-dimensional (3D) object, the method includes receiving sensor data from a plurality of sensors of an autonomous vehicle. The method also includes mapping features extracted from the sensor data to a first data structure corresponding to a feature space of a 3D representation of a spatial area. The method further includes classifying the merged features to identify an object within a vicinity of the autonomous vehicle and projecting a 3D representation of the identified object to a location of the 3D feature space. The method still further includes controlling an action of the autonomous vehicle based on the projected 3D representation.

Abstract: System, methods, and other embodiments described herein relate to improving the generation of realistic images. In one embodiment, a method includes acquiring a synthetic image including identified labels of simulated components within the synthetic image. The synthetic image is a simulated visualization and the identified labels distinguish between the components within the synthetic image. The method includes computing, from the simulated components, translated components that visually approximate real instances of the simulated components by using a generative module comprised of neural networks that are configured to separately generate the translated components. The method includes blending the translated components together to produce a new image from the simulated components of the synthetic image.

Abstract: An autonomous vehicle vision system for estimating a category of a detected object in an object pose unknown to the system includes a neural network to apply a mapping process to a region of interest in an image including the detected object in the object pose to obtain a point in a 3D manifold space. The system includes an object detector to estimate the category of the detected object in the object pose in the region of interest based on a relationship between the point representing the detected object in the object pose and a plurality of separate object clusters in the 3D manifold space. The system further includes a planner to select an improved route based on a predicted behavior of the category of the detected object in the object pose. The system also includes a controller to control operation of an autonomous vehicle according to the improved route.

Abstract: A method for full-stack verification of autonomous agents includes training a neural network to learn a noise model associated with an object detection module of an autonomous agent system of an autonomous vehicle. The method also includes replacing the object detection module of the autonomous agent system with the neural network and a sensory input of the object detection module with ground truth information to apply a surrogate function to the ground truth information. The method further includes verifying the autonomous agent system including the trained neural network to apply the surrogate function in response to the ground truth information to simulate sensor information data to at least a planner module of the autonomous agent system. The method also includes controlling a behavior of the autonomous vehicle using the verified autonomous agent system including the object detection module.

Abstract: A method for training a machine learning model includes receiving real data comprising a real element in a real environment. The training also includes annotating the real element with a first annotation based on predicted attributes of the real element. The first annotation having a first format. The training further includes converting the first format of the first annotation to a second format corresponding to a ground truth annotation of the real element. The training still further includes adjusting parameters of the machine learning model to minimize a difference between values of the ground truth annotation of the real element and the converted first annotation.

Abstract: A method for inferring a location of a three-dimensional (3D) object, the method includes receiving sensor data from a plurality of sensors of an autonomous vehicle. The method also includes mapping features extracted from the sensor data to a first data structure corresponding to a feature space of a 3D representation of a spatial area. The method further includes classifying the merged features to identify an object within a vicinity of the autonomous vehicle and projecting a 3D representation of the identified object to a location of the 3D feature space. The method still further includes controlling an action of the autonomous vehicle based on the projected 3D representation.

Abstract: System, methods, and other embodiments described herein relate to improving the generation of realistic images. In one embodiment, a method includes acquiring a synthetic image including identified labels of simulated components within the synthetic image. The synthetic image is a simulated visualization and the identified labels distinguish between the components within the synthetic image. The method includes computing, from the simulated components, translated components that visually approximate real instances of the simulated components by using a generative module comprised of neural networks that are configured to separately generate the translated components. The method includes blending the translated components together to produce a new image from the simulated components of the synthetic image.

Abstract: System, methods, and other embodiments described herein relate to conditionally generating custom images by sampling latent space of a generator. In one embodiment, a method includes, in response to receiving a request to generate a custom image, generating a component instruction by translating a description about requested characteristics for the object instance into a vector that identifies a portion of a latent space within a respective generator. The method includes computing the object instance by controlling the respective one of the generators according to the component instruction to produce the object instance. The respective one of the generators being configured to generate objects within a semantic object class. The method includes generating the custom image from at least the object instance to produce the custom image from the description as a photorealistic image approximating a real image corresponding to the description.

Abstract: An autonomous vehicle vision system for estimating a category of a detected object in an object pose unknown to the system includes a neural network to apply a mapping process to a region of interest in an image including the detected object in the object pose to obtain a point in a 3D manifold space. The system includes an object detector to estimate the category of the detected object in the object pose in the region of interest based on a relationship between the point representing the detected object in the object pose and a plurality of separate object clusters in the 3D manifold space. The system further includes a planner to select an improved route based on a predicted behavior of the category of the detected object in the object pose. The system also includes a controller to control operation of an autonomous vehicle according to the improved route.

Abstract: A method for full-stack verification of autonomous agents includes training a neural network to learn a noise model associated with an object detection module of an autonomous agent system of an autonomous vehicle. The method also includes replacing the object detection module of the autonomous agent system with the neural network and a sensory input of the object detection module with ground truth information to apply a surrogate function to the ground truth information. The method further includes verifying the autonomous agent system including the trained neural network to apply the surrogate function in response to the ground truth information to simulate sensor information data to at least a planner module of the autonomous agent system. The method also includes controlling a behavior of the autonomous vehicle using the verified autonomous agent system including the object detection module.

Abstract: System, methods, and other embodiments described herein relate to improving querying of a visual dataset of images through implementing system-aware cascades. In one embodiment, a method includes enumerating a set of cascade classifiers that are each separately comprised of transformation modules and machine learning modules arranged in multiple pairs. Classifiers of the set of cascade classifiers are configured to extract content from an image according to a query. The method includes selecting a query classifier from the set of cascade classifiers based, at least in part, on system costs that characterize computational resources consumed by the classifiers of the set of cascade classifiers. The computational resources include at least data handling costs. The method includes identifying content within the image using the query classifier.

Abstract: A source deconvolutional network is adaptively trained to perform semantic segmentation. Image data is then input to the source deconvolutional network and outputs of the S-Net are measured. The same image data and the measured outputs of the source deconvolutional network are then used to train a target deconvolutional network. The target deconvolutional network is defined by a substantially fewer numerical parameters than the source deconvolutional network.

Abstract: A source deconvolutional network is adaptively trained to perform semantic segmentation. Image data is then input to the source deconvolutional network and outputs of the S-Net are measured. The same image data and the measured outputs of the source deconvolutional network are then used to train a target deconvolutional network. The target deconvolutional network is defined by a substantially fewer numerical parameters than the source deconvolutional network.