Abstract: A system includes a processor for performing one or more autonomous driving or assisted driving tasks based on a neural network. The neural network includes a base portion for performing feature extraction simultaneously for a plurality of tasks on a single set of input data. The neural network includes a plurality of subtask portions for performing the plurality of tasks based on feature extraction output from the base portion. Each of the plurality of subtask portions comprise nodes or layers of a neutral network trained on different sets of training data, and the base portion comprises nodes or layers of a neural network trained using each of the different sets of training data constrained by elastic weight consolidation to limit the base portion from forgetting a previously learned task.

Abstract: Systems and methods for generating a panoptic segmentation mask for an input image. The methods include receiving the input image comprising a plurality of pixels, generating a semantic mask and an instance mask from the input image, and combining the semantic mask and the instance mask to generate a panoptic mask for the input image. The semantic mask includes a single-channel mask that associates each pixel in the input image with a corresponding one of a plurality of labels. The instance mask includes a plurality of masks, where each of the plurality of masks identifies an instance of a countable object in the input image, and is associated with an indication of whether that instance of the countable object is hidden behind another object in the input image.

Abstract: Systems, methods, and devices for detecting brake lights are disclosed herein. A system includes a mode component, a vehicle region component, and a classification component. The mode component is configured to select a night mode or day mode based on a pixel brightness in an image frame. The vehicle region component is configured to detect a region corresponding to a vehicle based on data from a range sensor when in a night mode or based on camera image data when in the day mode. The classification component is configured to classify a brake light of the vehicle as on or off based on image data in the region corresponding to the vehicle.

Abstract: Methods and systems for detecting objects near an autonomous vehicle (AV) are disclosed. An AV will capture an image. A trained network will process the image at a lower resolution and generate a first feature map that classifies object(s) within the image. The system will crop the image and use the network to process the cropped section at a higher resolution to generate a second feature map that classifies object(s) that appear within the cropped section. The system will crop the first feature map to match a corresponding region of the cropped section of the image. The system will fuse the cropped first and second feature maps to generate a third feature map. The system may output the object classifications in the third feature map to an AV system, such as a motion planning system that will use the object classifications to plan a trajectory for the AV.

Abstract: Systems, methods, and devices for detecting brake lights are disclosed herein. A system includes a mode component, a vehicle region component, and a classification component. The mode component is configured to select a night mode or day mode based on a pixel brightness in an image frame. The vehicle region component is configured to detect a region corresponding to a vehicle based on data from a range sensor when in a night mode or based on camera image data when in the day mode. The classification component is configured to classify a brake light of the vehicle as on or off based on image data in the region corresponding to the vehicle.

Abstract: Systems and methods for image-based perception. The methods comprise: capturing images by a plurality of cameras with overlapping fields of view; generating, by a computing device, spatial feature maps indicating locations of features in the images; identifying, by the computing device, overlapping portions of the spatial feature maps; generating, by the computing device, at least one combined spatial feature map by combining the overlapping portions of the spatial feature maps together; and/or using, by the computing device, the at least one combined spatial feature map to define a predicted cuboid for at least one object in the images.

Abstract: This document discloses methods of training a classifier to identify traffic signal states in images captured be a vehicle. The vehicle can then use the identified states when making movement decisions when traveling in an environment. The system determines that a traffic signal is within a field of view of the camera (i.e., within an image). The system also receives a signal with signal phase and timing data for the traffic signal. The system processes the images to identify an image that includes the traffic signal. The system analyzes the signal data to determine a state of the traffic signal at the time of image capture, labels the image with a label of determined state, and passes the image and a label to a classifier in order to train the classifier.

Abstract: This document discloses system, method, and computer program product embodiments for image-based pedestrian speed estimation. For example, the method includes receiving an image of a scene, wherein the image includes a pedestrian and predicting a speed of the pedestrian by applying a machine-learning model to at least a portion of the image that includes the pedestrian. The machine-learning model is trained using a data set including training images of pedestrians, the training images associated with corresponding known pedestrian speeds. The method further includes providing the predicted speed of the pedestrian to a motion-planning system that is configured to control a trajectory of an autonomous vehicle in the scene.

Abstract: A system includes a processor for performing one or more autonomous driving or assisted driving tasks based on a neural network. The neural network includes a base portion for performing feature extraction simultaneously for a plurality of tasks on a single set of input data. The neural network includes a plurality of subtask portions for performing the plurality of tasks based on feature extraction output from the base portion. Each of the plurality of subtask portions comprise nodes or layers of a neutral network trained on different sets of training data, and the base portion comprises nodes or layers of a neural network trained using each of the different sets of training data constrained by elastic weight consolidation to limit the base portion from forgetting a previously learned task.

Abstract: Systems, methods, and devices for detecting brake lights are disclosed herein. A system includes a mode component, a vehicle region component, and a classification component. The mode component is configured to select a night mode or day mode based on a pixel brightness in an image frame. The vehicle region component is configured to detect a region corresponding to a vehicle based on data from a range sensor when in a night mode or based on camera image data when in the day mode. The classification component is configured to classify a brake light of the vehicle as on or off based on image data in the region corresponding to the vehicle.

Abstract: A system includes a processor for performing one or more autonomous driving or assisted driving tasks based on a neural network. The neural network includes a base portion for performing feature extraction simultaneously for a plurality of tasks on a single set of input data. The neural network includes a plurality of subtask portions for performing the plurality of tasks based on feature extraction output from the base portion. Each of the plurality of subtask portions comprise nodes or layers of a neutral network trained on different sets of training data, and the base portion comprises nodes or layers of a neural network trained using each of the different sets of training data constrained by elastic weight consolidation to limit the base portion from forgetting a previously learned task.

Abstract: Systems and methods for generating a panoptic segmentation mask for an input image. The methods include receiving the input image comprising a plurality of pixels, generating a semantic mask and an instance mask from the input image, and combining the semantic mask and the instance mask to generate a panoptic mask for the input image. The semantic mask includes a single-channel mask that associates each pixel in the input image with a corresponding one of a plurality of labels. The instance mask includes a plurality of masks, where each of the plurality of masks identifies an instance of a countable object in the input image, and is associated with an indication of whether that instance of the countable object is hidden behind another object in the input image.

Abstract: A method is disclosed for evaluating a classifier used to determine a traffic light signal state in images. The method includes, by a computer vision system of a vehicle, receiving at least one image of a traffic signal device of an imminent intersection. The traffic signal device includes a traffic signal face including one or more traffic signal elements. The method includes classifying, by a traffic light classifier (TLC), a classification state of the traffic signal face using labeled images correlated to the received at least one image. The classification state controls an operation of the vehicle at the intersection. The method includes evaluating a performance of the classifying of the classification state generated by the TLC. The evaluation is a label-free performance evaluation based on unlabeled images. The method includes training the TLC based on the evaluated performance.

Abstract: Systems, methods, and devices for detecting brake lights are disclosed herein. A system includes a mode component, a vehicle region component, and a classification component. The mode component is configured to select a night mode or day mode based on a pixel brightness in an image frame. The vehicle region component is configured to detect a region corresponding to a vehicle based on data from a range sensor when in a night mode or based on camera image data when in the day mode. The classification component is configured to classify a brake light of the vehicle as on or off based on image data in the region corresponding to the vehicle.

Abstract: Systems and methods for image-based perception. The methods comprise: obtaining, by a computing device, images captured by a plurality of cameras with overlapping fields of view; generating, by the computing device, spatial feature maps indicating locations of features in the images; defining, by the computing device, predicted cuboids at each location of an object in the images based on the spatial feature maps; and assigning, by the computing device, at least two cuboids of said predicted cuboids to a given object when predictions from images captured by separate cameras of the plurality of cameras should be associated with a same detected object.

Abstract: A method and a system for generating a mesh representation of a surface. The method includes receiving a three-dimensional (3D) point cloud representing the surface, generating a reconstruction dataset having a higher resolution than the 3D point cloud in one or more regions corresponding to the surface from the 3D point cloud, and generate a polygon mesh representation of the surface by using a fine-to-coarse hash map for building polygons at a highest resolution first followed by progressively coarser resolution polygons, using the reconstruction dataset.

Abstract: Methods and systems for detecting objects near an autonomous vehicle (AV) are disclosed. An AV will capture an image. A trained network will process the image at a lower resolution and generate a first feature map that classifies object(s) within the image. The system will crop the image and use the network to process the cropped section at a higher resolution to generate a second feature map that classifies object(s) that appear within the cropped section. The system will crop the first feature map to match a corresponding region of the cropped section of the image. The system will fuse the cropped first and second feature maps to generate a third feature map. The system may output the object classifications in the third feature map to an AV system, such as a motion planning system that will use the object classifications to plan a trajectory for the AV.

Abstract: Systems and methods for image-based perception. The methods comprise: capturing images by a plurality of cameras with overlapping fields of view; generating, by a computing device, spatial feature maps indicating locations of features in the images; identifying, by the computing device, overlapping portions of the spatial feature maps; generating, by the computing device, at least one combined spatial feature map by combining the overlapping portions of the spatial feature maps together; and/or using, by the computing device, the at least one combined spatial feature map to define a predicted cuboid for at least one object in the images.

Abstract: Systems and methods for image-based perception. The methods comprise: obtaining, by a computing device, images captured by a plurality of cameras with overlapping fields of view; generating, by the computing device, spatial feature maps indicating locations of features in the images; defining, by the computing device, predicted cuboids at each location of an object in the images based on the spatial feature maps; and assigning, by the computing device, at least two cuboids of said predicted cuboids to a given object when predictions from images captured by separate cameras of the plurality of cameras should be associated with a same detected object.

Abstract: Systems, methods, and devices for detecting brake lights are disclosed herein. A system includes a mode component, a vehicle region component, and a classification component. The mode component is configured to select a night mode or day mode based on a pixel brightness in an image frame. The vehicle region component is configured to detect a region corresponding to a vehicle based on data from a range sensor when in a night mode or based on camera image data when in the day mode. The classification component is configured to classify a brake light of the vehicle as on or off based on image data in the region corresponding to the vehicle.