Abstract: System, methods, and other embodiments described herein relate to self-supervised training for monocular depth estimation. In one embodiment, a method includes filtering disfavored images from first training data to produce second training data that is a subsampled version of the first training data. The disfavored images correspond with anomaly maps within a set of depth maps. The first depth model is trained according to the first training data and generates the depth maps from the first training data after initially being trained with the first training data. The method includes training a second depth model according to a self-supervised training process using the second training data. The method includes providing the second depth model to infer distances from monocular images.

Abstract: System, methods, and other embodiments described herein relate to generating depth estimates of an environment depicted in a monocular image. In one embodiment, a method includes identifying semantic features in the monocular image according to a semantic model. The method includes injecting the semantic features into a depth model using pixel-adaptive convolutions. The method includes generating a depth map from the monocular image using the depth model that is guided by the semantic features. The pixel-adaptive convolutions are integrated into a decoder of the depth model. The method includes providing the depth map as the depth estimates for the monocular image.

Abstract: System, methods, and other embodiments described herein relate to semi-supervised training of a depth model using a neural camera model that is independent of a camera type. In one embodiment, a method includes acquiring training data including at least a pair of training images and depth data associated with the training images. The method includes training the depth model using the training data to generate a self-supervised loss from the pair of training images and a supervised loss from the depth data. Training the depth model includes learning the camera type by generating, using a ray surface model, a ray surface that approximates an image character of the training images as produced by a camera having the camera type. The method includes providing the depth model to infer depths from monocular images in a device.

Abstract: System, methods, and other embodiments described herein relate to estimating ego-motion. In one embodiment, a method for estimating ego-motion based on a plurality of input images in a self-supervised system includes receiving a source image and a target image, determining a depth estimation Dt based on the target image, determining a depth estimation Ds based on a source image, and determining an ego-motion estimation in a form of a six degrees-of-freedom (6 DOF) transformation between the target image and the source image by inputting the depth estimations (Dt, Ds), the target image, and the source image into a two-stream network architecture trained to output the 6 DOF transformation based at least in part on the depth estimations (Dt, Ds), the target image, and the source image.

Abstract: System, methods, and other embodiments described herein relate to semi-supervised training of a depth model for monocular depth estimation. In one embodiment, a method includes training the depth model according to a first stage that is self-supervised and that includes using first training data that comprises pairs of training images. Respective ones of the pairs including separate frames depicting a scene of a monocular video. The method includes training the depth model according to a second stage that is weakly supervised and that includes using second training data to produce depth maps according to the depth model. The second training data comprising individual images with corresponding sparse depth data. The second training data providing for updating the depth model according to second stage loss values that are based, at least in part, on the depth maps and the depth data.

Abstract: System, methods, and other embodiments described herein relate to training a depth model for monocular depth estimation. In one embodiment, a method includes generating, as part of training the depth model according to a supervised training stage, a depth map from a first image of a pair of training images using the depth model. The pair of training images are separate frames depicting a scene from a monocular video. The method includes generating a transformation from the first image and a second image of the pair using a pose model. The method includes computing a supervised loss based, at least in part, on reprojecting the depth map and training depth data onto an image space of the second image according to at least the transformation. The method includes updating the depth model and the pose model according to at least the supervised loss.

Abstract: A method for controlling a vehicle based on a panoptic map includes receiving an input from at least one sensor of the vehicle. The method also includes generating an instance map and a semantic map from the input. The method further includes generating the panoptic map from the instance map and the semantic map based on a binary mask. The method still further includes controlling the vehicle based on the panoptic map.

Abstract: Acquiring labeled data can be a significant bottleneck in the development of machine learning models that are accurate and efficient enough to enable safety-critical applications, such as automated driving. The process of labeling of driving logs can be automated. Unlabeled real-world driving logs, which include data captured by one or more vehicle sensors, can be automatically labeled to generate one or more labeled real-world driving logs. The automatic labeling can include analysis-by-synthesis on the unlabeled real-world driving logs to generate simulated driving logs, which can include reconstructed driving scenes or portions thereof. The automatic labeling can further include simulation-to-real automatic labeling on the simulated driving logs and the unlabeled real-world driving logs to generate one or more labeled real-world driving logs. The automatically labeled real-world driving logs can be stored in one or more data stores for subsequent training, validation, evaluation, and/or model management.

Abstract: System, methods, and other embodiments described herein relate to generating depth estimates from a monocular image. In one embodiment, a method includes, in response to receiving the monocular image, flipping, by a disparity model, the monocular image to generate a flipped image. The disparity model is a machine learning algorithm. The method includes analyzing, using the disparity model, the monocular image and the flipped image to generate disparity maps including a monocular disparity map corresponding to the monocular image and a flipped disparity map corresponding with the flipped image. The method includes generating, in the disparity model, a fused disparity map from the monocular disparity map and the flipped disparity map. The method includes providing the fused disparity map as the depth estimates of objects represented in the monocular image.

Abstract: A method for controlling a vehicle based on a panoptic map includes receiving an input from at least one sensor of the vehicle. The method also includes generating an instance map and a semantic map from the input. The method further includes generating the panoptic map from the instance map and the semantic map based on a binary mask. The method still further includes controlling the vehicle based on the panoptic map.