Abstract: The present disclosure provides architectures and techniques for absolute depth estimation from a single (e.g., monocular) image, using online depth scale transfer. For instance, estimation of up-to-scale depth maps from monocular images may be decoupled from estimation of the depth scale (e.g., such that additional online measurements, additional calibrations, etc. are not required). One or more aspects of the present disclosure include fine-tuning or training from scratch an absolute depth estimator using collected monocular images, as well as existing images and absolute depth measurements (e.g., from additional setups, such as LiDAR/stereo sensors). Collected monocular images may be used to create up-to-scale depth maps, and existing images and absolute depth measurements may be used to estimate the scale of a scene from the up-to-scale depth map. Scale transfer may thus be achieved between source images with known ground truth depth information and a new target domain of collected monocular images.

Abstract: Image processing systems and image processing techniques leveraging neural networks (e.g., convolutional neural networks (CNNs)) for image restoration tasks (e.g., for demosaicing tasks) are described. In certain aspects, Mixture of Experts (MoE) techniques may be employed, where multiple different expert networks are used to divide a problem space (e.g., image reconstruction tasks) into homogenous regions. For example, each MoE module may reconstruct a certain problem in an image, and a gating component may activate certain MoE modules to provide a reconstructed image. In some aspects, training and optimization techniques are described for each expert of the MoE architecture, to increase individual performance (e.g., a sub-task for each expert of an image processing system may be imposed in a residual manner, a gating function may be trained, etc.).

Abstract: An object detection system may generate regions of interest (ROIs) from an input image that can be processed by a wide range of object detectors. According to the techniques described herein, an image is processed by a light-weight neural network (e.g., a heatmap network) that outputs object center and object scale heat-maps. The heatmaps are processed to define ROIs that are likely to include objects. Overlapping ROIs are then merged to reduce the aggregate size of the ROIs, and merged ROIs are downscaled to a reduced set of pre-defined resolutions. Fully-convolutional, high-accuracy object detectors may then operate on the downscaled ROIs to output accurate detections at a fraction of the computations by operating on a reduced image. For example, fully-convolutional, high-accuracy object detectors may operate on a subset of the entire image (e.g., cropped images based on ROIs) thus reducing computations otherwise performed over the entire image.

Abstract: An object detection system may generate regions of interest (ROIs) from an input image that can be processed by a wide range of object detectors. According to the techniques described herein, an image is processed by a light-weight neural network (e.g., a heatmap network) that outputs object center and object scale heat-maps. The heatmaps are processed to define ROIs that are likely to include objects. Overlapping ROIs are then merged to reduce the aggregate size of the ROIs, and merged ROIs are downscaled to a reduced set of pre-defined resolutions. Fully-convolutional, high-accuracy object detectors may then operate on the downscaled ROIs to output accurate detections at a fraction of the computations by operating on a reduced image. For example, fully-convolutional, high-accuracy object detectors may operate on a subset of the entire image (e.g., cropped images based on ROIs) thus reducing computations otherwise performed over the entire image.

Abstract: A system of convolutional neural networks (CNNs) that synthesize middle non-existing frames from pairs of input frames includes a coarse CNN that receives a pair of images acquired at consecutive points of time, a registration module, a refinement CNN, an adder, and a motion-compensated frame interpolation (MC-FI) module. The coarse CNN outputs from the pair of images a previous feature map, a next feature map, a coarse interpolated motion vector field (IMVF) and an occlusion map, the registration module uses the coarse IMVF to warp the previous and next feature maps to be aligned with pixel locations of the IMVF frame, and outputs registered previous and next feature maps, the refinement CNN uses the registered previous and next feature maps to correct the coarse IMVF, and the adder sums the coarse IMVF with the correction and outputs a final IMVF.

Abstract: A system of convolutional neural networks (CNNs) that synthesize middle non-existing frames from pairs of input frames includes a coarse CNN that receives a pair of images acquired at consecutive points of time, a registration module, a refinement CNN, an adder, and a motion-compensated frame interpolation (MC-FI) module. The coarse CNN outputs from the pair of images a previous feature map, a next feature map, a coarse interpolated motion vector field (IMVF) and an occlusion map, the registration module uses the coarse IMVF to warp the previous and next feature maps to be aligned with pixel locations of the IMVF frame, and outputs registered previous and next feature maps, the refinement CNN uses the registered previous and next feature maps to correct the coarse IMVF, and the adder sums the coarse IMVF with the correction and outputs a final IMVF.

Abstract: A method of matching stereo images includes: determining, first information indicating which orientations are dominant in each pixel location of a right image and a left image among the stereo images; determining second information indicating for each pixel location in the right image and the left image whether it lies on a pure horizontal edge; detecting points in the left image and the right image based on the first information and the second information; and performing a matching on the left image and the right image using the detected points to estimate a sparse disparity map.

Abstract: A method of matching stereo images includes: determining, first information indicating which orientations are dominant in each pixel location of a right image and a left image among the stereo images; determining second information indicating for each pixel location in the right image and the left image whether it lies on a pure horizontal edge; detecting points in the left image and the right image based on the first information and the second information; and performing a matching on the left image and the right image using the detected points to estimate a sparse disparity map.