Abstract: Methods and systems for performing video prediction, including obtaining an input frame from among a plurality of frames included in an input video; extracting a first feature map by providing the input frame to a first plurality of feature extraction layers and a first strided convolutional layer included in an encoder; providing the first feature map and at least one neighboring first feature map corresponding to at least one neighboring frame to a first fusion module included in the encoder; fusing the first feature map with the at least one neighboring first feature map to generate a fused first feature map using the first fusion module; generating a prediction corresponding to the input frame based on the fused first feature map using a decoder; and performing a video prediction task using the prediction.

Abstract: A method of depth detection based on a plurality of video frames includes receiving a plurality of input frames including a first input frame, a second input frame, and a third input frame respectively corresponding to different capture times, convolving the first to third input frames to generate a first feature map, a second feature map, and a third feature map corresponding to the different capture times, calculating a temporal attention map based on the first to third feature maps, the temporal attention map including a plurality of weights corresponding to different pairs of feature maps from among the first to third feature maps, each weight of the plurality of weights indicating a similarity level of a corresponding pair of feature maps, and applying the temporal attention map to the first to third feature maps to generate a feature map with temporal attention.

Abstract: A computer vision (CV) training system, includes: a supervised learning system to estimate a supervision output from one or more input images according to a target CV application, and to determine a supervised loss according to the supervision output and a ground-truth of the supervision output; an unsupervised learning system to determine an unsupervised loss according to the supervision output and the one or more input images; a weakly supervised learning system to determine a weakly supervised loss according to the supervision output and a weak label corresponding to the one or more input images; and a joint optimizer to concurrently optimize the supervised loss, the unsupervised loss, and the weakly supervised loss.

Abstract: A method of depth detection based on a plurality of video frames includes receiving a plurality of input frames including a first input frame, a second input frame, and a third input frame respectively corresponding to different capture times, convolving the first to third input frames to generate a first feature map, a second feature map, and a third feature map corresponding to the different capture times, calculating a temporal attention map based on the first to third feature maps, the temporal attention map including a plurality of weights corresponding to different pairs of feature maps from among the first to third feature maps, each weight of the plurality of weights indicating a similarity level of a corresponding pair of feature maps, and applying the temporal attention map to the first to third feature maps to generate a feature map with temporal attention.

Abstract: Apparatuses and methods of manufacturing same, systems, and methods for generating a convolutional neural network (CNN) are described. In one aspect, a minimal CNN having, e.g., three or more layers is trained. Cascade training may be performed on the trained CNN to insert one or more intermediate layers until a training error is less than a threshold. When cascade training is complete, cascade network trimming of the CNN output from the cascade training may be performed to improve computational efficiency. To further reduce network parameters, convolutional filters may be replaced with dilated convolutional filters with the same receptive field, followed by additional training/fine-tuning.

Abstract: Method and systems are provided for robust disparity estimation based on cost-volume attention. A method includes extracting first feature maps from left images captured by a first camera; extracting second feature maps from right images captured by a second camera; calculating a matching cost based on a comparison of the first and second feature maps to generate a cost volume; generating an attention-aware cost volume from the generated cost volume; and aggregating the attention-aware cost volume to generate an output disparity.

Abstract: A method and apparatus are provided. The method includes generating a dataset for real-world super resolution (SR), training a first generative adversarial network (GAN), training a second GAN, and fusing an output of the first GAN and an output of the second GAN.

Abstract: A method and apparatus for providing a rotational invariant neural network is herein disclosed. According to one embodiment, a method includes receiving a first input of an image in a first orientation and training a kernel to be symmetric such that an output corresponding to the first input is the same as an output corresponding to a second input of the image in a second orientation.

Abstract: In a method for super resolution imaging, the method includes: receiving, by a processor, a low resolution image; generating, by the processor, an intermediate high resolution image having an improved resolution compared to the low resolution image; generating, by the processor, a final high resolution image based on the intermediate high resolution image and the low resolution image; and transmitting, by the processor, the final high resolution image to a display device for display thereby.

Abstract: A method of depth detection based on a plurality of video frames includes receiving a plurality of input frames including a first input frame, a second input frame, and a third input frame respectively corresponding to different capture times, convolving the first to third input frames to generate a first feature map, a second feature map, and a third feature map corresponding to the different capture times, calculating a temporal attention map based on the first to third feature maps, the temporal attention map including a plurality of weights corresponding to different pairs of feature maps from among the first to third feature maps, each weight of the plurality of weights indicating a similarity level of a corresponding pair of feature maps, and applying the temporal attention map to the first to third feature maps to generate a feature map with temporal attention.

Abstract: A method and an apparatus are provided. The method includes receiving a video with a first plurality of frames having a first resolution; generating a plurality of warped frames from the first plurality of frames based on a first type of motion compensation; generating a second plurality of frames having a second resolution, wherein the second resolution is of higher resolution than the first resolution, wherein each of the second plurality of frames having the second resolution is derived from a subset of the plurality of warped frames using a convolutional network; and generating a third plurality of frames having the second resolution based on a second type of motion compensation, wherein each of the third plurality of frames having the second resolution is derived from a fusing a subset of the second plurality of frames.

Abstract: A method of depth detection based on a plurality of video frames includes receiving a plurality of input frames including a first input frame, a second input frame, and a third input frame respectively corresponding to different capture times, convolving the first to third input frames to generate a first feature map, a second feature map, and a third feature map corresponding to the different capture times, calculating a temporal attention map based on the first to third feature maps, the temporal attention map including a plurality of weights corresponding to different pairs of feature maps from among the first to third feature maps, each weight of the plurality of weights indicating a similarity level of a corresponding pair of feature maps, and applying the temporal attention map to the first to third feature maps to generate a feature map with temporal attention.

Abstract: A computer vision (CV) training system, includes: a supervised learning system to estimate a supervision output from one or more input images according to a target CV application, and to determine a supervised loss according to the supervision output and a ground-truth of the supervision output; an unsupervised learning system to determine an unsupervised loss according to the supervision output and the one or more input images; a weakly supervised learning system to determine a weakly supervised loss according to the supervision output and a weak label corresponding to the one or more input images; and a joint optimizer to concurrently optimize the supervised loss, the unsupervised loss, and the weakly supervised loss.

Abstract: Various embodiments of the teachings herein include methods for amending or adding machine learning capabilities to an automation device in an automation system. The method may include: 1) providing a capability model of the automation device semantically representing capabilities of the device; 2) providing a machine learning model for semantically representing a machine learning functionality and including a semantic model of a neural network; 3) deploying the machine learning model within the automation device; 4) interpreting a semantic part of the machine learning model using a semantic reasoner and matching requirements of the machine learning model with device capabilities inferred by the capability model; and 5) executing the machine learning functionalities on the automation device.

Abstract: Apparatuses and methods of manufacturing same, systems, and methods are described. In one aspect, a method includes generating a convolutional neural network (CNN) by training a CNN having a plurality of convolutional layers, and performing cascade training on the trained CNN. The cascade training includes an iterative process of a plurality of stages, in which each stage includes inserting a residual block (ResBlock) and training the CNN with the inserted ResBlock.

Abstract: A computer vision (CV) training system, includes: a supervised learning system to estimate a supervision output from one or more input images according to a target CV application, and to determine a supervised loss according to the supervision output and a ground-truth of the supervision output; an unsupervised learning system to determine an unsupervised loss according to the supervision output and the one or more input images; a weakly supervised learning system to determine a weakly supervised loss according to the supervision output and a weak label corresponding to the one or more input images; and a joint optimizer to concurrently optimize the supervised loss, the unsupervised loss, and the weakly supervised loss.

Abstract: Apparatuses and methods of manufacturing same, systems, and methods are described. In one aspect, a method includes generating a convolutional neural network (CNN) by training a CNN having three or more convolutional layers, and performing cascade training on the trained CNN. The cascade training includes an iterative process of one or more stages, in which each stage includes inserting a residual block (ResBlock) including at least two additional convolutional layers and training the CNN with the inserted ResBlock.

Abstract: Method and systems are provided for robust disparity estimation based on cost-volume attention. A method includes extracting first feature maps from left images captured by a first camera; extracting second feature maps from right images captured by a second camera; calculating a matching cost based on a comparison of the first and second feature maps to generate a cost volume; generating an attention-aware cost volume from the generated cost volume; and aggregating the attention-aware cost volume to generate an output disparity.

Abstract: A method and apparatus are provided. The method includes generating a dataset for real-world super resolution (SR), training a first generative adversarial network (GAN), training a second GAN, and fusing an output of the first GAN and an output of the second GAN.

Abstract: A method and system for determining depth information of an image are herein provided. According to one embodiment, the method includes receiving an image input, classifying the input image into a depth range of a plurality of depth ranges, and determining a depth map of the image by applying depth estimation based on the depth range into which the input image is classified.