Abstract: Systems, methods, and other embodiments associated with contribution metric-based pruning of a neural network are described. In one embodiment, an example method includes accessing a trained neural network that has a plurality of channels. The neural network is to be evaluated for pruning of the channels. The example method may also include determining contribution metrics for the channels by measuring changes in error of the convolutional neural network with individual channels removed in turn. The contribution metrics are determined based at least in part on higher order analysis of the changes. And, the example method may also include pruning out of the convolutional neural network a set of the channels for which the contribution metrics do not satisfy a threshold.

Abstract: Embodiments of this disclosure provide techniques used in a location sensing service and a timeline service implemented in a host device for providing location history. In particular, the location sensing service may passively collect location coordinates of the device using requests made by third party applications. The location sensing service may receive a signal comprising an intent for a location request in response to a change in the user activity or in response to a change in a cellular identification of the device. The device may determine that a time elapsed from a previous location request and a time corresponding with receiving one or more of the signals exceed a time threshold and, based thereon, request a location coordinate of the device.

Abstract: Deep neural networks (DNN) model quantization may be used to reduce storage and computation burdens by decreasing the bit width. Presented herein are novel cursor-based adaptive quantization embodiments. In embodiments, a multiple bits quantization mechanism is formulated as a differentiable architecture search (DAS) process with a continuous cursor that represents a possible quantization bit. In embodiments, the cursor-based DAS adaptively searches for a quantization bit for each layer. The DAS process may be accelerated via an alternative approximate optimization process, which is designed for mixed quantization scheme of a DNN model. In embodiments, a new loss function is used in the search process to simultaneously optimize accuracy and parameter size of the model. In a quantization step, the closest two integers to the cursor may be adopted as the bits to quantize the DNN together to reduce the quantization noise and avoid the local convergence problem.

Abstract: Image super-resolution (SR) refers to the process of recovering high-resolution (HR) images from low-resolution (LR) inputs. Blind image SR is a more challenging task which involves unknown blurring kernels and characterizes the degradation process from HR to LR. In the present disclosure, embodiments of a variational autoencoder (VAE) are leveraged to train a kernel autoencoder for more accurate degradation representation and more efficient kernel estimation. In one or more embodiments, a kernel-agnostic loss is used to learn more robust kernel features in the latent space from LR inputs without using ground-truth kernel references. In addition, attention-based adaptive pooling is introduced to improve kernel estimation accuracy, and spatially non-uniform kernel features are passed into SR restoration resulting in additional kernel estimation error tolerance.

Abstract: Described herein are embodiments of a deep residual network dedicated to color filter array mosaic patterns. A mosaic stride convolution layer is introduced to match the mosaic pattern of a multispectral filter arrays (MSFA) or a color filter array raw image. Embodiments of a data augmentation using MSFA shifting and dynamic noise are applied to make the model robust to different noise levels. Embodiments of network optimization criteria may be created by using the noise standard deviation to normalize the L1 loss function. Comprehensive experiments demonstrate that embodiments of the disclosed deep residual network outperform the state-of-the-art denoising algorithms in MSFA field.

Abstract: A method for information processing for accelerating neural network training. The method includes: acquiring a neural network corresponding to a deep learning task; and performing iterations of iterative training on the neural network based on a training data set. The training data set includes task data corresponding to the deep learning task. The iterative training includes: processing the task data in the training data set using a current neural network, and determining, based on a processing result of the neural network on the task data in a current iterative training, prediction loss of the current iterative training; determining a learning rate and a momentum in the current iterative training; and updating weight parameters of the current neural network by gradient descent based on a preset weight decay, and the learning rate, the momentum, and the prediction loss in the current iterative training. This method achieves efficient and low-cost deep learning-based neural network training.

Abstract: Model pruning is used to trim large neural networks, like convolutional neural networks (CNNs), to reduce computation overheads. Existing model pruning methods mainly rely on heuristics rules or local relationships of CNN layers. A novel hypernetwork based on graph neural network is disclosed for generating and evaluating pruned networks. A graph is first constructed according to information flow of channels and layers in a CNN network, with channels and layers represented as nodes and information flows represented as edges. A graph neural network is applied to aggregate both local and global dependencies across all channels and layers of the CNN network, resulting in informative node embeddings. With such embeddings, pruned CNN networks including their architectures and weights may be effectively generated and evaluated.

Abstract: Described herein are systems and embodiments for multispectral image demosaicking using deep panchromatic image guided residual interpolation. Embodiments of a ResNet-based deep learning model are disclosed to reconstruct the full-resolution panchromatic image from multispectral filter array (MSFA) mosaic image. In one or more embodiments, the reconstructed deep panchromatic image (DPI) is deployed as the guide to recover the full-resolution multispectral image using a two-pass guided residual interpolation methodology. Experiment results demonstrate that the disclosed method embodiments outperform some state-of-the-art conventional and deep learning demosaicking methods both qualitatively and quantitatively.

Abstract: Network architecture search (NAS) received a lot of attention. The supernet-based differentiable approach is popular because it can effectively share the weights and lead to more efficient search. However, the mismatch between the architecture and weights caused by weight sharing still exists. Moreover, the coupling effects among different operators are also neglected. To alleviate these problems, embodiments of an effective NAS methodology by similarity-based operator ranking are presented herein. With the aim of approximating each layer's output in the supernet, a similarity-based operator ranking based on statistical random comparison is used. In one or more embodiments, then the operator that possibly causes the least change to feature distribution discrepancy is pruned. In one or more embodiments, a fair sampling process may be used to mitigate the operators' Matthew effect that happened frequently in previous supernet approaches.

Abstract: Presented herein are embodiments of a training deep learning models. In one or more embodiments, a compact deep learning model comprises fewer layers, which require fewer floating-point operations (FLOPs). Presented herein are also embodiments of a new learning rate function, which can adaptively change the learning rate between two linear functions. In one or more embodiments, combinations of half-precision floating point format training together with larger batch size in the training process may also be employed to aid the training process.

Abstract: Embodiments herein treat the action segmentation as a domain adaption (DA) problem and reduce the domain discrepancy by performing unsupervised DA with auxiliary unlabeled videos. In one or more embodiments, to reduce domain discrepancy for both the spatial and temporal directions, embodiments of a Mixed Temporal Domain Adaptation (MTDA) approach are presented to jointly align frame-level and video-level embedded feature spaces across domains, and, in one or more embodiments, further integrate with a domain attention mechanism to focus on aligning the frame-level features with higher domain discrepancy, leading to more effective domain adaptation. Comprehensive experiment results validate that embodiments outperform previous state-of-the-art methods. Embodiments can adapt models effectively by using auxiliary unlabeled videos, leading to further applications of large-scale problems, such as video surveillance and human activity analysis.

Abstract: Described herein are embodiments of a deep residual network dedicated to color filter array mosaic patterns. A mosaic stride convolution layer is introduced to match the mosaic pattern of a multispectral filter arrays (MSFA) or a color filter array raw image. Embodiments of a data augmentation using MSFA shifting and dynamic noise are applied to make the model robust to different noise levels. Embodiments of network optimization criteria may be created by using the noise standard deviation to normalize the L1 loss function. Comprehensive experiments demonstrate that embodiments of the disclosed deep residual network outperform the state-of-the-art denoising algorithms in MSFA field.

Abstract: Tensor decomposition can be advantageous for compressing deep neural networks (DNNs). In many applications of DNNs, reducing the number of parameters and computation workload is helpful to accelerate inference speed in deployment. Modern DNNs comprise multiple layers with multi-array weights where tensor decomposition is a natural way to perform compression—in which the weight tensors in convolutional layers or fully-connected layers are decomposed with specified tensor ranks (e.g., canonical ranks, tensor train ranks). Conventional tensor decomposition with DNNs involves selecting ranks manually, which requires tedious human efforts to finetune the performance. Accordingly, presented herein are rank selection embodiments, which are inspired by reinforcement learning, to automatically select ranks in tensor decomposition.

Abstract: Described herein are systems and embodiments for multispectral image demosaicking using deep panchromatic image guided residual interpolation. Embodiments of a ResNet-based deep learning model are disclosed to reconstruct the full-resolution panchromatic image from multispectral filter array (MSFA) mosaic image. In one or more embodiments, the reconstructed deep panchromatic image (DPI) is deployed as the guide to recover the full-resolution multispectral image using a two-pass guided residual interpolation methodology. Experiment results demonstrate that the disclosed method embodiments outperform some state-of-the-art conventional and deep learning demosaicking methods both qualitatively and quantitatively.

Abstract: Deep neural networks (DNN) model quantization may be used to reduce storage and computation burdens by decreasing the bit width. Presented herein are novel cursor-based adaptive quantization embodiments. In embodiments, a multiple bits quantization mechanism is formulated as a differentiable architecture search (DAS) process with a continuous cursor that represents a possible quantization bit. In embodiments, the cursor-based DAS adaptively searches for a quantization bit for each layer. The DAS process may be accelerated via an alternative approximate optimization process, which is designed for mixed quantization scheme of a DNN model. In embodiments, a new loss function is used in the search process to simultaneously optimize accuracy and parameter size of the model. In a quantization step, the closest two integers to the cursor may be adopted as the bits to quantize the DNN together to reduce the quantization noise and avoid the local convergence problem.

Abstract: Embodiments herein treat the action segmentation as a domain adaption (DA) problem and reduce the domain discrepancy by performing unsupervised DA with auxiliary unlabeled videos. In one or more embodiments, to reduce domain discrepancy for both the spatial and temporal directions, embodiments of a Mixed Temporal Domain Adaptation (MTDA) approach are presented to jointly align frame-level and video-level embedded feature spaces across domains, and, in one or more embodiments, further integrate with a domain attention mechanism to focus on aligning the frame-level features with higher domain discrepancy, leading to more effective domain adaptation. Comprehensive experiment results validate that embodiments outperform previous state-of-the-art methods. Embodiments can adapt models effectively by using auxiliary unlabeled videos, leading to further applications of large-scale problems, such as video surveillance and human activity analysis.

Abstract: A method for information processing for accelerating neural network training. The method includes: acquiring a neural network corresponding to a deep learning task; and performing iterations of iterative training on the neural network based on a training data set. The training data set includes task data corresponding to the deep learning task. The iterative training includes: processing the task data in the training data set using a current neural network, and determining, based on a processing result of the neural network on the task data in a current iterative training, prediction loss of the current iterative training; determining a learning rate and a momentum in the current iterative training; and updating weight parameters of the current neural network by gradient descent based on a preset weight decay, and the learning rate, the momentum, and the prediction loss in the current iterative training. This method achieves efficient and low-cost deep learning-based neural network training.

Abstract: Presented herein are embodiments of a training deep learning models. In one or more embodiments, a compact deep learning model comprises fewer layers, which require fewer floating-point operations (FLOPs). Presented herein are also embodiments of a new learning rate function, which can adaptively change the learning rate between two linear functions. In one or more embodiments, combinations of half-precision floating point format training together with larger batch size in the training process may also be employed to aid the training process.

Abstract: Embodiments of this disclosure provide techniques used in a location sensing service and a timeline service implemented in a host device for providing location history. In particular, the location sensing service may passively collect location coordinates of the device using requests made by third party applications. The location sensing service may receive a signal comprising an intent for a location request in response to a change in the user activity or in response to a change in a cellular identification of the device. The device may determine that a time elapsed from a previous location request and a time corresponding with receiving one or more of the signals exceed a time threshold and, based thereon, request a location coordinate of the device.

Abstract: The disclosure relates to technology for location-based services, and in particular, to geofencing. A computing device generates multiple circular shaped geofences to cover a geographic region defined by a polygon shaped geofence. The multiple circular shaped geofences are monitored to detect a current location of user equipment entering a boundary of any of the circular shaped geofences. Based on the detection, determining that the current location of the user equipment is within the polygon shaped geofence. A location based service is notified that the user equipment has entered the polygon shaped geofence.