Abstract: A method is disclosed. The method may include receiving an image or video; extracting a plurality of features from the image or video; executing a neural network using the plurality of features to obtain a performance score for the image or video, the neural network comprising an input layer, a plurality of intermediate layers subsequent to the input layer, and a regression layer or a classification layer; extracting values from one or more signals between an intermediate layer and the regression layer or the classification layer; for each of the plurality of features, calculating, based on at least one of the one or more values, an impact score indicating an impact the feature had on the performance score; and generating, based on one or more impact scores for the plurality of features, indications indicating an impact different features of the image or video had on the performance score.

Abstract: Generative adversarial models have several benefits; however, due to mode collapse, these generators face a quality-diversity trade-off (i.e., the generator models sacrifice generation diversity for increased generation quality). Presented herein are embodiments that improve the performance of adversarial content generation by decelerating mode collapse. In one or more embodiments, a cooperative training paradigm is employed where a second model is cooperatively trained with the generator and helps efficiently shape the data distribution of the generator against mode collapse. Moreover, embodiments of a meta learning mechanism may be used, where the cooperative update to the generator serves as a high-level meta task and which helps ensures the generator parameters after the adversarial update stay resistant against mode collapse. In experiments, tested employments demonstrated efficient slowdown of mode collapse for the adversarial text generators.

Abstract: Techniques for in-flight scaling of machine learning training jobs are described. A request to execute a machine learning (ML) training job is received within a provider network, and the ML training job is executed using a first one or more compute instances. Upon a determination that a performance characteristic of the ML training job satisfies a scaling condition, a second one or more compute instances are added to the ML training job while the first one or more compute instances continue to execute portions of the ML training job.

Abstract: A transformer-based neural network includes at least one mask attention network (MAN). The MAN computes an original attention data structure that expresses influence between pairs of data items in a sequence of data items. The MAN then modifies the original data structure by mask values in a mask data structure, to produce a modified attention data structure. Compared to the original attention data structure, the modified attention data structure better accounts for the influence of neighboring data items in the sequence of data items, given a particular data item under consideration. The mask data structure used by the MAN can have static and/or machine-trained mask values. In one implementation, the transformer-based neural network includes at least one MAN in combination with at least one other attention network that does not use a mask data structure, and at least one feed-forward neural network.

Abstract: A learning system according to an embodiment includes a model generation device and n calculation devices. The model generation device includes a splitting unit, a secret sharing unit, and a share transmission unit. The splitting unit splits m×n pieces of training data into n groups each including m training data pieces, the n groups corresponding to the respective n calculation devices on one-to-one basis. The secret sharing unit generates m distribution training data pieces for each of the n groups by distributing using a secret sharing scheme and generates distribution training data for each of the m training data pieces in an i-th group among the n groups, using an i-th element Pi among n elements P1, P2, . . . , Pi, . . . , Pn, by distributing using the secret sharing scheme. The share transmission unit transmits corresponding m distribution training data pieces to each of the n calculation devices.

Abstract: A method and system are disclosed for training a model that implements a machine-learning algorithm. The technique utilizes latent descriptor vectors to change a multiple-valued output problem into a single-valued output problem and includes the steps of receiving a set of training data, processing, by a model, the set of training data to generate a set of output vectors, and adjusting a set of model parameters and component values for at least one latent descriptor vector in the plurality of latent descriptor vectors based on the set of output vectors. The set of training data includes a plurality of input vectors and a plurality of desired output vectors, and each input vector in the plurality of input vectors is associated with a particular latent descriptor vector in a plurality of latent descriptor vectors. Each latent descriptor vector comprises a plurality of scalar values that are initialized prior to training the model.

Abstract: A method of operating a neural network device including a plurality of layers, includes receiving sensing data from at least one sensor, determining environmental information, based on the received sensing data, determining multiple layers corresponding to the determined environmental information, and dynamically reconstructing the neural network device by changing at least two layers, among the plurality of layers, to the determined multiple layers.

Abstract: Novel and useful system and methods of several functional safety mechanisms for use in an artificial neural network (ANN) processor. The mechanisms can be deployed individually or in combination to provide a desired level of safety in neural networks. Multiple strategies are applied involving redundancy by design, redundancy through spatial mapping as well as self-tuning procedures that modify static (weights) and monitor dynamic (activations) behavior. The various mechanisms of the present invention address ANN system level safety in situ, as a system level strategy that is tightly coupled with the processor architecture. The NN processor incorporates several functional safety concepts which reduce its risk of failure that occurs during operation from going unnoticed. The mechanisms function to detect and promptly flag and report the occurrence of an error with some mechanisms capable of correction as well.

Abstract: Methods, computer program products, and systems are presented. The methods include, for instance: obtaining communication data streams, extracting data relevant to a point of view of a user, and generating a point of view record in a knowledge base that may be utilized by another user communicating with the user.

Abstract: An operation method and apparatus for a network layer in a Deep Neural Network are provided. The method includes: acquiring a weighted tensor of the network layer in the Deep Neural Network, the weighted tensor comprising a plurality of filters; converting each filter into a linear combination of a plurality of fixed-point convolution kernels by splitting the filter, wherein a weight value of each of the fixed-point convolution kernels is a fixed-point quantized value having a specified bit-width; for each filter, performing a convolution operation on input data of the network layer and each of the fixed-point convolution kernels, respectively, to obtain a plurality of convolution results, and calculating a weighted sum of the obtained convolution results based on the linear combination of the plurality of fixed-point convolution kernels of the filter to obtain an operation result of the filter; determining output data of the network layer.

Abstract: A block-based inference method for a memory-efficient convolutional neural network implementation is performed to process an input image. A block-based inference step is performed to execute a multi-layer convolution operation on each of a plurality of input block data to generate an output block data and includes selecting a plurality of ith layer recomputing features according to a position of the output block data along a scanning line feed direction, and then selecting an ith layer recomputing input feature block data according to the position of the output block data and the ith layer recomputing features, and selecting a plurality of ith layer reusing features according to the ith layer recomputing input feature block data along a block scanning direction, and then combining the ith layer recomputing input feature block data with the ith layer reusing features to generate an ith layer reusing input feature block data.

Abstract: An apparatus includes circuitry for a neural network that is configured to perform forward propagation neural network operations on floating point numbers having a first n-bit floating point format. The first n-bit floating point format has a configuration consisting of a sign bit, m exponent bits and p mantissa bits where m is greater than p. The circuitry is further configured to perform backward propagation neural network operations on floating point numbers having a second n-bit floating point format that is different than the first n-bit floating point format. The second n-bit floating point format has a configuration consisting of a sign bit, q exponent bits and r mantissa bits where q is greater than m and r is less than p.

Abstract: A computing device includes one or more processors, a first random access memory (RAM) comprising magnetic random access memory (MRAM), a second random access memory of a type distinct from MRAM, and a non-transitory computer-readable storage medium storing instructions for execution by the one or more processors. The computing device receives first data on which to train an artificial neural network (ANN) and trains the ANN by, using the first RAM comprising the MRAM, performing a first set of training iterations to train the ANN using the first data, and, after performing the first set of training iterations, using the second RAM of the type distinct from MRAM, performing a second set of training iterations to train the ANN using the first data. The computing device stores values for the trained ANN. The trained ANN is configured to classify second data based on the stored values.

Abstract: Described herein is a neural network accelerator (NNA) with reconfigurable memory resources for forming a set of local memory buffers comprising at least one activation buffer, at least one weight buffer, and at least one output buffer. The NNA supports a plurality of predefined memory configurations that are optimized for maximizing throughput and reducing overall power consumption in different types of neural networks. The memory configurations differ with respect to at least one of a total amount of activation, weight, or output buffer memory, or a total number of activation, weight, or output buffers. Depending on which type of neural network is being executed and the memory behavior of the specific neural network, a memory configuration can be selected accordingly.

Abstract: Methods for simplifying a dependency graph in a neural network accelerator are provided. Computations and data movements for the neural network accelerator may be described with a flow graph, where graph nodes represent computation or data movement operations and graph edges represent dependencies between operations. A flow graph may contain redundant edges that can be removed while retaining the reachability of each of the nodes in the graph. To identify redundant edges, a compiler may generate vector clocks to track the relationships of operations performed by various execution engines prior to execution of a program reaching a given node or operation. Redundant edges may be identified and removed based on the relative values of the vector clocks to reduce the complexity of the graph.

Abstract: An artificial neural network and method are provided. The method includes receiving a set of input voltages; converting a respective input voltage in said set of input voltages into a respective set of local currents using a voltage-to-current conversion; multiplying said respective set of local currents by a respective set of binary signals to establish a respective set of conditionally inverted currents; summing said respective set of conditionally inverted currents into a respective local current; summing all respective local currents into a global current; and converting the global current into an output voltage using a load circuit.

Abstract: A neural network computing device and a cache management method thereof are provided. The neural network computing device includes a computing circuit, a cache circuit and a main memory. The computing circuit performs a neural network calculation including a first layer calculation and a second layer calculation. After the computing circuit completes the first layer calculation and generates a first calculation result required for the second layer calculation, the cache circuit retains the first calculation result in the cache circuit until the second layer calculation is completed. After the second layer calculation is completed, the cache circuit invalidates the first calculation result retained in the cache circuit to prevent the first calculation result from being written into the main memory.

Abstract: A generative adversarial network, a method of training a generator neural network, and a method of generating images using the generator network is provided. The generator neural network is configured to process an input comprising a noise vector and a pair of conditioning variables to generate an image according to the conditioning variables. The generator neural network includes a mixed-conditional batch normalization layer. The mixed-conditional batch normalization layer is configured to normalize a network layer output to generate a normalized network layer output, comprising transforming the network layer output in accordance with mixed-conditional batch normalization layer parameters to generate the normalized network layer output, wherein the mixed-conditional batch normalization layer parameters are computed by applying an affine transformation to the conditioning variables.

Abstract: Several applications capture data from sensors resulting in multi-sensor time series. Existing neural networks-based approaches for such multi-sensor/multivariate time series modeling assume fixed input-dimension/number of sensors. Such approaches can struggle in practical setting where different instances of same device/equipment come with different combinations of installed sensors. In the present disclosure, neural network models are trained from such multi-sensor time series having varying input dimensionality, owing to availability/installation of different sensors subset at each source of time series. Neural network (NN) architecture is provided for zero-shot transfer learning allowing robust inference for multivariate time series with previously unseen combination of available dimensions/sensors at test time.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining an architecture for a task neural network that is configured to perform a particular machine learning task on a target set of hardware resources. When deployed on a target set of hardware, such as a collection of datacenter accelerators, the task neural network may be capable of performing the particular machine learning task with enhanced accuracy and speed.