Abstract: During training of deep neural networks, a Copernican loss (LC) is designed to augment the standard Softmax loss to explicitly minimize intra-class variation and simultaneously maximize inter-class variation. Copernican loss operates using the cosine distance and thereby affects angles leading to a cosine embedding, which removes the disconnect between training and testing.

Abstract: Long-short term memory (LSTM) cells on spiking neuromorphic hardware are provided. In various embodiments, such systems comprise a spiking neurosynaptic core. The neurosynaptic core comprises a memory cell, an input gate operatively coupled to the memory cell and adapted to selectively admit an input to the memory cell, and an output gate operatively coupled to the memory cell an adapted to selectively release an output from the memory cell. The memory cell is adapted to maintain a value in the absence of input.

Abstract: Technologies for a neural belief reasoner model generative models that specifies belief functions are described. Aspects include receiving, by a device operatively coupled to a processor, a request for a belief function, and processing, by the device, the request for the belief function in the generative model based on trained probability parameters and a minimization function to determine a generalized belief function defined by fuzzy sets. Data corresponding to the generalized belief function is output, e.g., as a belief value and plausibility value.

Abstract: The domain adaptation problem is addressed by using the predictions of a trained model over both source and target domain to retain the model with the assistance of an auxiliary model and a modified objective function. Inaccuracy in the model's predictions in the target domain is treated as noise and is reduced by using a robust learning framework during retraining, enabling unsupervised training in the target domain. Applications include object detection models, where noise in retraining is reduced by explicitly representing label noise and geometry noise in the objective function and using the ancillary model to inject information about label noise.

Abstract: Knowledge transfer between recurrent neural networks is performed by obtaining a first output sequence from a bidirectional Recurrent Neural Network (RNN) model for an input sequence, obtaining a second output sequence from a unidirectional RNN model for the input sequence, selecting at least one first output from the first output sequence based on a similarity between the at least one first output and a second output from the second output sequence; and training the unidirectional RNN model to increase the similarity between the at least one first output and the second output.

Abstract: Systems and methods for data augmentation in a neural network system includes performing a first training process, using a first training dataset on a neural network system including an autoencoder including an encoder and a decoder to generate a trained autoencoder. A trained encoder is configured to receive a first plurality of input data in an N-dimensional data space and generate a first plurality of latent variables in an M-dimensional latent space, wherein M is an integer less than N. A sampling process is performed on the first plurality of latent variables to generate a first plurality of latent variable samples. A trained decoder is used to generate a second training dataset using the first plurality of latent variable samples. The second training dataset is used to train a first classifier including a first classifier neural network model to generate a trained classifier for providing transaction classification.

Abstract: Machine learning is utilized to learn an optimized quantization configuration for an artificial neural network (ANN). For example, an ANN can be utilized to learn an optimal bit width for quantizing weights for layers of the ANN. The ANN can also be utilized to learn an optimal bit width for quantizing activation values for the layers of the ANN. Once the bit widths have been learned, they can be utilized at inference time to improve the performance of the ANN by quantizing the weights and activation values of the layers of the ANN.

Abstract: Systems and methods for adaptive data processing associated with complex dynamics are provided. The method may include applying the two or more predictive algorithms or rule-sets to an atomized model to generate applied data models. After receipt of inputs, the method may further include processing at least two propositions during a learning mode based upon detection of an absolute pattern within the applied data models; wherein propositions are action proposals associated with each predictive algorithm. At least two propositions may compete against each other through the use of an associated rating cell, which may be updated based upon the detected patterns. The method may further include processing propositions during an execution mode based upon detection of an absolute condition, wherein the rating cells are updated based upon these detected conditions. Further, these updated rating cells may be provided as feedback to update the atomized model.

Abstract: A function creation method is disclosed. The method comprises defining one or more database function inputs, defining cluster processing information, defining a deep learning model, and defining one or more database function outputs. A database function is created based at least in part on the one or more database function inputs, the cluster set-up information, the deep learning model, and the one or more database function outputs. In some embodiments, the database function enables a non-technical user to utilize deep learning models.

Abstract: A synapse memory and a method for reading a weight value stored in a synapse memory are provided. The synapse memory includes a memory device configured to store a weight value. The memory device includes a read terminal, a write terminal, and a common terminal, the read terminal being configured to receive a read signal, the write terminal being configured to receive a write signal, and the common terminal being configured to output an output signal from the memory device. The synapse memory also includes a write transistor provided between the write terminal of the memory device and a write signal line configured to send the write signal. The synapse memory further includes a common transistor provided between the common terminal of the memory device and one of the dendrite lines.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive learning rate while also ensuring that the learning rate is non-increasing.

Abstract: A computer-implemented method comprises instantiating a policy function approximator. The policy function approximator is configured to calculate a plurality of estimated action probabilities in dependence on a given state of the environment. Each of the plurality of estimated action probabilities corresponds to a respective one of a plurality of discrete actions performable by the reinforcement learning agent within the environment. An initial plurality of estimated action probabilities in dependence on a first state of the environment are calculated. Two or more of the plurality of discrete actions are concurrently performed within the environment when the environment is in the first state. In response to the concurrent performance, a reward value is received. In response to the received reward value being greater than a baseline reward value, the policy function approximator is updated, such that it is configured to calculate an updated plurality of estimated action probabilities.

Abstract: A learning data processing unit accepts, as input, a plurality of pieces of learning data for a respective plurality of tasks, and calculates, for each of the tasks, a batch size which meets a condition that a value obtained by dividing a data size of corresponding one of the pieces of learning data by the corresponding batch size is the same between the tasks. A batch sampling unit samples, for each of the tasks, samples from corresponding one of the pieces of learning data with the corresponding batch size calculated by the learning data processing unit. A learning unit updates a weight of a discriminator for each of the tasks, using the samples sampled by the batch sampling unit.

Abstract: A neural network system is proposed, including an input network for extracting, from state data, respective entity data for each a plurality of entities which are present, or at least potentially present, in the environment. The entity data describes the entity. The neural network contains a relational network for parsing this data, which includes one or more attention blocks which may be stacked to perform successive actions on the entity data. The attention blocks each include a respective transform network for each of the entities. The transform network for each entity is able to transform data which the transform network receives for the entity into modified entity data for the entity, based on data for a plurality of the other entities. An output network is arranged to receive data output by the relational network, and use the received data to select a respective action.

Abstract: Described herein are embodiments for systems and methods for mutual machine learning with global topic discovery and local word embedding. Both topic modeling and word embedding map documents onto a low-dimensional space, with the former clustering words into a global topic space and the latter mapping word into a local continuous embedding space. Embodiments of Topic Modeling and Sparse Autoencoder (TMSA) framework unify these two complementary patterns by constructing a mutual learning mechanism between word co-occurrence based topic modeling and autoencoder. In embodiments, word topics generated with topic modeling are passed into auto-encoder to impose topic sparsity for the autoencoder to learn topic-relevant word representations. In return, word embedding learned by autoencoder is sent back to topic modeling to improve the quality of topic generations. Performance evaluation on various datasets demonstrates the effectiveness of the disclosed TMSA framework in discovering topics and embedding words.

Abstract: An electronic apparatus is provided. The electronic apparatus includes a first memory configured to store a first artificial intelligence (AI) model including a plurality of first elements and a processor configured to include a second memory. The second memory is configured to store a second AI model including a plurality of second elements. The processor is configured to acquire output data from input data based on the second AI model. The first AI model is trained through an AI algorithm. Each of the plurality of second elements includes at least one higher bit of a plurality of bits included in a respective one of the plurality of first elements.

Abstract: Logic may reduce the size of runtime memory for deep neural network inference computations. Logic may determine, for two or more stages of a neural network, a count of shared block allocations, or shared memory block allocations, that concurrently exist during execution of the two or more stages. Logic may compare counts of the shared block allocations to determine a maximum count of the counts. Logic may reduce inference computation time for deep neural network inference computations. Logic may determine a size for each of the shared block allocations of the count of shared memory block allocations, to accommodate data to store in a shared memory during execution of the two or more stages of the cascaded neural network. Logic may determine a batch size per stage of the two or more stages of a cascaded neural network based on a lack interdependencies between input data.

Abstract: The disclosed technology reveals a hierarchical policy network, for use by a software agent, to accomplish an objective that requires execution of multiple tasks. A terminal policy learned by training the agent on a terminal task set, serves as a base task set of the intermediate task set. An intermediate policy learned by training the agent on an intermediate task set serves as a base policy of the top policy. A top policy learned by training the agent on a top task set serves as a base task set of the top task set. The agent is configurable to accomplish the objective by traversal of the hierarchical policy network. A current task in a current task set is executed by executing a previously-learned task selected from a corresponding base task set governed by a corresponding base policy, or performing a primitive action selected from a library of primitive actions.

Abstract: Aspects of the present disclosure relate to machine learning techniques for identifying the incremental impact of different past events on the likelihood that a target outcome will occur. The technology can use a recurrent neural network to analyze two different representations of an event sequence—one in which some particular event occurs, and another in which that particular event does not occur. The incremental impact of that particular event can be determined based on the calculated difference between the probabilities of the target outcome occurring after these two sequences.

Abstract: According to an embodiment of the present disclosure, a method of training a machine learning model is provided. Input data is received from at least one remote device. A classifier is evaluated by determining a classification accuracy of the input data. A training data matrix of the input data is applied to a selected context autoencoder of a knowledge bank of autoencoders including at least one context autoencoder and the training data matrix is determined to be out of context for the selected autoencoder. The training data matrix is applied to each other context autoencoder of the at least one autoencoder and the training data matrix is determined to be out of context for each other context autoencoder. A new context autoencoder is constructed.