Abstract: An intelligent table-based question answering system receives an input user query requesting information from an input table to generate a response to the input query. Two types of user queries including textual user queries and scalar user queries that require at least one mathematical operation to be executed can be processed for answer generation. The input table is processed to generate a paragraph of table text which includes row and column information extracted from the input table. The input query is provided along with the paragraph of table text to a deep Quans model which outputs a candidate answer that forms a textual portion of the response. Also, the candidate answer is reverse mapped to the input table and a portion of the input table including the candidate answer is provided as a tabular portion of the response.

Abstract: Persistent storage contains a representation of a neural network including an input layer, and output layer, and a hidden layer, wherein nodes of the hidden layer incorporate serialized activation functions, wherein the serialized activation functions for each of the nodes include a sigmoid function and a Beta function, wherein the sigmoid function is applied to weighted outputs from nodes of a previous layer of the neural network, wherein the Beta function is applied to a conductance hyper-parameter and respective outputs of the sigmoid function, and wherein outputs of the Beta function are provided to a subsequent layer of the neural network. One or more processors are configured to train the neural network until the outputs of the sigmoid function for the nodes of the hidden layer are substantially binary.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive learning rate while also ensuring that the learning rate is non-increasing.

Abstract: A method of operating a neural network. The input layer of the network may have n input nodes connected to output nodes via a hidden layer. The hidden layer may include m hidden nodes. The n input nodes may connect to a subset of k nodes of the m hidden nodes via respective synaptic connections, to which training weights are associated, which form an n×k input matrix Win, whereas a subset of m?k nodes of the hidden layer are not connected by any node of the input layer. Running the network may include performing a first matrix vector multiplication between the input matrix Win and a vector of values obtained in output of the input nodes and a second matrix vector multiplication between a fixed matrix Wrec of fixed weights and a vector of values obtained in output of the m nodes of the hidden layer.

Abstract: During training of deep neural networks, a Copernican loss (LC) is designed to augment a primary loss function, for example, a standard Softmax loss, to explicitly minimize intra-class variation and simultaneously maximize inter-class variation. Copernican loss operates using the cosine distance and thereby affects angles leading to a cosine embedding, which removes the disconnect between training and testing.

Abstract: Knowledge transfer between recurrent neural networks is performed by obtaining a first output sequence from a bidirectional Recurrent Neural Network (RNN) model for an input sequence, obtaining a second output sequence from a unidirectional RNN model for the input sequence, selecting at least one first output from the first output sequence based on a similarity between the at least one first output and a second output from the second output sequence; and training the unidirectional RNN model to increase the similarity between the at least one first output and the second output.

Abstract: Embodiments of this application disclose a method for training an image caption model, the image caption model including an encoding convolutional neural network (CNN) and a decoding recurrent neural network (RNN). The method includes: obtaining an image eigenvector of an image sample by using the encoding CNN; decoding the image eigenvector by using the decoding RNN, to obtain a sentence used for describing the image sample; determining a matching degree between the sentence obtained through decoding and the image sample and a smoothness degree of the sentence obtained through decoding, respectively; and adjusting the decoding RNN according to the matching degree and the smoothness degree.

Abstract: Systems and methods are provided for data shuffling for distributed machine learning training, including each training node in the network receiving a shard of training data, wherein the training data set is divided into shards having data items. Each data item is assigned to a working set such that each of the working set includes data items from multiple shards. The training nodes perform training using the data items of a first working set that are in each node's shard. Upon completion of the training using the data items of the first working set, the training nodes performing training using the data items of a second working set that are in their shards; and while the training nodes are performing training on their respective subsets of shards of the second working set, the nodes randomly shuffling data items in the first working set to create a shuffled first working set.

Abstract: A method for action automation includes determining, using an electronic device, an action based on domain information. Activity patterns associated with the action are retrieved. For each activity pattern, a candidate action rule is determined. Each candidate action rule specifies one or more pre-conditions when the action occurs. One or more preferred candidate action rules are determined from multiple candidate action rules for automation of the action.

Abstract: A method may include embedding, in a hidden layer and/or an output layer of a first machine learning model, a first digital watermark. The first digital watermark may correspond to input samples altering the low probabilistic regions of an activation map associated with the hidden layer of the first machine learning model. Alternatively, the first digital watermark may correspond to input samples rarely encountered by the first machine learning model. The first digital watermark may be embedded in the first machine learning model by at least training, based on training data including the input samples, the first machine learning model. A second machine learning model may be determined to be a duplicate of the first machine learning model based on a comparison of the first digital watermark embedded in the first machine learning model and a second digital watermark extracted from the second machine learning model.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for a neural network system comprising one or more gated linear networks. A system includes: one or more gated linear networks, wherein each gated linear network corresponds to a respective data value in an output data sample and is configured to generate a network probability output that defines a probability distribution over possible values for the corresponding data value, wherein each gated linear network comprises a plurality of layers, wherein the plurality of layers comprises a plurality of gated linear layers, wherein each gated linear layer has one or more nodes, and wherein each node is configured to: receive a plurality of inputs, receive side information for the node; combine the plurality of inputs according to a set of weights defined by the side information, and generate and output a node probability output for the corresponding data value.

Abstract: A method of operating a neural network system includes parsing, by a processor, at least one item of information related to a neural network operation from an input neural network model; determining, by the processor, information of at least one dedicated hardware device; and generating, by the processor, a reshaped neural network model by changing information of the input neural network model according to a result of determining the information of the at least one dedicated hardware device such that the reshaped neural network model is tailored for execution by the dedicated hardware device.

Abstract: According to embodiments, an encoder neural network receives a one-hot representation of a real text. The encoder neural network outputs a latent representation of the real text. A decoder neural network receives random noise data or artificial code generated by a generator neural network from random noise data. The decoder neural network outputs softmax representation of artificial text. The decoder neural network receives the latent representation of the real text. The decoder neural network outputs a reconstructed softmax representation of the real text. A hybrid discriminator neural network receives a first combination of the soft-text and the latent representation of the real text and a second combination of the softmax representation of artificial text and the artificial code. The hybrid discriminator neural network outputs a probability indicating whether the second combination is similar to the first combination. Additional embodiments for utilizing latent representation are also disclosed.

Abstract: During training of deep neural networks, a Copernican loss (LC) is designed to augment the standard Softmax loss to explicitly minimize intra-class variation and simultaneously maximize inter-class variation. Copernican loss operates using the cosine distance and thereby affects angles leading to a cosine embedding, which removes the disconnect between training and testing.

Abstract: Knowledge transfer between recurrent neural networks is performed by obtaining a first output sequence from a bidirectional Recurrent Neural Network (RNN) model for an input sequence, obtaining a second output sequence from a unidirectional RNN model for the input sequence, selecting at least one first output from the first output sequence based on a similarity between the at least one first output and a second output from the second output sequence; and training the unidirectional RNN model to increase the similarity between the at least one first output and the second output.

Abstract: Machine learning is utilized to learn an optimized quantization configuration for an artificial neural network (ANN). For example, an ANN can be utilized to learn an optimal bit width for quantizing weights for layers of the ANN. The ANN can also be utilized to learn an optimal bit width for quantizing activation values for the layers of the ANN. Once the bit widths have been learned, they can be utilized at inference time to improve the performance of the ANN by quantizing the weights and activation values of the layers of the ANN.

Abstract: Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive learning rate while also ensuring that the learning rate is non-increasing.

Abstract: An electronic apparatus is provided. The electronic apparatus includes a first memory configured to store a first artificial intelligence (AI) model including a plurality of first elements and a processor configured to include a second memory. The second memory is configured to store a second AI model including a plurality of second elements. The processor is configured to acquire output data from input data based on the second AI model. The first AI model is trained through an AI algorithm. Each of the plurality of second elements includes at least one higher bit of a plurality of bits included in a respective one of the plurality of first elements.

Abstract: Described herein are embodiments for systems and methods for mutual machine learning with global topic discovery and local word embedding. Both topic modeling and word embedding map documents onto a low-dimensional space, with the former clustering words into a global topic space and the latter mapping word into a local continuous embedding space. Embodiments of Topic Modeling and Sparse Autoencoder (TMSA) framework unify these two complementary patterns by constructing a mutual learning mechanism between word co-occurrence based topic modeling and autoencoder. In embodiments, word topics generated with topic modeling are passed into auto-encoder to impose topic sparsity for the autoencoder to learn topic-relevant word representations. In return, word embedding learned by autoencoder is sent back to topic modeling to improve the quality of topic generations. Performance evaluation on various datasets demonstrates the effectiveness of the disclosed TMSA framework in discovering topics and embedding words.

Abstract: Logic may reduce the size of runtime memory for deep neural network inference computations. Logic may determine, for two or more stages of a neural network, a count of shared block allocations, or shared memory block allocations, that concurrently exist during execution of the two or more stages. Logic may compare counts of the shared block allocations to determine a maximum count of the counts. Logic may reduce inference computation time for deep neural network inference computations. Logic may determine a size for each of the shared block allocations of the count of shared memory block allocations, to accommodate data to store in a shared memory during execution of the two or more stages of the cascaded neural network. Logic may determine a batch size per stage of the two or more stages of a cascaded neural network based on a lack interdependencies between input data.