Abstract: A neural network comprises trained interconnected neurons. The neural network is configured to constrain the relationship between one or more inputs and one or more outputs of the neural network so the relationships between them are consistent with expectations of the relationships; and/or the neural network is trained by creating a set of data comprising input data and associated outputs that represent archetypal results and providing real exemplary input data and associated output data and the created data to neural network. The real exemplary output data and the created associated output data is compared to the actual output of the neural network, which is adjusted to create a best fit to the real exemplary data and the created data.

Abstract: In the design and implementation of neural networks, training is determined by a series of architectural and parametric decisions. A method is disclosed that, using genetic algorithms, improves the training characteristics of a neural network. The method begins with a population and iteratively modifies one or more parameters in each generation based on the network with the best training response in the previous generation.

Abstract: A neural network circuit for performing a processing of recognizing voices, images and the like comprises a weight memory for holding a lot of weight values (initial weight values) which correspond to a plurality of input terminals of each of a plurality of neurons forming a neural network and have been initially learned, and a difference value memory for storing difference values between the weight values of the weight memory and additionally learned weight values. The weight memory is formed by a ROM. The difference value memory is formed by a SRAM, for example. During operation of recognizing input data, the initial weight values of the weight memory and the difference values of the difference value memory are added together. The added weight values are used to calculate an output value of each neuron of an output layer. Accordingly, the initial weight values can be additionally learned at a high speed by existence of the difference value memory having a small capacity.

Abstract: Disclosed are various embodiments of memristive neural networks comprising neural nodes. Memristive nanofibers are used to form artificial synapses in the neural networks. Each memristive nanofiber may couple one or more neural nodes to one or more other neural nodes. In one case, a memristive neural network includes a first neural node, a second neural node, and a memristive fiber that couples the first neural node to the second neural node. The memristive fiber comprises a conductive core and a memristive shell, where the conductive core forms a communications path between the first neural node and the second neural node and the memristive shell forms a memristor synapse between the first neural node and the second neural node.

Abstract: An electronic apparatus for compressing a neural network model may acquire training data pairs based on an original, trained neural network model and train a compressed neural network model compressed from the original, trained neural network model using the acquired training data pairs.

Abstract: A method for calculating an output of a neural network, including the steps of generating a first neural network that includes discrete edge weights from a neural network that includes precise edge weights by stochastic rounding; of generating a second neural network that includes discrete edge weights from the neural network that includes precise edge weights by stochastic rounding; and of calculating an output by adding together the output of the first neural network and of the second neural network.

Abstract: The present disclosure relates to a method for allocating resources of an accelerator to two or more neural networks for execution. The two or more neural networks may include a first neural network and a second neural network. The method comprises analyzing workloads of the first neural network and the second neural network, wherein the first neural network and second neural network each includes multiple computational layers, evaluating computational resources of the accelerator for executing each computational layer of the first and second neural networks, and scheduling computational resources of the accelerator to execute one computational layer of the multiple computation layers of the first neural network and to execute one or more computational layers of the multiple computational layers of the second neural network.

Abstract: Methods, apparatus, and computer readable media related to combining and/or training one or more neural network modules based on version identifier(s) assigned to the neural network module(s). Some implementations are directed to using version identifiers of neural network modules in determining whether and/or how to combine multiple neural network modules to generate a combined neural network model for use by a robot and/or other apparatus. Some implementations are additionally or alternatively directed to assigning a version identifier to an endpoint of a neural network module based on one or more other neural network modules to which the neural network module is joined during training of the neural network module.

Abstract: The present disclosure provides systems, methods and computer-readable media for optimizing the neural architecture search for the automated machine learning process. In one aspect, neural architecture search method including selecting a neural architecture for training as part of an automated machine learning process; collecting statistical parameters on individual nodes of the neural architecture during the training; determining, based on the statistical parameters, active nodes of the neural architecture to form a candidate neural architecture; and validating the candidate neural architecture to produce a trained neural architecture to be used in implemented an application or a service.

Abstract: Constructing and simulating artificial neural networks and components thereof within a spreadsheet environment results in user friendly neural networks which do not require algorithmic based software in order to train or operate. Such neural networks can be easily cascaded to form complex neural networks and neural network systems, including neural networks capable of self-organizing so as to self-train within a spreadsheet, neural networks which train simultaneously within a spreadsheet, and neural networks capable of autonomously moving, monitoring, analyzing, and altering data within a spreadsheet. Neural networks can also be cascaded together in self training neural network form to achieve a device prototyping system.

Abstract: The present invention provides an event-driven universal neural network circuit. The circuit comprises a plurality of neural modules. Each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module. An interconnection network comprising a plurality of digital synapses interconnects the neural modules. Each synapse interconnects a first neural module to a second neural module by interconnecting a neuron in the first neural module to a corresponding neuron in the second neural module. Corresponding neurons in the first neural module and the second neural module communicate via the synapses. Each synapse comprises a learning rule associating a neuron in the first neural module with a corresponding neuron in the second neural module. A control module generates signals which define a set of time steps for event-driven operation of the neurons and event communication via the interconnection network.

Abstract: Embodiments of the invention relate to mapping neural dynamics of a neural model on to a lookup table. One embodiment comprises defining a phase plane for a neural model. The phase plane represents neural dynamics of the neural model. The phase plane is coarsely sampled to obtain state transition information for multiple neuronal states. The state transition information is mapped on to a lookup table.

Abstract: A method of training an action selection neural network to perform a demonstrated task using a supervised learning technique. The action selection neural network is configured to receive demonstration data comprising actions to perform the task and rewards received for performing the actions. The action selection neural network has auxiliary prediction task neural networks on one or more of its intermediate outputs. The action selection policy neural network is trained using multiple combined losses, concurrently with the auxiliary prediction task neural networks.

Abstract: According to various embodiments, a method for generating one or more optimal neural network architectures is disclosed. The method includes providing an initial seed neural network architecture and utilizing sequential phases to synthesize the neural network until a desired neural network architecture is reached. The phases include a gradient-based growth phase and a magnitude-based pruning phase.

Abstract: Constructing and simulating artificial neural networks and components thereof within a spreadsheet environment results in user friendly neural networks which do not require algorithmic based software in order to train or operate. Such neural networks can be easily cascaded to form complex neural networks and neural network systems, including neural networks capable of self-organizing so as to self-train within a spreadsheet, neural networks which train simultaneously within a spreadsheet, and neural networks capable of autonomously moving, monitoring, analyzing, and altering data within a spreadsheet. Neural networks can also be cascaded together in self training neural network form to achieve a device prototyping system.

Abstract: A method of updating a neural network may be provided. A method may include selecting a number of neurons for a layer for a neural network such that the number of neurons in the layer is less than at least one of a number of neurons in a first layer of the neural network and a number of neurons in a second, adjacent layer of the neural network. The method may further include and at least one of inserting the layer between the first layer and the second layer of the neural network and replacing one of the first layer and the second layer with the layer to reduce a number of connections in the neural network.

Abstract: Certain aspects of the present disclosure support techniques for unsupervised neural replay, learning refinement, association and memory transfer.

Abstract: The present invention provides an event-driven universal neural network circuit. The circuit comprises a plurality of neural modules. Each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module. An interconnection network comprising a plurality of digital synapses interconnects the neural modules. Each synapse interconnects a first neural module to a second neural module by interconnecting a neuron in the first neural module to a corresponding neuron in the second neural module. Corresponding neurons in the first neural module and the second neural module communicate via the synapses. Each synapse comprises a learning rule associating a neuron in the first neural module with a corresponding neuron in the second neural module. A control module generates signals which define a set of time steps for event-driven operation of the neurons and event communication via the interconnection network.

Abstract: Certain aspects of the present disclosure support techniques for unsupervised neural replay, learning refinement, association and memory transfer.

Abstract: A neural network processing unit (NPU) includes a processing element array, an NPU memory system configured to store at least a portion of data of an artificial neural network model processed in the processing element array, and an NPU scheduler configured to control the processing element array and the NPU memory system based on artificial neural network model structure data or artificial neural network data locality information.