Abstract: Systems and methods are disclosed for generating an optimized shading graph in a modeling application. One method comprises receiving a plurality of nodes in a shading graph, the plurality of nodes being connected via a plurality of explicit connections and one or more of the plurality of nodes comprising one or more internal connections. One or more implicit context connections are determined between the plurality of nodes. Additionally, one or more actual implicit context connections are determined based on the one or more implicit context connections and the one or more internal connections. Furthermore, one or more shortcuts are determined based on one or more of i) the plurality of explicit connections, ii) the one or more implicit context connections, and iii) the one or more actual implicit context connections. One or more inactive nodes and/or one or more inactive connections are also removed from the shading graph.

Abstract: A processing node in a temporal memory system includes a spatial pooler and a sequence processor. The spatial pooler generates a spatial pooler signal representing similarity between received spatial patterns in an input signal and stored co-occurrence patterns. The spatial pooler signal is represented by a combination of elements that are active or inactive. Each co-occurrence pattern is mapped to different subsets of elements of an input signal. The spatial pooler signal is fed to a sequence processor receiving and processed to learn, recognize and predict temporal sequences in the input signal. The sequence processor includes one or more columns, each column including one or more cells. A subset of columns may be selected by the spatial pooler signal, causing one or more cells in these columns to activate.

Abstract: Embodiments of the invention relate to a scalable neural hardware for the noisy-OR model of Bayesian networks. One embodiment comprises a neural core circuit including a pseudo-random number generator for generating random numbers. The neural core circuit further comprises a plurality of incoming electronic axons, a plurality of neural modules, and a plurality of electronic synapses interconnecting the axons to the neural modules. Each synapse interconnects an axon with a neural module. Each neural module receives incoming spikes from interconnected axons. Each neural module represents a noisy-OR gate. Each neural module spikes probabilistically based on at least one random number generated by the pseudo-random number generator unit.

Abstract: A neuromorphic device may include a pre-synaptic neuron, a row line extending from the pre-synaptic neuron in a row direction, a post-synaptic neuron, a column line extending from the post-synaptic neuron in a column direction, and a synapse coupled between the row line and the column line. The synapse may be disposed in an intersection region between the row line and the column line. The synapse may include a first unit synapse and a second unit synapse. The first unit synapse may include a resistive memory device. The second unit synapse may include a phase-changeable memory device.

Abstract: A neuromorphic computing system is provided which comprises: a synapse core; and a pre-synaptic neuron, a first post-synaptic neuron, and a second post-synaptic neuron coupled to the synaptic core, wherein the synapse core is to: receive a request from the pre-synaptic neuron, generate, in response to the request, a first address of the first post-synaptic neuron and a second address of the second post-synaptic neuron, wherein the first address and the second address are not stored in the synapse core prior to receiving the request.

Abstract: A configurable spiking neural network based accelerator system is provided. The accelerator system may be executed on an expansion card which may be a printed circuit board. The system includes one or more application specific integrated circuits comprising at least one spiking neural processing unit and a programmable logic device mounted on the printed circuit board. The spiking neural processing unit includes digital neuron circuits and digital, dynamic synaptic circuits. The programmable logic device is compatible with a local system bus. The spiking neural processing units contain digital circuits comprises a Spiking Neural Network that handles all of the neural processing. The Spiking Neural Network requires no software programming, but can be configured to perform a specific task via the Signal Coupling device and software executing on the host computer.

Abstract: Self-adjusting thresholds for synaptic activity in neural networks are provided. In various embodiments, for each of a plurality of neurons within an artificial neural network, an overlap value is determined corresponding to active inputs connected to the neuron via synapses having non-zero synaptic weights. A count of those of the plurality of neurons whose overlap exceeds an activation threshold of the neural network is determined. The count is compared to a predetermined neuronal activity target. The activation threshold of the neural network is adjusted to approach the predetermined neuronal activity target.

Abstract: A method includes detecting an event occurring on a vehicle. The vehicle includes at least one computing device that controls at least one operation of the vehicle. The at least one computing device includes a first computing device comprising system memory. In response to detecting the event, data is downloaded from the system memory to a non-volatile memory device of the vehicle. In some cases, a control action for the vehicle is implemented based on analysis of the downloaded data.

Abstract: In one embodiment, the present invention provides a neural network circuit comprising multiple symmetric core circuits. Each symmetric core circuit comprises a first core module and a second core module. Each core module comprises a plurality of electronic neurons, a plurality of electronic axons, and an interconnection network comprising multiple electronic synapses interconnecting the axons to the neurons. Each synapse interconnects an axon to a neuron. The first core module and the second core module are logically overlayed on one another such that neurons in the first core module are proximal to axons in the second core module, and axons in the first core module are proximal to neurons in the second core module. Each neuron in each core module receives axonal firing events via interconnected axons and generates a neuronal firing event according to a neuronal activation function.

Abstract: The present invention provides a system comprising multiple core circuits. Each core circuit comprises multiple electronic axons for receiving event packets, multiple electronic neurons for generating event packets, and a fanout crossbar including multiple electronic synapse devices for interconnecting the neurons with the axons. The system further comprises a routing system for routing event packets between the core circuits. The routing system virtually connects each neuron with one or more programmable target axons for the neuron by routing each event packet generated by the neuron to the target axons. Each target axon for each neuron of each core circuit is an axon located on the same core circuit as, or a different core circuit than, the neuron.

Abstract: Computer-based systems and methods guide the learning of features in middle layers of a deep neural network. The guidance can be provided by aligning sets of nodes or entire layers in a network being trained with sets of nodes in a reference system. This guidance facilitates the trained network to more efficiently learn features learned by the reference system using fewer parameters and with faster training. The guidance also enables training of a new system with a deeper network, i.e., more layers, which tend to perform better than shallow networks. Also, with fewer parameters, the new network has fewer tendencies to overfit the training data.

Abstract: In a method for super resolution imaging, the method includes: receiving, by a processor, a low resolution image; generating, by the processor, an intermediate high resolution image having an improved resolution compared to the low resolution image; generating, by the processor, a final high resolution image based on the intermediate high resolution image and the low resolution image; and transmitting, by the processor, the final high resolution image to a display device for display thereby.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a deep neural network. One of the methods includes generating a plurality of feature vectors that each model a different portion of an audio waveform, generating a first posterior probability vector for a first feature vector using a first neural network, determining whether one of the scores in the first posterior probability vector satisfies a first threshold value, generating a second posterior probability vector for each subsequent feature vector using a second neural network, wherein the second neural network is trained to identify the same key words and key phrases and includes more inner layer nodes than the first neural network, and determining whether one of the scores in the second posterior probability vector satisfies a second threshold value.

Abstract: A method for determining a final architecture for a neural network to perform a particular machine learning task is described.

Abstract: A calculation device includes: M coefficient storage units provided corresponding to the M coefficients, each of the M coefficient storage units including a positive-side coefficient and a negative-side coefficient representing a coefficient corresponding to a sign of a difference; M multiplication units provided corresponding to the M input values, each of the M multiplication units calculating a positive-side multiply value obtained by multiplying the positive-side coefficient included in the corresponding coefficient storage unit by a sign inverted according to the corresponding input value and a negative-side multiply value obtained by multiplying the negative-side coefficient included in the corresponding coefficient storage unit by a sign inverted according to the corresponding input value; and an output unit outputting an value according to a difference between a positive-side sum value obtained by summing the M positive-side multiplication values and a negative-side sum value obtained by summing the M

Abstract: A spiking neural network (SNN) includes artificial neurons interconnected by artificial synapses to model a particular network. A first neuron emits spikes to neighboring neurons to cause a wave of spikes to propagate through the SNN. Weights of a portion of the synapses are increased responsive to the wave of spikes based on a spike timing dependent plasticity (STDP) rule. A second neuron emits spike to cause a chain of spikes to propagate to the first neuron on a path based on the increase in the synaptic weights. The path is determined to represent a shortest path in the particular network from a first network node represented by the second neuron to a second network node represented by the first neuron.

Abstract: An artificial neural network system for image classification, formed of multiple independent individual convolutional neural networks (CNNs), each CNN being configured to process an input image patch to calculate a classification for the center pixel of the patch. The multiple CNNs have different receptive field of views for processing image patches of different sizes centered at the same pixel. A final classification for the center pixel is calculated by combining the classification results from the multiple CNNs. An image patch generator is provided to generate the multiple input image patches of different sizes by cropping them from the original input image. The multiple CNNs have similar configurations, and when training the artificial neural network system, one CNN is trained first, and the learned parameters are transferred to another CNN as initial parameters and the other CNN is further trained. The classification includes three classes, namely background, foreground, and edge.

Abstract: A three-dimensional integration of synapse circuitry is formed. One or more neuron layers each comprises a plurality of computing elements, and one or more synapse layers each comprising an array of memory elements are formed on top of the one or more neuron layers. A plurality of staggered through-silicon vias (TSVs) connect the one or more neuron layers to the one or more synapse layers and operate as communication links between one or more computing elements in the one or more neuron layers and one or more memory elements in the one or more synapse layers.

Abstract: Embodiments of the present invention provide a method for feature extraction comprising generating synaptic connectivity information for a neurosynaptic core circuit. The core circuit comprises one or more electronic neurons, one or more electronic axons, and an interconnect fabric including a plurality of synapse devices for interconnecting the neurons with the axons. The method further comprises initializing the interconnect fabric based on the synaptic connectivity information generated, and extracting a set of features from input received via the electronic axons. The set of features extracted comprises a set of features with reduced correlation.

Abstract: A distributed and shared memory controller (DSMC) comprises at least one building block. comprising a plurality of switches distributed into a plurality of stages; a plurality of master ports coupled to a first stage of the switches; and a plurality of bank controllers with associated memory banks coupled to a last stage of the switches; wherein each of the switches connects to lower stage switches via internal connections, each of the switches of the first stage connects to at least one of the master ports via master connections and each of the switches of the last stage connects to at least one of the bank controllers via memory connections; wherein each of the switches of the first stage connects to second stage switches of a neighboring building block via outward connections and each of the switches of a second stage connects to first stage switches of the neighboring building block via inward connections.

Abstract: An anomaly detection method includes generating second data by adding normal noise to first data; generating third data by adding abnormal noise to the first data; inputting the first data, the second data, and the third data to a neural network; calculating a first normal score, a second normal score, and a third normal score; calculating a third difference based on a first difference and a second difference, the first difference being based on a difference between the first normal score and the second normal score, the second difference being based on a difference between the first normal score and the third normal score; changing the neural network so that the third difference becomes smallest; inputting, to the changed neural network, fourth data that is unknown in terms of whether the fourth data is normal or abnormal; and determining whether the fourth data is normal or abnormal.

Abstract: Methods, apparatus, and system determine if a data class in a plurality of data classes is separable, such as by determining an average intra-class similarity within each data class, inter-class similarity across all data classes in the plurality of data classes, and determining separability based on the average intra-class similarity relative to the inter-class similarity. Data classes determined to be highly variable may be removed. Pair(s) of data classes not separable from one another may be combined into one class or one of the data classes may be dropped. A hardware accelerator, which may comprise artificial neurons, accelerate performance of the data analysis.

Abstract: One embodiment of the invention provides a system comprising at least one spike-to-data converter unit for converting spike event data generated by neurons to output numeric data. Each spike-to-data converter unit is configured to support one or more spike codes.

Abstract: A system for predicting future behavior for a dynamic system comprises a processor configured to implement an artificial intelligence system implementing nonlinear modeling and forecasting processing for analyzing the dynamic system. The nonlinear modeling and forecasting processing configures the processor to generate a time series group of data from the dynamic system. The nonlinear modeling and forecasting processing further configures the processor to generate prediction values of future behavior of the dynamic system by using the nonlinear modeling and forecasting implemented on the artificial intelligence system on the time series group of data.

Abstract: Some embodiments provide an IC for implementing a machine-trained network with multiple layers. The IC includes a set of circuits to compute a dot product of (i) a first number of input values computed by other circuits of the IC and (ii) a set of predefined weight values, several of which are zero, with a weight value for each of the input values. The set of circuits includes (i) a dot product computation circuit to compute the dot product based on a second number of inputs and (ii) for each input value, at least two sets of wires for providing the input value to at least two of the dot product computation circuit inputs. The second number is less than the first number. Each input value with a corresponding weight value that is not equal to zero is provided to a different one of the dot product computation circuit inputs.

Abstract: Systems, device and techniques are disclosed for outlier discovery system selection. A set of time series data including time series data objects may be received. A sample of time series data objects may be extracted from the time series data. The sample of time series data objects may be decomposed into sub-components. Statistical classification may be used to select an outlier discovery system based on the sub-components. A neural network may be used to select an outlier discovery system based on the sub-components. A level of error of the neural network may be determined based on a comparison of the outlier discovery system selection made using statistical classification and the outlier discovery system selection made by the neural network. Weight of the neural network may be updated based on the level of error of the neural network.

Abstract: A processor includes a front-end portion that issues instructions to execution units that execute the issued instructions. A hardware neural network unit (NNU) execution unit includes a first memory that holds data words associated with artificial neural networks (ANN) neuron outputs, a second memory that holds weight words associated with connections between ANN neurons, and a third memory that holds a program comprising NNU instructions that are distinct, with respect to their instruction set, from the instructions issued to the NNU by the front-end portion of the processor. The program performs ANN-associated computations on the data and weight words. A first instruction instructs the NNU to transfer NNU instructions of the program from architectural general purpose registers to the third memory. A second instruction instructs the NNU to invoke the program stored in the third memory.

Abstract: A data warehouse system may include a connection pool manager. The connection pool manager may invoke a first client-provided function for rerouting a connection request from a first database to a second database based on a query that is to be executed using the connection. The connection pool manager may invoke a second client-provided function for rewriting the query to be executed to leverage database aspects found on the second database.

Abstract: A neuromorphic memory circuit including a memory cell with a programmable resistive memory element. A postsynaptic capacitor builds up a leaky integrate and fire (LIF) charge. An axon LIF pulse generator activates a LIF discharge path from the postsynaptic capacitor through the resistive memory element when the axon LIF pulse generator generates axon LIF pulses. A postsynaptic comparator compares the capacitor voltage to a threshold voltage and generates postsynaptic output pulses when the capacitor voltage passes the threshold voltage. The postsynaptic output pulses include a postsynaptic firing characteristic dependent on a frequency of the axon LIF pulses. A refractory circuit prevents the postsynaptic comparator from generating additional postsynaptic output pulses until a refractory time passes since a preceding postsynaptic output pulse. A training circuit adjusts the postsynaptic firing characteristic to match a target firing characteristic.

Abstract: Embodiments of the invention provide a neural network comprising multiple functional neural core circuits, and a dynamically reconfigurable switch interconnect between the functional neural core circuits. The interconnect comprises multiple connectivity neural core circuits. Each functional neural core circuit comprises a first and a second core module. Each core module comprises a plurality of electronic neurons, a plurality of incoming electronic axons, and multiple electronic synapses interconnecting the incoming axons to the neurons. Each neuron has a corresponding outgoing electronic axon. In one embodiment, zero or more sets of connectivity neural core circuits interconnect outgoing axons in a functional neural core circuit to incoming axons in the same functional neural core circuit.

Abstract: A computer-implemented method is disclosed. The method may include receiving, at a neuron of an artificial neural network, a sequence of synapse messages. Each synapse message may include a timestamp of the time the synapse message was sent. The method may include determining, based on the timestamp of each synapse message, whether the neuron processed the sequence of synapse messages out of order with respect to the timestamps. The method may include, in response to the neuron processing the sequence of synapse messages out of order, reverse-computing at least one computation performed by the neuron in response to processing the sequence of synapse messages out of order. The method may include performing the at least one computation based on receipt of the sequence in a correct order as determined by the timestamp of each synapse message in the sequence of synapse messages.

Abstract: An apparatus and method are described for a neuromorphic processor design in which neuron timing information is duplicated on a neuromorphic core.

Abstract: Embodiments of the invention relate to a time-division multiplexed neurosynaptic module with implicit memory addressing for implementing a neural network. One embodiment comprises maintaining neuron attributes for multiple neurons and maintaining incoming firing events for different time steps. For each time step, incoming firing events for said time step are integrated in a time-division multiplexing manner. Incoming firing events are integrated based on the neuron attributes maintained. For each time step, the neuron attributes maintained are updated in parallel based on the integrated incoming firing events for said time step.

Abstract: A computer-implemented method is provided for optimizing core utilization in a neurosynaptic network. The computer-implemented method comprises identifying one or more unused portions of a neurosynaptic network. Additionally, the computer-implemented method comprises, for each of the one or more unused portions of the neurosynaptic network, disconnecting the unused portion from the neurosynaptic network.

Abstract: Systems, apparatuses, and methods for k-nearest neighbor (KNN) searches are described. In particular, embodiments of a KNN accelerator and its uses are described. In some embodiments, the KNN accelerator includes a plurality of vector partial distance computation circuits each to calculate a partial sum, a minimum sort network to sort partial sums from the plurality of vector partial distance computation circuits to find k nearest neighbor matches and a global control circuit to control aspects of operations of the plurality of vector partial distance computation circuits.

Abstract: A recognition system includes an image capturing device and a server. The image capturing device generates a Mth-layer calculation result based on image data and a convolutional neural network (CNN), and transmits feature information associated with the Mth-layer calculation result. M is a positive integer, M is equal to or greater than 1, M is less than or equal to N, and N is a predetermined positive integer. The server receives the feature information. The server generates a Kth-layer calculation result based on the feature information and the CNN by an iterative method when M is less than N. K is a positive integer which is greater than M and less than or equal to N. The server generates a first recognition result associated with the image data based on the Kth-layer calculation result and a first recognition model when K is equal to N.

Abstract: Methods and apparatus are provided for processing in an artificial nervous system. According to certain aspects, resolution of one or more functions performed by processing units of a neuron model may be reduced, based at least in part on availability of computational resources or a power target or budget. The reduction in resolution may be compensated for by adjusting one or more network weights.

Abstract: A process of using a logical entanglement device such as a non-volatile logic gate as a failsafe to constrain the behavior of an autonomous machine controlled by an artificial intelligence (AI). Such a device may be employed to extend an AI self-boundary to include other objects or entities such as humans. This logical entanglement device may act much like a mirror neuron and cause the AI to respond to human nonfunctionality or suffering as if it were its own, causing the AI's behavior to reliably mimic empathy and compassion when interacting with humans and limiting the possibility of the AI devaluing the functionality and well-being of humans.

Abstract: A memory cell structure includes a synapse memory cell including plural cell components, each of the plural cell components including a unit cell, plural write lines arranged for writing a synapse state to the synapse memory cell, each of the plural write lines being used for writing one of a first set of a predetermined number of states to a corresponding cell component by writing one of a second set of the predetermined number of states to the unit cell included in the corresponding cell component, the first set depending on the second set and a number of the unit cell included in the corresponding cell component, and a read line arranged for reading the synapse state from the synapse memory cell, the read line being used for reading one of the first set of the predetermined number of states from all of the plural cell components simultaneously.

Abstract: Embodiments of the present invention provide a method for feature extraction comprising generating synaptic connectivity information for a neurosynaptic core circuit. The core circuit comprises one or more electronic neurons, one or more electronic axons, and an interconnect fabric including a plurality of synapse devices for interconnecting the neurons with the axons. The method further comprises initializing the interconnect fabric based on the synaptic connectivity information generated, and extracting a set of features from input received via the electronic axons. The set of features extracted comprises a set of features with reduced correlation.

Abstract: One embodiment of the invention provides a system comprising at least one spike-to-data converter unit for converting spike event data generated by neurons to output numeric data. Each spike-to-data converter unit is configured to support one or more spike codes.

Abstract: One embodiment of the invention provides a system comprising at least one data-to-spike converter unit for converting input numeric data received by the system to spike event data. Each data-to-spike converter unit is configured to support one or more spike codes.

Abstract: Computerized appliances may be operated by users remotely. In one exemplary implementation, a learning controller apparatus may be operated to determine association between a user indication and an action by the appliance. The user indications, e.g., gestures, posture changes, audio signals may trigger an event associated with the controller. The event may be linked to a plurality of instructions configured to communicate a command to the appliance. The learning apparatus may receive sensory input conveying information about robot's state and environment (context). The sensory input may be used to determine the user indications. During operation, upon determine the indication using sensory input, the controller may cause execution of the respective instructions in order to trigger action by the appliance. Device animation methodology may enable users to operate computerized appliances using gestures, voice commands, posture changes, and/or other customized control elements.

Abstract: Embodiments of the invention relate to a time-division multiplexed neurosynaptic module with implicit memory addressing for implementing a universal substrate of adaptation. One embodiment comprises a neurosynaptic device including a memory device that maintains neuron attributes for multiple neurons. The module further includes multiple bit maps that maintain incoming firing events for different periods of delay and a multi-way processor. The processor includes a memory array that maintains a plurality of synaptic weights. The processor integrates incoming firing events in a time-division multiplexing manner. Incoming firing events are integrated based on the neuron attributes and the synaptic weights maintained.

Abstract: Reduction in signal intensity of a harmonic component included in an output of a delta-sigma modulator is suppressed. A signal processing device includes: a delta-sigma modulator 11 that outputs a pulse signal; a first processor 12 that generates, from the pulse signal PO outputted from the delta-sigma modulator 11, a discontinuous pulse signal PC in which each of one-pulse sections in the pulse signal PO has a low level region on at least one of a rear end and a front end of the one-pulse section; and a second processor that generates a short-width pulse signal PS having a pulse width shorter than a pulse width of the discontinuous pulse signal PC generated by the first processor 12.

Abstract: Embodiments of the invention relate to a globally asynchronous and locally synchronous neuromorphic network. One embodiment comprises generating a synchronization signal that is distributed to a plurality of neural core circuits. In response to the synchronization signal, in at least one core circuit, incoming spike events maintained by said at least one core circuit are processed to generate an outgoing spike event. Spike events are asynchronously communicated between the core circuits via a routing fabric comprising multiple asynchronous routers.

Abstract: The present invention provides an event-driven universal neural network circuit. The circuit comprises a plurality of neural modules. Each neural module comprises multiple digital neurons such that each neuron in a neural module has a corresponding neuron in another neural module. An interconnection network comprising a plurality of digital synapses interconnects the neural modules. Each synapse interconnects a first neural module to a second neural module by interconnecting a neuron in the first neural module to a corresponding neuron in the second neural module. Corresponding neurons in the first neural module and the second neural module communicate via the synapses. Each synapse comprises a learning rule associating a neuron in the first neural module with a corresponding neuron in the second neural module. A control module generates signals which define a set of time steps for event-driven operation of the neurons and event communication via the interconnection network.

Abstract: A method comprising providing an input signal to at least one input node of a computing reservoir by temporally encoding the input signal by modulating the at least one photonic wave as function of the input signal is described. The method further comprises propagating the at least one photonic wave via passive guided or unguided propagation between discrete nodes of the computing reservoir, in which each discrete node is adapted for passively relaying the at least one photonic wave over the passive interconnections connected thereto. The method also comprises obtaining a plurality of readout signals, in which each readout signal is determined by a non-linear relation to the at least one photonic wave in at least one readout node of the computing reservoir, and combining this plurality of readout signals into an output signal by taking into account a plurality of training parameters.

Abstract: Methods and apparatus are provided for effecting modulation using global scalar values in a spiking neural network. One example method for operating an artificial nervous system generally includes determining one or more updated values for artificial neuromodulators to be used by a plurality of entities in a neuron model and providing the updated values to the plurality of entities.

Abstract: Apparatus and methods for processing inputs by one or more neurons of a network. The neuron(s) may generate spikes based on receipt of multiple inputs. Latency of spike generation may be determined based on an input magnitude. Inputs may be scaled using for example a non-linear concave transform. Scaling may increase neuron sensitivity to lower magnitude inputs, thereby improving latency encoding of small amplitude inputs. The transformation function may be configured compatible with existing non-scaling neuron processes and used as a plug-in to existing neuron models. Use of input scaling may allow for an improved network operation and reduce task simulation time.