Abstract: Disclosed are systems and methods for performing efficient vector-matrix multiplication using a sparsely-connected conductance matrix and analog mixed signal (AMS) techniques. Metal electrodes are sparsely connected using coaxial nanowires. Each electrode can be used as an input/output node or neuron in a neural network layer. Neural network synapses are created by random connections provided by coaxial nanowires. A subset of the metal electrodes can be used to receive a vector of input voltages and the complementary subset of the metal electrodes can be used to read output currents. The output currents are the result of vector-matrix multiplication of the vector of input voltages with the sparsely-connected matrix of conductances.

Abstract: A method for performing learning is described. A free inference is performed on a learning network for input signals. The input signals correspond to target output signals. The learning network includes inputs that receive the input signals, neurons, weights interconnecting the neurons, and outputs. The learning network is described by an energy for the free inference. The energy includes an interaction term corresponding to interactions consisting of neuron pair interactions. The free inference results in output signals. A first portion of the plurality of weights corresponding to data flow for the free inference. A biased inference is performed on the learning network by providing the input signals to the inputs and bias signals to the outputs. The bias signals are based on the target output signals and the output signals. The bias signals are fed back to the learning network through a second portion of the weights corresponding to a transpose of the first portion of the weights.

Abstract: A computing device is described. The computing device includes first and second arrays of compute units and first and second arrays of routers. The first array of compute units is arranged on a first substrate and includes a first plurality of compute-in-memory (CIM) modules. The first array of routers is configured to route information horizontally among the first array of compute units. The second array of compute units is arranged on a second substrate and includes a second plurality of CIM modules. The second substrate is disposed vertically from the first substrate. The second array of routers is configured to route the information horizontally among the second array of compute units on the second substrate. The first array of routers and the second array of routers send the information vertically between the first substrate and the second substrate.

Abstract: A method for performing learning in a dissipative learning network is described. The method includes determining a trajectory for the dissipative learning network and determining a perturbed trajectory for the dissipative learning network based on a plurality of target outputs. Gradients for a portion of the dissipative learning network are determined based on the trajectory and the perturbed trajectory. The portion of the dissipative learning network is adjusted based on the gradients.

Abstract: A memristive device and mechanisms for providing and using the memristive device are described. The memristive device includes a nanowire, a plurality of memristive plugs and a plurality of electrodes. The nanowire has a conductive core and an insulator coating at least a portion of the conductive core. The insulator has a plurality of apertures therein. The memristive plugs are for the apertures. At least a portion of each of the memristive plugs resides in each of the apertures. The memristive plugs are between the conductive core and the electrodes.

Abstract: A method of training a learning network is described. The method includes generating a first estimate of a gradient for the learning network and generating subsequent estimates of the gradient using a feedback network. The feedback network generates improved perturbations for the subsequent gradient estimates. Gradient estimates include the first estimate of the gradient and the subsequent estimates of the gradient. The method also includes using the gradient estimates to determine weights in the learning network. The improved perturbations may include lower variance perturbations.

Abstract: A system includes inputs, outputs, and nodes between the inputs and the outputs. The nodes include hidden nodes. Connections between the nodes are determined based on a gradient computable using symmetric solution submatrices.

Abstract: A method for performing learning is described. A free inference is performed on a learning network for input signals. The input signals correspond to target output signals. The learning network includes inputs that receive the input signals, neurons, weights interconnecting the neurons, and outputs. The learning network is described by an energy for the free inference. The energy includes an interaction term corresponding to interactions consisting of neuron pair interactions. The free inference results in output signals. A first portion of the plurality of weights corresponding to data flow for the free inference. A biased inference is performed on the learning network by providing the input signals to the inputs and bias signals to the outputs. The bias signals are based on the target output signals and the output signals. The bias signals are fedback to the learning network through a second portion of the weights corresponding to a transpose of the first portion of the weights.

Abstract: A method of training a learning network is described. The method includes generating a first estimate of a gradient for the learning network and generating subsequent estimates of the gradient using a feedback network. The feedback network generates improved perturbations for the subsequent gradient estimates. Gradient estimates include the first estimate of the gradient and the subsequent estimates of the gradient. The method also includes using the gradient estimates to determine weights in the learning network. The improved perturbations may include lower variance perturbations.

Abstract: A memristive device is described. The memristive device includes a first layer having a first plurality of conductive lines, a second layer having a second plurality of conductive lines, and memristive interlayer connectors. The first and second layers differ. The first and second pluralities of conductive lines are each lithographically defined. The first and second pluralities of conductive lines are insulated from each other. The memristive interlayer connectors are memristively coupled with a first portion of the first plurality of conductive lines and memristively coupled with a second portion of the second plurality of conductive lines. The memristive interlayer connectors are thus sparsely coupled with the first and second pluralities of conductive lines. Each memristive interlayer connector includes a conductive portion and a memristive portion.

Abstract: A method for performing learning in a dissipative learning network is described. The method includes determining a trajectory for the dissipative learning network and determining a perturbed trajectory for the dissipative learning network based on a plurality of target outputs. Gradients for a portion of the dissipative learning network are determined based on the trajectory and the perturbed trajectory. The portion of the dissipative learning network is adjusted based on the gradients.

Abstract: A memristive device and mechanisms for providing and using the memristive device are described. The memristive device includes a nanowire, a plurality of memristive plugs and a plurality of electrodes. The nanowire has a conductive core and an insulator coating at least a portion of the conductive core. The insulator has a plurality of apertures therein. The memristive plugs are for the apertures. At least a portion of each of the memristive plugs resides in each of the apertures. The memristive plugs are between the conductive core and the electrodes.

Abstract: Disclosed are systems and methods for performing efficient vector-matrix multiplication using a sparsely-connected conductance matrix and analog mixed signal (AMS) techniques. Metal electrodes are sparsely connected using coaxial nanowires. Each electrode can be used as an input/output node or neuron in a neural network layer. Neural network synapses are created by random connections provided by coaxial nanowires. A subset of the metal electrodes can be used to receive a vector of input voltages and the complementary subset of the metal electrodes can be used to read output currents. The output currents are the result of vector-matrix multiplication of the vector of input voltages with the sparsely-connected matrix of conductances.

Abstract: Disclosed are systems and methods for performing efficient vector-matrix multiplication using a sparsely-connected conductance matrix and analog mixed signal (AMS) techniques. Metal electrodes are sparsely connected using coaxial nanowires. Each electrode can be used as an input/output node or neuron in a neural network layer. Neural network synapses are created by random connections provided by coaxial nanowires. A subset of the metal electrodes can be used to receive a vector of input voltages and the complementary subset of the metal electrodes can be used to read output currents. The output currents are the result of vector-matrix multiplication of the vector of input voltages with the sparsely-connected matrix of conductances.