Abstract: A system, having a memory that stores computer executable components, and a processor that executes the computer executable components, reduces data size in connection with training a neural network by exploiting spatial locality to weight matrices and effecting frequency transformation and compression. A receiving component receives neural network data in the form of a compressed frequency-domain weight matrix. A segmentation component segments the initial weight matrix into original sub-components, wherein respective original sub-components have spatial weights. A sampling component applies a generalized weight distribution to the respective original sub-components to generate respective normalized sub-components. A transform component applies a transform to the respective normalized sub-components.

Abstract: The present disclosure describes a system and method for preempting a long-running process with a higher priority process in a machine learning system, such as a hardware accelerator. The machine learning hardware accelerator can be a multi-chip system including semiconductor chips that can be application-specific integrated circuits (ASIC) designed to perform machine learning operations. An ASIC is an integrated circuit (IC) that is customized for a particular use.

Abstract: Components on an IC chip may operate faster or provide higher performance relative to power consumption if allowed access to sufficient memory resources. If every component is provided its own memory, however, the chip becomes expensive. In described implementations, memory is shared between two or more components. For example, a processing component can include computational circuitry and a memory coupled thereto. A multi-component cache controller is coupled to the memory. Logic circuitry is coupled to the cache controller and the memory. The logic circuitry selectively separates the memory into multiple memory partitions. A first memory partition can be allocated to the computational circuitry and provide storage to the computational circuitry. A second memory partition can be allocated to the cache controller and provide storage to multiple components.

Abstract: Methods, systems, and apparatus, including computer-readable media, are described for a hardware circuit configured to implement a neural network. The circuit includes multiple super tiles. Each super tile includes a unified memory for storing inputs to a neural network layer and weights for the layer. Each super tile includes multiple compute tiles. Each compute tile executes a compute thread that is used to perform the computations to generate an output for the neural network layer. Each super tile includes arbitration logic coupled to the unified memory and each compute tile. The arbitration logic is configured to: pass inputs stored in the unified memory to the compute tiles; pass weights stored in the unified memory to the compute tiles; and pass, to the unified memory, the output generated for the layer based on computations performed at the compute tiles using the inputs and the weights for the layer.

Abstract: Technical solutions are described to accelerate training of a multi-layer convolutional neural network. According to one aspect, a computer implemented method is described. A convolutional layer includes input maps, convolutional kernels, and output maps. The method includes a forward pass, a backward pass, and an update pass that each include convolution calculations. The described method performs the convolutional operations involved in the forward, the backward, and the update passes based on a first, a second, and a third perforation map respectively. The perforation maps are stochastically generated, and distinct from each other. The method further includes interpolating results of the selective convolution operations to obtain remaining results. The method includes iteratively repeating the forward pass, the backward pass, and the update pass until the convolutional neural network is trained. Other aspects such as a system, apparatus, and computer program product are also described.

Abstract: A method (and system) for generating random numbers includes setting a drain voltage Vd on an MOSFET (metal oxide semiconductor field effect transistor) device and a gate voltage Vg of the MOSFET device so that the MOSFET device comprises a noise source configured in a manner such as to tune as desired a random number statistical distribution of an output of the MOSFET device. An output voltage of the MOSFET is provided as an input signal into a low noise amplifier and an output voltage of the low noise amplifier provides values for a random number generator.

Abstract: Technical solutions are described to accelerate training of a multi-layer convolutional neural network. According to one aspect, a computer implemented method is described. A convolutional layer includes input maps, convolutional kernels, and output maps. The method includes a forward pass, a backward pass, and an update pass that each include convolution calculations. The described method performs the convolutional operations involved in the forward, the backward, and the update passes based on a first, a second, and a third perforation map respectively. The perforation maps are stochastically generated, and distinct from each other. The method further includes interpolating results of the selective convolution operations to obtain remaining results. The method includes iteratively repeating the forward pass, the backward pass, and the update pass until the convolutional neural network is trained. Other aspects such as a system, apparatus, and computer program product are also described.

Abstract: Techniques facilitating synchronization of processing engines for parallel deep learning are provided. In one example, a first processing component associated with a processor and processing components can: generate first output data based on input data associated with a machine learning process, wherein the processing components are communicatively coupled with an assignment component via a network; transmit the first output data to a second processing component of the processing components, wherein the first processing component and the second processing component comprise a first group of the processing components and the first group of the processing components is determined by the assignment component based on a first defined criterion; receive communication data generated by the second processing component; and generate second output data based on the communication data, wherein the second output data is an updated version of the first output data stored in the memory of the first processing component.

Abstract: A system, having a memory that stores computer executable components, and a processor that executes the computer executable components, reduces data size in connection with training a neural network by exploiting spatial locality to weight matrices and effecting frequency transformation and compression. A receiving component receives neural network data in the form of a compressed frequency-domain weight matrix. A segmentation component segments the initial weight matrix into original sub-components, wherein respective original sub-components have spatial weights. A sampling component applies a generalized weight distribution to the respective original sub-components to generate respective normalized sub-components. A transform component applies a transform to the respective normalized sub-components.

Abstract: A method (and system) for generating random numbers includes setting a drain voltage Vd on an MOSFET (metal oxide semiconductor field effect transistor) device and a gate voltage Vg of the MOSFET device so that the MOSFET device comprises a noise source configured in a manner such as to tune as desired a random number statistical distribution of an output of the MOSFET device. An output voltage of the MOSFET is provided as an input signal into a low noise amplifier and an output voltage of the low noise amplifier provides values for a random number generator.

Abstract: A method (and system) for generating random numbers includes setting a drain voltage Vd on an MOSFET device to maximize a transconductance of the MOSFET device and setting a gate voltage Vg of the MOSFET device to tune as desired a random number statistical distribution of an output of the MOSFET device. The MOSFET device includes a gate structure with an oxide layer including at least one artificial trapping layer in which carrier traps are designed to occupy a predetermined distance from conduction and valance bands of material of the artificial trapping layer.

Abstract: An apparatus is presented for generating a true random number generator (TRNG). The apparatus includes a magnetic tunnel junction (MTJ) device including a first layer, a second layer, and third layer, as well as a bias circuit to bias the MTJ device along with a pulse height discriminator and a time-to-amplitude convertor to generate random bit-streams. The second layer is a barrier layer with an energy barrier height in the order of 20 kT, where k is the Boltzmann constant and T is the absolute temperature. Random flipping of an orientation of magnetization of the third layer is induced by thermal fluctuations in the MTJ device.

Abstract: An apparatus is presented for generating a true random number generator (TRNG). The apparatus includes a magnetic tunnel junction (MTJ) device including a first layer, a second layer, and third layer, as well as a bias circuit to bias the MTJ device along with a pulse height discriminator and a time-to-amplitude convertor to generate random bit-streams. The second layer is a barrier layer with an energy barrier height in the order of 20kT, where k is the Boltzmann constant and T is the absolute temperature. Random flipping of an orientation of magnetization of the third layer is induced by thermal fluctuations in the MTJ device.

Abstract: A method includes forming a tunneling dielectric layer on a semiconductor substrate, a first portion of the tunneling dielectric layer is directly above a channel region in the semiconductor substrate and a second portion of the tunneling dielectric layer is directly above source-drain regions located on opposing sides of the channel region, the second portion of the tunneling dielectric layer is thicker than the first portion of the tunneling dielectric layer, forming a floating gate directly above the first portion of the tunneling dielectric layer and the second portion of the tunneling dielectric layer, and forming a control dielectric layer directly above the floating gate.

Abstract: A method includes forming a tunneling dielectric layer on a semiconductor substrate, a first portion of the tunneling dielectric layer is directly above a channel region in the semiconductor substrate and a second portion of the tunneling dielectric layer is directly above source-drain regions located on opposing sides of the channel region, the second portion of the tunneling dielectric layer is thicker than the first portion of the tunneling dielectric layer, forming a floating gate directly above the first portion of the tunneling dielectric layer and the second portion of the tunneling dielectric layer, and forming a control dielectric layer directly above the floating gate.

Abstract: A method includes forming a tunneling dielectric layer on a semiconductor substrate, a first portion of the tunneling dielectric layer is directly above a channel region in the semiconductor substrate and a second portion of the tunneling dielectric layer is directly above source-drain regions located on opposing sides of the channel region, the second portion of the tunneling dielectric layer is thicker than the first portion of the tunneling dielectric layer, forming a floating gate directly above the first portion of the tunneling dielectric layer and the second portion of the tunneling dielectric layer, and forming a control dielectric layer directly above the floating gate.

Abstract: A method includes forming a tunneling dielectric layer on a semiconductor substrate, a first portion of the tunneling dielectric layer is directly above a channel region in the semiconductor substrate and a second portion of the tunneling dielectric layer is directly above source-drain regions located on opposing sides of the channel region, the second portion of the tunneling dielectric layer is thicker than the first portion of the tunneling dielectric layer, forming a floating gate directly above the first portion of the tunneling dielectric layer and the second portion of the tunneling dielectric layer, and forming a control dielectric layer directly above the floating gate.

Abstract: A method includes forming a tunneling dielectric layer on a semiconductor substrate, a first portion of the tunneling dielectric layer is directly above a channel region in the semiconductor substrate and a second portion of the tunneling dielectric layer is directly above source-drain regions located on opposing sides of the channel region, the second portion of the tunneling dielectric layer is thicker than the first portion of the tunneling dielectric layer, forming a floating gate directly above the first portion of the tunneling dielectric layer and the second portion of the tunneling dielectric layer, and forming a control dielectric layer directly above the floating gate.

Abstract: Techniques facilitating synchronization of processing engines for parallel deep learning are provided. In one example, a first processing component associated with a processor and processing components can: generate first output data based on input data associated with a machine learning process, wherein the processing components are communicatively coupled with an assignment component via a network; transmit the first output data to a second processing component of the processing components, wherein the first processing component and the second processing component comprise a first group of the processing components and the first group of the processing components is determined by the assignment component based on a first defined criterion; receive communication data generated by the second processing component; and generate second output data based on the communication data, wherein the second output data is an updated version of the first output data stored in the memory of the first processing component.

Abstract: A method includes forming a tunneling dielectric layer on a semiconductor substrate, a first portion of the tunneling dielectric layer is directly above a channel region in the semiconductor substrate and a second portion of the tunneling dielectric layer is directly above source-drain regions located on opposing sides of the channel region, the second portion of the tunneling dielectric layer is thicker than the first portion of the tunneling dielectric layer, forming a floating gate directly above the first portion of the tunneling dielectric layer and the second portion of the tunneling dielectric layer, and forming a control dielectric layer directly above the floating gate.