Abstract: A deep neural network (“DNN”) module can determine whether processing of certain values in an input buffer or a weight buffer by neurons can be skipped. For example, the DNN module might determine whether neurons can skip the processing of values in entire columns of a neuron buffer. Processing of these values might be skipped if an entire column of an input buffer or a weight buffer are zeros, for example. The DNN module can also determine whether processing of single values in rows of the input buffer or the weight buffer can be skipped (e.g. if the values are zero). Neurons that complete their processing early as a result of skipping operations can assist other neurons with their processing. A combination operation can be performed following the completion of processing that transfers the results of the processing operations performed by a neuron to their correct owner.

Abstract: The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. Using a directed line buffer that operatively inserts one or more shifting bits in data blocks to be processed, data read/writes to the line buffer can be optimized for processing by the NN/DNN thereby enhancing the overall performance of a NN/DNN. Operatively, an operations controller and/or iterator can generate one or more instructions having a calculated shifting bit(s) for communication to the line buffer. Illustratively, the shifting bit(s) can be calculated using various characteristics of the input data as well as the NN/DNN inclusive of the data dimensions. The line buffer can read data for processing, insert the shifting bits and write the data in the line buffer for subsequent processing by cooperating processing unit(s).

Abstract: A deep neural network (“DNN”) module can compress and decompress neuron-generated activation data to reduce the utilization of memory bus bandwidth. The compression unit can receive an uncompressed chunk of data generated by a neuron in the DNN module. The compression unit generates a mask portion and a data portion of a compressed output chunk. The mask portion encodes the presence and location of the zero and non-zero bytes in the uncompressed chunk of data. The data portion stores truncated non-zero bytes from the uncompressed chunk of data. A decompression unit can receive a compressed chunk of data from memory in the DNN processor or memory of an application host. The decompression unit decompresses the compressed chunk of data using the mask portion and the data portion. This can reduce memory bus utilization, allow a DNN module to complete processing operations more quickly, and reduce power consumption.

Abstract: The performance of a neural network (NN) can be limited by the number of operations being performed. Using a line buffer that is directed to shift a memory block by a selected shift stride for cooperating neurons, data that is operatively residing memory and which would require multiple write cycles into a cooperating line buffer can be processed as in a single line buffer write cycle thereby enhancing the performance of a NN/DNN. A controller and/or iterator can generate one or more instructions having the memory block shifting values for communication to the line buffer. The shifting values can be calculated using various characteristics of the input data as well as the NN/DNN inclusive of the data dimensions. The line buffer can read data for processing, shift the data of the memory block and write the data in the line buffer for subsequent processing.

Abstract: An exemplary artificial intelligence/machine learning hardware computing environment having an exemplary DNN module cooperating with one or more memory components can perform data sharing and distribution as well reuse of a buffer data to reduce the number of memory component read/writes thereby enhancing overall hardware performance and reducing power consumption. Illustratively, data from a cooperating memory component is read according to a selected operation of the exemplary hardware and written to corresponding other memory component for use by one or more processing elements (e.g., neurons). The data is read in such a manner to optimize the engagement of the one or more processing elements for each processing cycle as well as to reuse data previously stored in the one or more cooperating memory components. Operatively, the written data is copied to a shadow memory buffer prior to being consumed by the processing elements.

Abstract: The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. Using virtualized hardware iterators, data for processing by the NN/DNN can be traversed and configured to optimize the number of operations as well as memory utilization to enhance the overall performance of a NN/DNN. Operatively, an iterator controller can generate instructions for execution by the NN/DNN representative of one more desired iterator operation types and to perform one or more iterator operations. Data can be iterated according to a selected iterator operation and communicated to one or more neuron processors of the NN/DD for processing and output to a destination memory. The iterator operations can be applied to various volumes of data (e.g., blobs) in parallel or multiple slices of the same volume.

Abstract: A deep neural network (DNN) module utilizes parallel kernel and parallel input processing to decrease bandwidth utilization, reduce power consumption, improve neuron multiplier stability, and provide other technical benefits. Parallel kernel processing enables the DNN module to load input data only once for processing by multiple kernels. Parallel input processing enables the DNN module to load kernel data only once for processing with multiple input data. The DNN module can implement other power-saving techniques like clock-gating (i.e. removing the clock from) and power-gating (i.e. removing the power from) banks of accumulators based upon usage of the accumulators. For example, individual banks of accumulators can be power-gated when all accumulators in a bank are not in use, and do not store data for a future calculation. Banks of accumulators can also be clock-gated when all accumulators in a bank are not in use, but store data for a future calculation.

Abstract: A deep neural network (DNN) processor is configured to execute descriptors in layer descriptor lists. The descriptors define instructions for performing a pass of a DNN by the DNN processor. Several types of descriptors can be utilized: memory-to-memory move (M2M) descriptors; operation descriptors; host communication descriptors; configuration descriptors; branch descriptors; and synchronization descriptors. A DMA engine uses M2M descriptors to perform multi-dimensional strided DMA operations. Operation descriptors define the type of operation to be performed by neurons in the DNN processor and the activation function to be used by the neurons. M2M descriptors are buffered separately from operation descriptors and can be executed at soon as possible, subject to explicitly set dependencies. As a result, latency can be reduced and, consequently, the neurons can complete their processing faster. The DNN module can then be powered down earlier than it otherwise would have, thereby saving power.

Abstract: Hardware accelerated synchronization of data movement across multiple direct memory access (DMA) engines is provided using techniques in which the order of descriptor processing is guaranteed for scenarios involving a single CPU and multiple DMA engines as well as those involving multiple CPUs and multiple DMA engines.

Abstract: Hardware accelerated synchronization of data movement across multiple direct memory access (DMA) engines is provided using techniques in which the order of descriptor processing is guaranteed for scenarios involving a single CPU and multiple DMA engines as well as those involving multiple CPUs and multiple DMA engines.

Abstract: Hardware accelerated synchronization of data movement across multiple direct memory access (DMA) engines is provided using techniques in which the order of descriptor processing is guaranteed for scenarios involving a single CPU and multiple DMA engines as well as those involving multiple CPUs and multiple DMA engines.

Abstract: Hardware accelerated synchronization of data movement across multiple direct memory access (DMA) engines is provided using techniques in which the order of descriptor processing is guaranteed for scenarios involving a single CPU and multiple DMA engines as well as those involving multiple CPUs and multiple DMA engines.

Abstract: A method and an apparatus for controlling voltage level and clock signal frequency supplied to a system. The apparatus includes a hardware module, adapted to receive at least one indication of a load of the system and to determine a voltage level and a clock signal frequency to be provided to the system, and a software module, adapted to configure a voltage source and a clock signal source in response to the determination. The method includes: (i) receiving, at a hardware module, indication of a load of a system; (ii) determining, by the hardware module, a voltage level and a clock signal frequency to be provided to the system; and (iii) configuring, by a software module, a voltage source and a clock signal source in response to the determination.

Abstract: A method for determining system and software configuration that includes: calculating a power consumption estimate of a modeled system associated with an execution of a certain software code; and altering, in response to the power consumption estimate, the certain software code or the modeled system. A method of determining a power consumption of a system that executed a software code, the method includes the stages of: providing a reduced instruction set representation of the software code; and calculating a power consumption estimate of a modeled system associated with an execution of the reduced instruction set representation of the software code.

Abstract: A method and an apparatus for controlling voltage level and clock signal frequency supplied to a system. The apparatus includes a hardware module, adapted to receive at least one indication of a load of the system and to determine a voltage level and a clock signal frequency to be provided to the system, and a software module, adapted to configure a voltage source and a clock signal source in response to the determination. The method includes: (i) receiving, at a hardware module, indication of a load of a system; (ii) determining, by the hardware module, a voltage level and a clock signal frequency to be provided to the system; and (iii) configuring, by a software module, a voltage source and a clock signal source in response to the determination.

Abstract: A device for regulating a voltage supply to a semiconductor device, the device comprising memory for storing a plurality of performance ranges, wherein the respective performance ranges are associated with a respective supply voltage; means for measuring the performance of the semiconductor device; and a regulator for modifying the supply voltage to the semiconductor device if the measured performance of the semiconductor device is not within a predetermined portion of the performance range associated with the voltage supplied to the semiconductor device.