Abstract: Processors and methods for neural network processing are provided. A method in a processor including a matrix vector unit is provided. The method includes receiving vector data and actuation vector data corresponding to at least one layer of a neural network model for processing using the matrix vector unit, where each of digital values corresponding to the vector data and the actuation vector data is represented in a sign magnitude format. The method further includes converting each of the digital values corresponding to at least one of the vector data or the actuation vector data to corresponding analog values and multiplying the vector data and the actuation vector data in an analog domain and providing corresponding multiplication results in a digital domain.

Abstract: Aspects extend to methods, systems, and computer program products for partially reconfiguring acceleration components. Partial reconfiguration can be implemented for any of a variety of reasons, including to address an error in functionality at the acceleration component or to update functionality at the acceleration component. During partial reconfiguration, connectivity can be maintained for any other functionality at the acceleration component untouched by the partial reconfiguration. Partial reconfiguration is more efficient to deploy than full reconfiguration of an acceleration component.

Abstract: Aspects extend to methods, systems, and computer program products for allocating acceleration component functionality for supporting services. A service manager uses a finite number of acceleration components to accelerate services. Acceleration components can be allocated in a manner that balances load in a hardware acceleration plane, minimizes role switching, and adapts to demand changes. When role switching is appropriate, less extensive mechanisms (e.g., based on configuration data versus image files) can be used to switch roles to the extent possible.

Abstract: Perplexity scores are computed for training data samples during ANN training. Perplexity scores can be computed as a divergence between data defining a class associated with a current training data sample and a probability vector generated by the ANN model. Perplexity scores can alternately be computed by learning a probability density function (“PDF”) fitting activation maps generated by an ANN model during training. A perplexity score can then be computed for a current training data sample by computing a probability for the current training data sample based on the PDF. If the perplexity score for a training data sample is lower than a threshold, the training data sample is removed from the training data set so that it will not be utilized for training during subsequent epochs. Training of the ANN model continues following the removal of training data samples from the training data set.

Abstract: Systems and methods are disclosed for supporting debugging of programs in block-based processor architectures. In one example of the disclosed technology, a processor includes a block-based processor core for executing an instruction block comprising an instruction header and a plurality of instructions. The block-based processor core includes execution control logic and core state access logic. The execution control logic can be configured to schedule respective instructions of the plurality of instructions for execution in a dynamic order during a default execution mode and to schedule the respective instructions for execution in a static order during a debug mode. The core state access logic can be configured to read intermediate states of the block-based processor core and to provide the intermediate states outside of the block-based processor core during the debug mode.

Abstract: Systems, apparatuses, and methods related to a block-based processor core topology register are disclosed. In one example of the disclosed technology, a processor can include a plurality of block-based processor cores for executing a program including a plurality of instruction blocks. A respective block-based processor core can include a sharable resource and a programmable composition topology register. The programmable composition topology register can be used to assign a group of the physical processor cores that share the sharable resource.

Abstract: Technology related to incremental training of machine learning tools is disclosed. In one example of the disclosed technology, a method can include receiving operational parameters of a machine learning tool based on a primary set of training data. The machine learning tool can be a deep neural network. Input data can be applied to the machine learning tool to generate an output of the machine learning tool. A measure of prediction quality can be generated for the output of the machine learning tool. In response to determining the measure of prediction quality is below a threshold, incremental training of the operational parameters can be initiated using the input data as training data for the machine learning tool. Operational parameters of the machine learning tool can be updated based on the incremental training. The updated operational parameters can be stored.

Abstract: Technology related to prefetching instruction blocks is disclosed. In one example of the disclosed technology, a processor comprises a block-based processor core for executing a program comprising a plurality of instruction blocks. The block-based processor core can include prefetch logic and a local buffer. The prefetch logic can be configured to receive a reference to a predicted instruction block and to determine a mapping of the predicted instruction block to one or more lines. The local buffer can be configured to selectively store portions of the predicted instruction block and to provide the stored portions of the predicted instruction block when control of the program passes along a predicted execution path to the predicted instruction block.

Abstract: Apparatus and methods for training neural networks based on a performance metric, including adjusting numerical precision and topology as training progresses are disclosed. In some examples, block floating-point formats having relatively lower accuracy are used during early stages of training. Accuracy of the floating-point format can be increased as training progresses based on a determined performance metric. In some examples, values for the neural network are transformed to normal precision floating-point formats. The performance metric can be determined based on entropy of values for the neural network, accuracy of the neural network, or by other suitable techniques. Accelerator hardware can be used to implement certain implementations, including hardware having direct support for block floating-point formats.

Abstract: Methods and apparatus are disclosed for adjusting hyper-parameters of a neural network to compensate for noise, such as noise introduced via quantization of one or more parameters of the neural network. In some examples, the adjustment can include scaling the hyper-parameter based on at least one metric representing noise present in the neural network. The at least one metric can include a noise-to-signal ratio for weights of the neural network, such as edge weights and activation weights. In a quantized neural network, a learning rate hyper-parameter used to compute a gradient update for a layer during back propagation can be scaled based on the at least one metric. In some examples, the same scaled learning rate can be used when computing gradient updates for other layers.

Abstract: A system for block floating point computation in a neural network receives a block floating point number comprising a mantissa portion. A bit-width of the block floating point number is reduced by decomposing the block floating point number into a plurality of numbers each having a mantissa portion with a bit-width that is smaller than a bit-width of the mantissa portion of the block floating point number. One or more dot product operations are performed separately on each of the plurality of numbers to obtain individual results, which are summed to generate a final dot product value. The final dot product value is used to implement the neural network. The reduced bit width computations allow higher precision mathematical operations to be performed on lower-precision processors with improved accuracy.

Abstract: Technology related to training a neural network accelerator using mixed precision data formats is disclosed. In one example of the disclosed technology, a neural network accelerator is configured to accelerate a given layer of a multi-layer neural network. An input tensor for the given layer can be converted from a normal-precision floating-point format to a quantized-precision floating-point format. A tensor operation can be performed using the converted input tensor. A result of the tensor operation can be converted from the block floating-point format to the normal-precision floating-point format. The converted result can be used to generate an output tensor of the layer of the neural network, where the output tensor is in normal-precision floating-point format.

Abstract: Apparatus and methods are disclosed for initiating instruction block execution using a register access instruction (e.g., a register Read instruction). In some examples of the disclosed technology, a block-based computing system can include a plurality of processor cores configured to execute at least one instruction block. The at least one instruction block encodes a data-flow instruction set architecture (ISA). The ISA includes a first plurality of instructions and a second plurality of instructions. One or more of the first plurality of instructions specify at least a first target instruction without specifying a data source operand. One or more of the second plurality of instructions specify at least a second target instruction and a data source operand that specifies a register.

Abstract: Computer systems and methods for generating and interacting with a micro-service framework are provided. A micro-service corresponds to one or more deep link/API calls that carry out some particular function. A static analysis of an app is conducted, from one or more starting sources of the app to identify one or more valid and feasible execution paths, as well as corresponding input parameters within the app. Each valid execution path with corresponding input parameters represent a “deep link” or “API” for that app. The information regarding the deep link is collected and stored as a micro-service in a micro-service catalog. A micro-service framework is implemented that receives a micro-service request (i.e., a request that the micro-service be carried out on behalf of a computer user) from a UX client and executes that micro-service request via execution of the deep link.

Abstract: A hardware acceleration component is provided that includes a plurality of hardware clusters, each hardware cluster comprising a plurality of soft processor cores and a functional circuit. The plurality of soft processor cores share the functional circuit.

Abstract: A system is provided that includes a first processor and a second processor. The first processor includes first hardware logic circuitry that performs a Lempel-Ziv-Markov chain algorithm (LZMA) forward pass compression process on a portion of source data to provide first output data. The second processor that performs an LZMA backward pass compression process on the first output data to provide second output data.

Abstract: A method is provided for processing on an acceleration component a deep neural network. The method includes configuring the acceleration component to perform forward propagation and backpropagation stages of the deep neural network. The acceleration component includes an acceleration component die and a memory stack disposed in an integrated circuit package. The memory stack has a memory bandwidth greater than about 50 GB/sec and a power efficiency of greater than about 20 MB/sec/mW.

Abstract: A server system is provided that includes a plurality of servers, each server including at least one hardware acceleration device and at least one processor communicatively coupled to the hardware acceleration device by an internal data bus and executing a host server instance, the host server instances of the plurality of servers collectively providing a software plane, and the hardware acceleration devices of the plurality of servers collectively providing a hardware acceleration plane that implements a plurality of hardware accelerated services, wherein each hardware acceleration device maintains in memory a data structure that contains load data indicating a load of each of a plurality of target hardware acceleration devices, and wherein a requesting hardware acceleration device routes the request to a target hardware acceleration device that is indicated by the load data in the data structure to have a lower load than other of the target hardware acceleration devices.

Abstract: Aspects extend to methods, systems, and computer program products for changing between different roles at acceleration components. Changing roles at an acceleration component can be facilitated without loading an image file to configure or partially reconfigure the acceleration component. At configuration time, an acceleration component can be configured with a framework and a plurality of selectable roles. The framework also provides a mechanism for loading different selectable roles for execution at the acceleration component (e.g., the framework can include a superset of instructions for providing any of a plurality of different roles). The framework can receive requests for specified roles from other components and switch to a subset of instructions for the specified roles. Switching between subsets of instructions at an acceleration component is a lower overhead operation relative to reconfiguring or partially reconfiguring an acceleration component by loading an image file.

Abstract: Methods and apparatus are disclosed for providing emulation of quantized precision operations. In some examples, the quantized precision operations are performed for neural network models. Parameters of the quantized precision operations can be selected to emulate operation of hardware accelerators adapted to perform quantized format operations. In some examples, the quantized precision operations are performed in a block floating-point format where one or more values of a tensor, matrix, or vectors share a common exponent. Techniques for selecting the exponent, reshaping the input tensors, and training neural networks for use with quantized precision models are also disclosed. In some examples, a neural network model is further retrained based on the quantized model. For example, a normal precision model or a quantized precision model can be retrained by evaluating loss induced by performing operations in the quantized format.