Abstract: A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Abstract: Disclosed are systems and techniques for compressing a dense matrix into an expressive sparse matrix representation with limited metadata. The techniques include generating a sparse matrix with corresponding metadata based on a dense matrix. Generating the sparse matrix with corresponding metadata includes identifying a first number (M) of elements to compress, a second number (N) of elements to retain, a third number (P) of positions, and a format; determining a metadata value for each of N elements of the dense matrix based on the identified P and the identified format, wherein the dense matrix includes at least M elements; and generating the sparse matrix containing the N elements of the dense matrix. The techniques include storing the sparse matrix and the corresponding metadata, wherein the corresponding metadata comprises the metadata value for each of the N elements of the dense matrix.

Abstract: Disclosed are systems and techniques for performing matrix multiply operations on an expressive sparse matrix representation with limited metadata. The techniques include receiving a sparse matrix, metadata corresponding to the sparse matrix, and a matrix operand. The sparse matrix contains a first number (N) of elements to retain from a dense matrix which comprises at least a second number (M) of elements. The metadata corresponding to the sparse matrix is based on a third number (P) of positions and a format determined during compression of the dense matrix. The techniques include selecting, by one or more selection circuits, a subset of elements of the matrix operand based on the metadata corresponding to the sparse matrix and performing one or more matrix multiply operations on the sparse matrix and the subset of elements of the matrix operand.

Abstract: Disclosed are systems and techniques for decompressing an expressive sparse matrix representation with limited metadata. The techniques include receiving a sparse matrix and metadata corresponding to the sparse matrix. The sparse matrix is a compressed representation of a dense matrix. The sparse matrix contains a first number (N) of elements to retain from the dense matrix which comprises at least a second number (M) of elements. The metadata corresponding to the sparse matrix is based on a third number (P) of positions and a format determined during compression of the dense matrix. The techniques include generating an uncompressed matrix based on the sparse matrix and the metadata corresponding to the sparse matrix.

Abstract: Disclosed are systems and techniques for compressing a dense matrix into a fully-expressive sparse matrix representation with limited metadata. The techniques include generating a sparse matrix with corresponding metadata based on a dense matrix. Generating the sparse matrix with corresponding metadata includes identifying a first number (M) of elements to compress, a second number (N) of elements to retain, and a third number (B) indicating the number of bits each metadata value uses; determining a metadata value for each of N elements of the dense matrix; packing a first metadata value having more than B bits into a second metadata value having B bits; and generating the sparse matrix containing the N elements of the dense matrix. The techniques include storing the sparse matrix and the corresponding metadata, wherein the corresponding metadata comprises the second metadata value.

Abstract: Distribution of data in a neural network data set is used to determine an optimal compressor configuration for compressing the neural network data set and/or the underlying data type of the neural network data set. By using a generalizable optimization of examining the data prior to compressor invocation, the example non-limiting technology herein makes it possible to tune a compressor to better target the incoming data. For sparse data compression, this step may involve examining the distribution of data (e.g., in one example, zeros in the data). For other algorithms, it may involve other types of inspection. This changes the fundamental behavior of the compressor itself. By inspecting the distribution of data (e.g., zeros in the data), it also possible to very accurately predict the data width of the underlying data.

Abstract: Apparatuses, systems, and techniques to losslessly compress neural networks via semi-structured sparsity. In at least one embodiment, a weighted average of candidate masks for semi-structured sparsity is learned for each parameter block of a neural network, and a composite mask is determined by selecting candidate masks based on the learned weighted averages. In at least one embodiment, computational resources required for inference are reduced, thereby contributing to more sustainable and environmentally friendly AI applications.

Abstract: Apparatuses, systems, and techniques to transform data sets, such as matrices representing layers of neural networks, to increase sparsity and/or other characteristics of said data sets to improve performance in computations, such as neural network computations. In at least one embodiment, one or more subsets of data in one or more sets of data are rearranged as part of a process to increase sparsity in said one or more sets of data to satisfy one or more one or more structural sparsity constraints.

Abstract: A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Abstract: Machine learning is a process that learns a model from a given dataset, where the model can then be used to make a prediction about new data. In order to reduce the size, computation, and latency of a machine learning model, a compression technique can be employed which includes model sparsification and quantization. To limit the extent to which the quality of the model is impacted when uniformly applying sparsification and quantization to all values of the model, the present disclosure provides for a hybrid sparsification and quantization of the model.

Abstract: A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Abstract: Apparatuses, systems, and techniques to determine a configuration based at least in part on data stored by at least one data structure of a workload at runtime, and transform the workload into a sparse workload based at least in part on the configuration. In at least one embodiment, one or more sparse workloads (e.g., one or more sparse neural networks) are generated based at least in part on, for example, one or more workloads (e.g., one or more neural networks).

Abstract: A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Abstract: A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Abstract: A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Abstract: A packed error correction code (ECC) technique opportunistically embeds ECC check-bits with compressed data. When compressed, the data is encoded in fewer bits and is therefore fragmented when stored or transmitted compared with the uncompressed data. The ECC check-bits may be packed with compressed data at “source” points. The check-bits are transmitted along with the compressed data and, at any “intermediate” point between the source and a “destination” the check-bits may be used to detect and correct errors in the compressed data. In contrast with conventional systems, packed ECC enables end-to-end coverage for sufficiently-compressed data within the processor and also externally. While storage circuitry typically is protected by structure-specific ECC, protection is also beneficial for data as it is transmitted between processing and/or storage units.

Abstract: A packed error correction code (ECC) technique opportunistically embeds ECC check-bits with compressed data. When compressed, the data is encoded in fewer bits and is therefore fragmented when stored or transmitted compared with the uncompressed data. The ECC check-bits may be packed with compressed data at “source” points. The check-bits are transmitted along with the compressed data and, at any “intermediate” point between the source and a “destination” the check-bits may be used to detect and correct errors in the compressed data. In contrast with conventional systems, packed ECC enables end-to-end coverage for sufficiently-compressed data within the processor and also externally. While storage circuitry typically is protected by structure-specific ECC, protection is also beneficial for data as it is transmitted between processing and/or storage units.

Abstract: Apparatuses, systems, and techniques to transform data sets, such as matrices representing layers of neural networks, to increase sparsity and/or other characteristics of said data sets to improve performance in computations, such as neural network computations. In at least one embodiment, one or more subsets of data in one or more sets of data are rearranged as part of a process to increase sparsity in said one or more sets of data to satisfy one or more one or more structural sparsity constraints.

Abstract: A method, computer readable medium, and processor are described herein for inline data inspection by using a decoder to decode a load instruction, including a signal to cause a circuit in a processor to indicate whether data loaded by a load instruction exceeds a threshold value. Moreover, an indication of whether data loaded by a load instruction exceeds a threshold value may be stored.

Abstract: A method, computer readable medium, and system are disclosed for inline data inspection. The method includes the steps of receiving, by a load/store unit, a load instruction and obtaining, by an inspection circuit that is coupled to the load/store unit, data specified by the load instruction. Additional steps include determining that the data equals zero and transmitting the data and a predicate signal to the load/store unit, wherein the predicate signal indicates that the data equals zero. Alternative additional steps include computing a predicate value based on a comparison between the data and a threshold value and transmitting the data and the predicate value to the load/store unit, wherein the predicate value is asserted when the data is less than the threshold value and is negated when the data is not less than the threshold value.