Abstract: Aspects of the invention include a system for accumulating a plurality of input numbers to output a result. The system includes an accumulation subsystem that accepts input numbers that each have a sign bit, an exponent, and a mantissa. The accumulation subsystem includes: a plurality of partial sum registers, each for a particular one of the exponents accumulating a sum of mantissas of numbers having the particular exponent; an adder/subtractor for accumulating the sums; an exponent-indexed decoder to enable a corresponding partial sum register; and an exponent output multiplexor to select the corresponding partial sum register for output. The system further includes a reconstruction subsystem into which are read the partial sums from the partial sum registers, the reconstruction subsystem comprising an adder, a shifter and an output accumulator-register to output said result.

Abstract: One aspect of the invention includes an apparatus comprising a compressor accepting a model variable such as a weight or activation of an ML model such as an LLM, the compressor converting the model variable to a coding pair, the coding pair consisting of a code and additional data. The model variable may be one of a variety of formats including an integer format, a posit format, part of a binary code group, part of a ternary code group, and one of a plurality of floating-pint formats. Another aspect is an interface apparatus for compressing internal floating-point numbers in memory to a data stream of compressed model variables. Another aspect is an interface apparatus for decompressing a data stream of compressed model variables to internal floating-point numbers in memory.

Abstract: A CNN inference engine that convolves an input data set with a weight data set is disclosed together with components that facilitate such computation. The engine includes a plurality of multiply and accumulate processors (MACs), each MAC causing a value in the accumulator to be augmented by a product of a data value received on an input data port, a weight value received on a weight port. The engine also includes a slice buffer having a plurality of output ports, each output port being connected to one of the MAC input data value ports. The engine causes the slice buffer to connect one of the slices to the plurality of slice buffer output ports, and causes a weight received on an inference engine weight port to be input to each MAC weight port. The MACs process the input data values on the output ports in the slice in parallel.

Abstract: A CNN inference engine that convolves an input data set with a weight data set is disclosed together with components that facilitate such computation. The engine includes a plurality of multiply and accumulate processors (MACs), each MAC causing a value in the accumulator to be augmented by a product of a data value received on an input data port, a weight value received on a weight port. The engine also includes a slice buffer having a plurality of output ports, each output port being connected to one of the MAC input data value ports. The engine causes the slice buffer to connect one of the slices to the plurality of slice buffer output ports, and causes a weight received on an inference engine weight port to be input to each MAC weight port. The MACs process the input data values on the output ports in the slice in parallel.

Abstract: The present invention includes a method for operating a data processing system to compute an approximation to a scalar product between first and second vectors in which each vector is characterized by N components. The method includes replacing the first vector by a third vector that is a pyramid integer vector characterized by N components and an integer K equal to the sum of the absolute values of the N components, and computing a scalar product of the third vector with the second vector to provide the approximation to the scalar product between the first and second vectors. Computing the scalar product of the second and third vectors can be carried out by K additions followed by one floating point multiply.

Abstract: The present invention includes a method for operating a data processing system to compute an approximation to a scalar product between first and second vectors in which each vector is characterized by N components. The method includes replacing the first vector by a third vector that is a pyramid integer vector characterized by N components and an integer K equal to the sum of the absolute values of the N components, and computing a scalar product of the third vector with the second vector to provide the approximation to the scalar product between the first and second vectors. Computing the scalar product of the second and third vectors can be carried out by K additions followed by one floating point multiply.