Abstract: A device for deep learning acceleration with mixed precision may include a first data port configured to receive a map data segment and a second data port configured to receive a kernel data segment. The device may include a precision mode port configured to receive an indication of an input precision mode that indicates a word length for the map data segment and for the kernel data segment. The device may include a multiplier component configured to generate a multiplier component output based on the input precision mode and based on multiplying the map data segment and the kernel data segment. The device may include an adder component configured to generate an adder component output based on the input precision mode and based on the multiplier component output. The device may include an output port configured to output the adder component output.

Abstract: A device for deep learning acceleration with mixed precision may include multiple matrix-matrix (MM) components that each include multiple map memory components configured to store map data, multiple kernel memory components configured to store kernel data, and multiple matrix-vector (MV) components. The MV components may each include multiple vector-vector (VV) components that are each configured to generate a VV output based on an input precision mode, an output precision mode, and an accumulation of products that is based on the map data and the kernel data. Each VV component included in a particular MV component may be coupled with each map memory component and may be coupled with a single kernel memory component. The device may include a data distribution component coupled with the multiple MM components and configured to load the map data into the multiple map memory components.

Abstract: A device for deep learning acceleration with mixed precision may include a token generator configured to generate a token value and may include multiple multiplexers. Each multiplexer may include a load port configured to receive map data, a max pool port configured to receive max pool data, and matrix-matrix (MM) data input ports each configured to receive MM data based on MM output generated by an MM component. Each multiplexer may include an output port configured to provide output data to a single MM component. Each multiplexer may provide corresponding output data to a different MM component. Each multiplexer may be configured to select, based on the token value, an input from one of the load port, the max pool port, or a single MM data input port, of the MM data input ports, as the output data to be provided to the output port.

Abstract: A device for deep learning acceleration with mixed precision may include a precision mode port configured to receive an indication of an output precision mode, a data input port configured to receive an input value, and a truncation component configured to truncate the input value into a keep segment value and a truncate segment value. The device may be configured to add the keep segment value and a carry bit to generate a rounded keep segment value, and to generate a rounded output based on the rounded keep segment value and the output precision mode. The rounded output generation component may be configured to generate the rounded output to include a sign bit of the keep segment value and either a first quantity or a second quantity of lower bits of the keep segment value based on the output precision mode being either a first value or a second value.

Abstract: A device for deep learning acceleration with mixed precision may include vector-vector (VV) components that are each configured to generate a VV output based on an input precision mode, an output precision mode, and at least one accumulation of products. Each accumulation of products may be calculated by adding products based on the input precision mode. Each product may be calculated by multiplying a map word and a kernel word based on the input precision mode. The input precision mode may indicate an input word length for the map word and for the kernel word, and the output precision mode may indicate an output word length for the VV output. The device may include one or more components configured to concatenate VV outputs, corresponding to the VV components, to generate a concatenated VV output. The device may include an output port configured to output the concatenated VV output.

Abstract: A device for deep learning acceleration with mixed precision may include a first precision mode port to receive an indication of an input precision mode and a second precision mode port to receive an indication of an output precision mode. The device may include a first data port to receive map data and a second data port to receive kernel data. The device may include multiply-accumulate (MAC) components that are each configured to generate a MAC output based on the input precision mode, the map data, and the kernel data. The device may include an adder component to generate an adder component output based on the input precision mode and one or more MAC outputs. The device may include a rounding component to round the adder component output, based on the output precision mode, to generate a rounded output, and an output port to output the rounded output.

Abstract: A device for deep learning acceleration with mixed precision may include matrix-vector (MV) components that each include vector-vector (VV) components that are each configured to generate a respective VV output based on an input precision mode, an output precision mode, and an accumulation of products. The accumulation of products may be calculated by adding products based on the input precision mode. Each product may be calculated by multiplying, based on the input precision mode, a map data segment and a kernel data segment. Each MV component may include one or more components configured to concatenate VV outputs to generate a concatenated VV output. The device may include activation function components that are each configured to receive a corresponding concatenated VV output, generate an activation function output based on the corresponding concatenated VV output and the output precision mode, and output the activation function output.