Abstract: Reconfigurable dataflow architecture is an emerging design for deep learning training accelerator. This architecture maps model operators to an accelerator in a spatial way, enabling pipeline parallelization for high throughput. An essential ingredient to exploit this throughput advantage is compiler Performance Optimization (PO) which searches for optimal model mappings. The convention in industry-leading dataflow compilation uses hand-tuned rules to guide PO, requiring immense engineering cost to develop. This paper challenges this convention and asks if data-driven learned performance optimization can reduce the engineering cost while improving training throughput over hand-tuned rules. We present a workflow which guides PO using simple machine learning models trained from throughput observations of randomly generated mappings.

Abstract: A device may receive a floating-point function. The floating-point function may be a function described in a programming language that uses floating-point representation. The device may determine that fixed-point program code, associated with the floating-point function, is to be generated. The device may determine that the floating-point function is to be replaced with a replacement construct before the fixed-point program code is generated. The replacement construct may be described in the programming language and may be capable of conversion from the floating-point representation to a fixed-point representation. The device may determine parameters associated with generating the replacement construct. The parameters may be determined based on an evaluation of the floating-point function. The device may generate the replacement construct based on the parameters. The device may replace the floating-point function with the replacement construct.