Abstract: A processor-implemented method includes receiving a graph representing an artificial neural network (ANN). The graph includes multiple nodes connected by edges and each node represents an operation. Retention intervals are determined for the multiple node outputs based on rematerialization constraints and paging constraints. The retention intervals correspond to a time interval for retaining each node output in at least one local memory. A sequence of tasks for executing the multiple nodes of the graph representing the ANN is determined based on the retention intervals.

Abstract: A processor-implemented method for rematerialization for an artificial neural network (ANN) includes receiving a graph representing the ANN. The graph includes multiple nodes connected by edges and each node represents an operation. Retention intervals for the nodes are determined based on a precedence constraint for the nodes. The retention intervals correspond to a time interval for retaining each node output in a local memory. One of the nodes to recompute is determined based on the retention intervals.

Abstract: A processor-implemented method includes sampling, according to a priority sampling policy, a set of node priorities from a computation graph. Each node priority of the set of node priorities may be associated with a respective node on the computation graph. Additionally, each node may represent an operation of a task performed by an artificial neural network. The method also includes converting, via a list scheduling function, the node priorities to a schedule that associates each node of the computation graph with a processor of a group of processors of a device associated with the artificial neural network, the schedule associated with a makespan. The method further includes performing the task in accordance with the schedule.