Abstract: Techniques for selectively offloading data that is computed by a first processing unit during training of an artificial neural network onto memory associated with a second processing unit and transferring the data back to the first processing unit when the data is needed for further processing are described herein. For example, the first processing unit may compute activations for operations associated with forward propagation. During the forward propagation, one or more of the activations may be transferred to a second processing unit for storage. Then, during backpropagation for the artificial neural network, the activations may be transferred back to the first processing unit as needed to compute gradients.

Abstract: Techniques are disclosed for re-executing a data processing pipeline following a failure of at least one of its components. The techniques may include a syntax for defining a compute graph associated with the data processing pipeline and receiving such a compute graph in association with a specific data processing pipeline. The technique may include executing the data processing pipeline, determining that a component of the data processing pipeline failed, and determining a portion of the data processing pipeline to execute/re-execute based at least in part on dependencies defined by the data processing pipeline in association with the failed component. Re-executing the one or more components may comprise retrieving an output saved in association with a component upon which the failed component depends.

Abstract: Techniques are disclosed for re-executing a data processing pipeline following a failure of at least one of its components. The techniques may include a syntax for defining a compute graph associated with the data processing pipeline and receiving such a compute graph in association with a specific data processing pipeline. The technique may include executing the data processing pipeline, determining that a component of the data processing pipeline failed, and determining a portion of the data processing pipeline to execute/re-execute based at least in part on dependencies defined by the data processing pipeline in association with the failed component. Re-executing the one or more components may comprise retrieving an output saved in association with a component upon which the failed component depends.