Abstract: According to a computing method a compiler determines a recompute node included in a dataflow application and a checkpoint tensor produced by the recompute node. The compiler determines a recompute cost to recompute the checkpoint tensor, and a memory cost to checkpoint the checkpoint tensor in a memory. Based on the recompute cost and/or the memory cost, the compiler determines a solution cost and compares the solution cost to a solution threshold. Based on comparing the solution cost to the solution threshold, the compiler determines a checkpoint solution to execute the dataflow application. The checkpoint solution can comprise recomputing or checkpointing the checkpoint tensor. In some implementations, the compiler can determine a recompute ratio of the recompute cost to the memory cost and can compare the recompute ratio to the solution threshold. A computer program product and a computing system can implement aspects of the method.

Abstract: Embodiments described herein provide systems and techniques for training large language models. In one aspect, a process for performing in-context training of a language model is disclosed. This process may begin by receiving a language model that includes a context window of a predetermined size, as well as receiving a set of in-context prompt/completion pairs prepared for a target task. The process then constructs a first token sequence based on the set of in-context prompt/completion pairs. Next, the process fits the first token sequence into the context window. The process subsequently performs a first in-context training pass using the first token sequence to train the language model to generate a next token in accordance with the target task.

Abstract: Embodiments described herein provide systems and techniques for training large language models. In one aspect, a process for performing in-context few-shot training for a transformer-based language model is disclosed. This process may begin by receiving the transformer-based language model having a context window of a predetermined size, as well as a training dataset comprising a set of prompt/completion examples. The process then constructs a training sequence based on the training dataset. Next, the process performs a single forward pass using the training sequence as input. The process subsequently performs a set of backward passes from a subset of examples in the training sequence, wherein each backward pass is conditioned on a selected subset of prompt/completion examples in the training sequence.

Abstract: A system includes one or more processors and a statically reconfigurable dataflow processor (SRDAP) coupled to the processors which are programmed to receive a first request to generate an instantiation of a computation graph to generate a probability distribution for N classes and retrieve a compiled graph of the computation graph. The computation graph includes a bias node and a probability distribution node for M classes. The bias node provides a biased tensor of size M to the probability distribution node by adding a bias tensor. The processors generate a bias tensor having N entries equal to zero and M?N entries having negative values and then load the compiled graph with the first bias tensor into a first set coarse-grained reconfigurable units of the SRDAP. Execution of the computation graph is initiated on the SRDAP to generate the probability distribution and a first inference is provided.

Abstract: According to a computing method a compiler determines a recompute node included in a dataflow application and a checkpoint tensor produced by the recompute node. The compiler determines a recompute cost to recompute the checkpoint tensor, and a memory cost to checkpoint the checkpoint tensor in a memory. Based on the recompute cost and/or the memory cost, the compiler determines a solution cost and compares the solution cost to a solution threshold. Based on comparing the solution cost to the solution threshold, the compiler determines a checkpoint solution to execute the dataflow application. The checkpoint solution can comprise recomputing or checkpointing the checkpoint tensor. In some implementations, the compiler can determine a recompute ratio of the recompute cost to the memory cost and can compare the recompute ratio to the solution threshold. A computer program product and a computing system can implement aspects of the method.