Abstract: A system and method for executing computational workloads on a computational processing unit is provided. The method includes deploying a virtual machine (VM) by abstracting program behavior of the computational processing unit into instructions associated with groups of execution threads and providing a data layout system that defines distribution of data elements across threads within the groups of execution threads. The method further includes executing computational workloads by processing data elements distributed according to the data layout system.

Abstract: A system and method of executing a deep-learning workload are provided. The method includes selecting at least one parallelization strategy for a first processing stage of the workload, selecting at least one different parallelization strategy for a second processing stage of the workload, and switching between the different strategies used for the first and second processing stages by reconfiguring a partitioning of the deep-learning workload across computing resources.

Abstract: A system and method are described which provide a unique obfuscation mechanism for conducting performance optimization of deep neural network (DNN) computational graphs. The method obfuscates performance optimization in three steps. First, an obfuscation step where the original computation graph is obfuscated such that an adversary cannot feasibly identify the original model, thus providing confidentiality. Second, the optimization step is carried out flexibly and independently by the optimizer party on the obfuscated computational graph, providing performance speedups. Finally, the de-obfuscation step where the original model is retrieved by the model owner in its optimized form.

Abstract: A system method for performing distributed training of models. The method includes co-optimizing memory optimizations with parallelism to increase model training throughput under a memory constraint by orchestrating a plurality of optimizations to utilize system resources in consideration of computation, communication and memory footprint. The system can include an overlap-centric schedule template that determines granularity and order of how techniques utilized by the plurality of optimizations are applied to a model. The overlap-centric schedule template mitigates tuning complexity by applying heuristics to orchestrate optimizations in an overlapped manner.

Abstract: A system and method are provided for facilitating graphics processing unit (GPU) sharing. The method includes providing a transparent virtualization layer positioned between one or more applications and one or more GPUs to be used by the one or more applications; intercepting device application programming interface (API) calls at the virtualization layer; and executing a task-agnostic scheduling algorithm to prioritize execution of tasks associated with the API calls, by the one or more GPUs, by prioritizing high-priority tasks while opportunistically scheduling kernels on the one or more GPUs from best effort tasks during GPU idle cycles.

Abstract: A computer system is provided for compiling computer programs using machine learning compilers. The method includes obtaining a first computer program written in a script with a defined programming language dialect for a particular machine learning compiler, defining a higher level intermediate representation (IR) that represents the first computer program written with the defined programming language dialect; generating a second computer program represented in the higher level IR; deriving a plurality of new optimization passes based on the higher level IR; converting the second computer program represented in the higher level IR into a third computer program represented in one or more lower-level IRs that exist in the machine learning compiler; applying a plurality of existing optimization passes to generate an optimized fourth computer program represented in the existing lower-level IR; and converting the optimized fourth computer program represented in the existing lower-level IR-to-machine instructions.