Abstract: The subject matter described herein provides systems and techniques to address the challenges of growing hardware and workload heterogeneity using a Warehouse-Scale Computer (WSC) design that improves the efficiency and utilization of WSCs. The WSC design may include an abstraction layer and an efficiency layer in the software stack of the WSC. The abstraction layer and the efficiency layer may be designed to improve job scheduling, simplify resource management, and drive hardware-software co-optimization using machine learning techniques and automation in order to customize the WSC for applications at scale. The abstraction layer may embrace platform/hardware and workload diversity through greater coordination between hardware and higher layers of the WSC software stack in the WSC design. The efficiency layer may employ machine learning techniques at scale to realize hardware/software co-optimizations as a part of the autonomous WSC design.

Abstract: Aspects of the disclosure are directed to jointly searching machine learning model architectures and hardware architectures in a combined space of models, hardware, and mapping strategies. A search strategy is utilized where all models, hardware, and mappings are evaluated together at once via weight sharing and a supernetwork. A multi-objective reward function is utilized with objectives for quality, performance, power, and area.

Abstract: The present disclosure includes systems, methods, and computer-readable mediums for discovering capabilities of a hardware (HW) accelerator card. A processor may communicate a request for a listing of acceleration services to a HW accelerator card connected to the processor via the communication interface. The HW accelerator card may retrieve the listing from memory and provide a response to the processor that includes a listing of the HW acceleration services provided by the HW accelerator card.

Abstract: According to various implementations, generally disclosed herein is a hybrid and hierarchical neural architecture search (NAS) approach. The approach includes performing a search space partitioning scheme to divide the search space into sub-search spaces. The approach further includes performing a first type of NAS, such as a Multi-trial NAS, to cover a search across the sub-search spaces. The approach also includes performing a second type of NAS, such as a One-Shot NAS, to cover each sub-search space. The approach further includes automatically stopping the second type of NAS based on one or more early stopping criteria.

Abstract: Methods, systems, and computer storage media storing instructions for managing processing system efficiency. One of the methods includes obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

Abstract: The present disclosure includes systems, methods, and computer-readable mediums for discovering capabilities of local and remote hardware (HW) accelerator cards. A local hardware (HW) accelerator card may provide, via a communication interface, a listing of acceleration services from the local HW accelerator card. The listing of acceleration services may include a first set of acceleration services provided by one or more accelerators of the local HW accelerator card and a second set of acceleration services provided by one or more accelerators of a remote HW accelerator card. A workload instruction defining a workload for processing by at least one of the acceleration services of the second set of acceleration services may be received from a processor of a computing device. The workload instruction may be forwarded to the remote HW accelerator card.

Abstract: Aspects of the disclosure provide for hardware-aware progressive training of machine learning models. A training system trains a model in accordance with a training process and different values specified in a training schedule for both hardware-level and model-level performance settings. Hardware-level performance settings can cause hardware features of computing resources used to train the model to be enabled, disabled, or modified at various points during training. Model-level performance settings can take on a variety of values to adjust characteristics of the machine learning model being trained or of the training process, during different stages of training. The training system can identify and apply complementary values of hardware- and model-level performance settings to generate training schedules that improve model training speed at earlier stages of training, while improving model quality at later stages of training.

Abstract: The present disclosure includes systems, methods, and computer-readable mediums for discovering capabilities of a hardware (HW) accelerator card. A processor may communicate a request for a listing of acceleration services to a HW accelerator card connected to the processor via the communication interface. The HW accelerator card may retrieve the listing from memory and provide a response to the processor that includes a listing of the HW acceleration services provided by the HW accelerator card.

Abstract: The subject matter described herein provides systems and techniques to address the challenges of growing hardware and workload heterogeneity using a Warehouse-Scale Computer (WSC) design that improves the efficiency and utilization of WSCs. The WSC design may include an abstraction layer and an efficiency layer in the software stack of the WSC. The abstraction layer and the efficiency layer may be designed to improve job scheduling, simplify resource management, and drive hardware-software co-optimization using machine learning techniques and automation in order to customize the WSC for applications at scale. The abstraction layer may embrace platform/hardware and workload diversity through greater coordination between hardware and higher layers of the WSC software stack in the WSC design. The efficiency layer may employ machine learning techniques at scale to realize hardware/software co-optimizations as a part of the autonomous WSC design.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for disaggregating latent causes for computer system optimization. In one aspect, a method includes accessing a data stream for data values resulting from operations performed by a computer system; providing the data values as input to a data disaggregation machine learning model that generates descriptors of latent causes of the data values; providing the data values and the descriptors of the latent causes of the data values as inputs to a control system model that generates embedded representations of commands to modify the operations performed by the computer system; determining commands to modify the operations performed by the computer system based on the embedded representations of commands to modify the operations performed by the computer system; and providing the commands to the computer system.

Abstract: Methods, systems, and computer storage media storing instructions for managing processing system efficiency. One of the methods includes obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

Abstract: Methods, systems, and computer storage media storing instructions for managing processing system efficiency. One of the methods includes obtaining data splitting a plurality of general-purpose processing units in a processing system into a high-priority domain and a low-priority domain, wherein the general-purpose processing units in the high-priority domain are assigned to perform one or more tasks comprising one or more high-priority tasks, and the general-purpose processing units in the low-priority domain are assigned to perform one or more low-priority tasks; and during runtime of the processing system, obtaining memory usage measurements that characterize usage of system memory by the high-priority domain and the low-priority domain; and adjusting, based on the memory usage measurements, a configuration of (i) the high-priority domain, (ii) the low-priority domain, or (iii) both to adjust utilization of the system memory by the general-purpose processing units.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for caching data not frequently accessed. One of the methods includes receiving a request for data from a component of a device, determining that the data satisfies an infrequency condition, in response to determining that the data satisfies the infrequency condition: determining a target cache level which defines a cache level within a cache level hierarchy of a particular cache at which to store infrequently accessed data, the target cache level being lower than a highest cache level in the cache level hierarchy, requesting and receiving the data from a memory that is not a cache of the device, and storing the data in a level of the particular cache that is at or below the target cache level in the cache level hierarchy, and providing the data to the component.

Abstract: A method for compressing and compacting memory on a memory device is described. The method includes organizing a number of compressed memory pages referenced in a number of compaction table entries based on a size of the number of compressed memory pages and compressing the number of compaction table entries, in which a compaction table entry comprise a number of fields.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for disaggregating latent causes for computer system optimization. In one aspect, a method includes accessing a data stream for data values resulting from operations performed by a computer system; providing the data values as input to a data disaggregation machine learning model that generates descriptors of latent causes of the data values; providing the data values and the descriptors of the latent causes of the data values as inputs to a control system model that generates embedded representations of commands to modify the operations performed by the computer system; determining commands to modify the operations performed by the computer system based on the embedded representations of commands to modify the operations performed by the computer system; and providing the commands to the computer system.

Abstract: A first memory controller receives an access command from a second memory controller, where the access command is timing non-deterministic with respect to a timing specification of a memory. The first memory controller sends at least one access command signal corresponding to the access command to the memory, wherein the at least one access command signal complies with the timing specification. The first memory controller determines a latency of access of the memory. The first memory controller sends feedback information relating to the latency to the second memory controller.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for disaggregating latent causes for computer system optimization. In one aspect, a method includes accessing a data stream for data values resulting from operations performed by a computer system; providing the data values as input to a data disaggregation machine learning model that generates descriptors of latent causes of the data values; providing the data values and the descriptors of the latent causes of the data values as inputs to a control system model that generates embedded representations of commands to modify the operations performed by the computer system; determining commands to modify the operations performed by the computer system based on the embedded representations of commands to modify the operations performed by the computer system; and providing the commands to the computer system.

Abstract: An example method involves receiving, at a first memory node, data to be written at a memory location in the first memory node. The data is received from a device. At the first memory node, old data is read from the memory location, without sending the old data to the device. The data is written to the memory location. The data and the old data are sent from the first memory node to a second memory node to store parity information in the second memory node without the device determining the parity information. The parity information is based on the data stored in the first memory node.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for pre-fetching data from memory using neural networks. One example system receives a sequence of prior program counter addresses of a computer program and corresponding delta values. The system creates an input representation based on the sequence. The system provides the input representation as input to a recurrent neural network. The system receives from the recurrent neural network an output that defines a probability distribution over future delta values. Each probability in the distribution represents a likelihood that execution of a future instruction of the computer program will cause data to be fetched from a particular future memory address.

Abstract: An example method includes during execution of a software application by a processor, receiving, by a copy processor separate from the processor, a request for an asynchronous data copy operation to copy data within a memory accessible by the copy processor, wherein the request is received from a copy manager accessible by the software application in a user space of an operating system managing execution of the software application; in response to the request, initiating, by the copy processor, the asynchronous data copy operation; continuing execution of the software application by the processor; determining, by the copy processor, that the asynchronous data copy operation has completed; and in response to determining that the asynchronous copy operation has completed, selectively notifying, by the copy processor, the software application that the asynchronous copy operation has completed.