Abstract: Systems and methods are provided for implementing memory checkpointing using a distributed non-volatile memory system. For example, an application runs on a plurality of server nodes in a server cluster. Each server node includes system memory having volatile system memory and non-volatile system memory. A current application state of the application is maintained in the system memory of one or more server nodes. A checkpoint operation is performed to generate a distributed checkpoint image of the current application state of the application. The distributed checkpoint image is stored in the non-volatile system memory of the plurality of server nodes. Fault-tolerant parity data is generated for the distributed checkpoint image, and the fault-tolerant parity data for the distributed checkpoint image is stored in the non-volatile system memory of one or more of the plurality of server nodes.

Abstract: Techniques are provided to decouple data pipeline tasks from an execution flow of a high-performance computing task (e.g., distributed deep model training) in a distributed computing system. For example, a method includes receiving a client request to provision resources for executing a computing job, provisioning accelerator resources of one or more accelerator server nodes in the distributed computing system to perform tasks associated with an execution flow of the computing job, and provisioning a logical nodes within the distributed computing system to compose a data flow pipeline which is configured to perform data flow operations associated with the computing job for providing data to the provisioned accelerator resources to perform the tasks associated with the execution flow of the computing job. The data flow operations include, e.g., data storage input/output operations, data pre-processing operations, and data staging operations, which are decoupled from the execution flow of the computing job.

Abstract: Systems and methods are provided for dynamically reallocating resources during run-time execution of workloads in a distributed accelerator-as-a-service computing system to increase workload execution performance and resource utilization. A workload is executed in the distributed accelerator-as-a-service computing system using an initial set of resources allocated to the executing workload, wherein the allocated resources include accelerator resources (e.g., physical and/or virtual accelerator resources). The performance of the executing workload is monitored to detect a bottleneck condition which causes a decrease in the performance of the executing workload. In response to detecting the bottleneck condition, another set of resources is reallocated to the executing workload, which is determined to reduce or eliminate the bottleneck condition.

Abstract: One example method includes receiving measurement information concerning the operation of a computing system component, comparing the measurement information to a standard, based on the comparison, determining whether or not the measurement information indicates the presence of an anomaly in the operation of the computing system component, when an anomaly is indicated, generating a prediction, based upon the measurement information, as to when the computing system component is expected to fail, and implementing, prior to failure of the computing system component, a proactive data protection action which protects data associated with the computing system component.

Abstract: Systems and methods are provided for implementing memory checkpointing using a distributed non-volatile memory system. For example, an application runs on a plurality of server nodes in a server cluster. Each server node includes system memory having volatile system memory and non-volatile system memory. A current application state of the application is maintained in the system memory of one or more server nodes. A checkpoint operation is performed to generate a distributed checkpoint image of the current application state of the application. The distributed checkpoint image is stored in the non-volatile system memory of the plurality of server nodes. Fault-tolerant parity data is generated for the distributed checkpoint image, and the fault-tolerant parity data for the distributed checkpoint image is stored in the non-volatile system memory of one or more of the plurality of server nodes.

Abstract: A system, computer program product, and computer-executable method of managing a converged infrastructure, the system, computer program product, and computer-executable method including receiving system configuration information and updating the converged infrastructure with the configuration information.

Abstract: A Computer-executable method, system, and computer program product for managing tiers of data storage in a data storage environment using a data storage system, the Computer-executable method, system, and computer program product comprising a non-transitory computer readable medium encoded with computer-executable program code for using read signatures in replication, the code configured to enable the execution of initializing a data stack, updating a map of the data stack; and distributing the map of the data stack to a compute node.

Abstract: Systems, methods, and articles of manufacture comprising processor-readable storage media are provided for implementing an accelerated data storage synchronization to protect data in a storage system. For example, a first server node accesses a data block from a protected storage system in response to a request from an application executing on the first server node. A modified data block is written to a write-back cache of the first server node. A synchronization operation is performed in response to a request by the application, which includes sending a copy of the modified data block in the write-back cache to a second server node to temporarily store the copy of the modified data block in a storage media of the second server node, and informing the application that the synchronization operation is deemed complete after the copy of the modified data block is stored in the storage media of the second server node.

Abstract: A method implemented by a server enables sharing of GPU resources by multiple clients. The server receives a request from a first client for GPU services. The request includes a first block of GPU code of an application executing on the first client. A first task corresponding to the first block of GPU code is enqueued in a task queue. The task queue includes a second task that corresponds to a second block of GPU code of an application executing on a second client. The server schedules a time for executing the first task using a GPU device that is assigned to the first client, and dispatches the first task to a GPU worker process to execute the first task at the scheduled time using the GPU device. The GPU device is shared, either temporally or spatially, by the first and second clients for executing the first and second tasks.

Abstract: An apparatus comprises a processing platform implementing a plurality of resources of converged infrastructure of an information processing system. A controller associated with the plurality of resources comprises a resource discovery module and a hierarchical mapper. The resource discovery module is configured to identify and categorize the resources of the converged infrastructure. The hierarchical mapper is configured to map at least first and second applications of the processing platform to respective sets of services utilized by those applications and to map each of the services to respective sets of resources utilized by those services so as to provide a hierarchical mapping of applications, services and resources for the converged infrastructure. One or more resulting hierarchical mappings generated by the hierarchical mapper are stored in a database and utilized by the controller to implement orchestration functionality for one or more additional applications of the processing platform.

Abstract: Source and replica data in a storage area network is tracked during management of data encryption keys. Association of source and replica data allows for all copies of customer information in an enterprise to be managed as a single entity for deletion or tracked for management purposes by using referenced data encryption keys upon creation of replicas. Any replica from a source storage object can be created using the source storage object data encryption key or an associated key and tracked by these keys as a subset of the number of replicas created. Management of the data encryption keys can control the lifetime of data on a storage array and in the storage area network without managing every replicated instance for the lifetime of the data.

Abstract: An apparatus comprises a processing platform implementing a plurality of resources of converged infrastructure of an information processing system. A multi-dimensional scheduler is associated with the plurality of resources. First and second ones of the plurality of resources of the converged infrastructure are logically partitioned by the multi-dimensional scheduler into multiple virtual sets of resources for performance of processing jobs associated with respective applications such that the virtual sets of resources comprise respective portions of each of the first and second ones of the plurality of resources. An additional one of the plurality of resources of the converged infrastructure is time-shared between at least a subset of the multiple virtual sets of resources under the control of the multi-dimensional scheduler in performing the corresponding processing jobs associated with the respective applications.

Abstract: A data processing system stores encrypted data. Object identifiers are assigned to storage objects, and data encryption keys are assigned to the storage objects. When performing an operation upon a storage object, data encryption key failure may occur due to a corrupt or incorrect key. In this case, a copy of the data encryption key is fetched from a key server. It is possible for the association of the object identifiers with the data encryption keys to become lost or confused, so that the key server may fail to provide the correct key for a specified object identifier. Therefore, an absolute key identifier that is unique across the key server namespace also is stored in association with the object identifier in the storage system and in the key store of the key server, and the absolute key identifier is used as a failsafe for recovery of encrypted data.

Abstract: A method, computer program product, and computing system for includes processing a log file to determine the location of one or more data files within a data environment. The log file indicates the occurrence of a data operation event within the data environment. A data operation is performed on at least a portion of the one or more data files located via the log file.

Abstract: A system, method, and computer program product for establishing a memory-mapped file, enabling the memory-mapped file data to be paged to a non-volatile storage medium, marking a portion of the memory-mapped file as protected, wherein a write to the memory mapped file throws a segmentation fault; receiving a write at the memory mapped file, throwing a segmentation fault; and handling the segmentation fault in a segmentation handler, where the handling comprises reading the information in the memory mapped file facility into the undo log, and writing the write IO to the storage medium.

Abstract: A computer implemented method, system, and program product for data consistency comprising establishing a memory-mapped file, enabling the memory-mapped file data to be paged to the non-volatile storage medium, receiving, at a consistency layer, a write IO; wherein the write IO corresponds to a space on the storage medium, issuing a read IO to the storage medium, wherein the read IO corresponds to the space on the storage medium to be overwritten by the write IO, writing the data of the read IO to a undo log and writing the write IO to the storage medium.

Abstract: A method is used in managing storage operations in a data storage environment. An I/O request is received a server having a server cache where the server cache contains cached data previously supplied from a storage system in communication with the server. At the storage system, determine if the I/O request can be at least partially satisfied from the cached data.

Abstract: By placing an encryption function below a RAID function and requiring independence of encryption functionality along the same boundaries of data member independence of the RAID function, failures of the encryption and key management for individual encryption functions can be recovered with the same data rebuild mechanism as the RAID function. For example, in a RAID set of disk drives, each data partition and each parity partition has a respective data encryption key for storing encrypted data or encrypted parity in the partition, and a LUN or logical volume is mapped to a stripe of data partitions and an associated parity partition across the RAID set so that the data rebuild mechanism of the RAID function may recover from a loss of a single data encryption key without compromising security of the LUN or logical volume.

Abstract: A method of securely operating a computerized system includes forming a connection to a user-removable physical security device (PSD) which is uniquely paired with the computerized system and which stories cryptographically secured data required for performing a protected function on the computerized system. The PSD may be realized as a USB or similar peripheral device containing security-related data and potentially security processing capability as well. The protected function could be decrypting of encrypted data encryption keys used to encrypt/decrypt user data for example. A user who has an established association with the PSD (e.g. by some preceding registration process) is authenticated, resulting in activation of the PSD on the computerized system. Upon such activation of the PSD, the computerized system engages in a security operation using the cryptographically secured data from the PSD to enable the protected function to be performed under control of the user on the computerized system.

Abstract: A method, system and program product for dynamic management of system resources in a data storage system comprising producing a hint at a hint producer; wherein the hint contains information about the system, consuming the hint at a hint consumer, and determining, at the hint consumer, whether or not to act on the information in the hint.