Abstract: Techniques described herein relate to a method for using pipeline data for deploying workflows. The method may include determining that a pipeline testing trigger occurred for a workflow; decomposing a pipeline testing manifest of the workflow; generating a testing execution plan using the decomposed workflow; adding instrumentation to the testing execution plan; determining that the instrumented testing execution plan is valid; deploying computing devices within a CI/CD pipeline ecosystem for performing the instrumented testing execution plan; capturing deployment logs; initiating telemetry capture; executing pipeline testing of the workflow based on the instrumented testing execution plan; generating a pipeline deployment information set based on the pipeline testing; and providing the pipeline deployment information set to an orchestrator of a production device ecosystem.

Abstract: Techniques described herein relate to a method for deploying workflows. The method may include receiving, by a global orchestrator of a device ecosystem, a request to execute a workflow; decomposing, by the global orchestrator, the workflow into a plurality of workflow portions; executing, by the global orchestrator, a metaheuristic algorithm to generate a result comprising a plurality of domains of the device ecosystem in which to execute the plurality of workflow portions; and providing, by the global orchestrator, the plurality of workflow portions to respective local orchestrators of the plurality of domains based on the result of executing the metaheuristic algorithm.

Abstract: Techniques described herein relate to a method for performing data services. The method may include receiving, from a monitor agent, a global name and metadata corresponding to data written to a data structure of storage of a data host; performing a services assessment using the global name, the metadata, and a data service policy rule to obtain at least one data service to perform on the data; and performing the at least one data service based on the services assessment.

Abstract: A system for managing storage of data in a information handling systems includes a first information handling system, and a specialized information handling system comprising an enhanced networking interface, wherein the enhanced networking interface is programmed to: obtain data to be processed by the system, perform a data sharding analysis using telemetry data to identify the first information handling system, and transmit the data to the first information handling system based on the identifying.

Abstract: Techniques described herein relate to a method for deploying workflows. The method may include receiving, by a platform controller of a domain, a workflow portion from a service controller of a federated controller, provisioning a set of devices in the domain to the workflow portion based on a first fit, generating, by the platform controller, a workflow fingerprint based on the provisioning of the set of devices and based on the workflow portion, executing the workflow portion in the domain using the set of devices, making a determination that the workflow portion requires additional resources, based on the determination, provisioning additional resources of the domain to the workflow portion to obtain an updated execution resource set, and updating the workflow fingerprint based on the updated execution resource set to obtain an updated workflow fingerprint, and executing the workflow portion using the updated execution resource set.

Abstract: Techniques described herein relate to a method for deploying workflows. The method may include receiving, at a service controller of a federated controller, a request to deploy a workflow in a device ecosystem; decomposing, by the service controller, the workflow into a plurality of workflow portions; performing, by the service controller, a search in a capabilities and constraints data construct to identify a domain in which to perform a workflow portion of the plurality of workflow portions; providing the workflow portion and workflow constraints to a platform controller in the domain; performing, by the platform controller, a subgraph similarity check to determine that a previously executed workflow portion was successfully executed on a subgraph in the domain; provisioning, by the platform controller, a set of devices in the domain based on the subgraph; and executing the workflow portion in the domain.

Abstract: Techniques described herein relate to a method for managing workflows. The method may include obtaining, from a client, by a registration manager associated with accelerator pools, a request to perform a portion of a workflow using accelerators; identifying a minimum quantity and a maximum quantity of accelerators associated with the request; identifying a storage to store data associated with the portion of the workflow; identifying an accelerator pool of the accelerator pools that includes at least the maximum quantity of accelerators; establishing a connection between the client, the storage, and accelerators of the accelerator pool to perform the portion of the workflow; and initiating the performance of the portion of the workflow using the storage, the client, and the accelerators of the accelerator pool.

Abstract: Techniques described herein relate to a method for deploying workflows. The method may include obtaining, by a registration manager associated with accelerator pools, a first request from a client to perform a portion of a first workflow using accelerators; identifying a minimum quantity and a maximum quantity of accelerators associated with the first request; identifying an accelerator pool of the accelerator pools to perform the portion of the first workflow based on the minimum quantity and the maximum quantity of accelerators, where the accelerator pool includes at least the maximum quantity of accelerators; establishing a connection between the client and the accelerators of the accelerator pool to perform the portion of the first workflow; and initiating performance of the portion of the first workflow, wherein the portion of the first workflow is performed using at least the minimum quantity of accelerators.

Abstract: Techniques described herein relate to a method for deploying workflows.

Abstract: Techniques described herein relate to a method for deploying workflows. The method may include receiving, by a platform controller of a domain, a workflow portion and workflow constraints from a service controller of a federated controller, provisioning, by the platform controller, a set of devices in the domain to the workflow portion, executing the workflow portion in the domain using the set of devices, monitoring the executing of the workflow portion to determine an occupancy of the workflow portion, making a determination that the occupancy of the workflow portion exceeds an available capacity of the domain, providing, based on the determination, the occupancy to the service controller, and in response to the occupancy, performing, by the service controller, a remediation action.

Abstract: Techniques described herein relate to a method for deploying workflows. The method may include receiving, by a platform controller of a domain, a workflow portion and workflow constraints from a service controller of a federated controller, provisioning, by the platform controller, a set of devices in the domain, executing the workflow portion in the domain using the set of devices, applying a chaos action set on the set of devices, generating results of the chaos action set, and providing the results of the chaos action set to the service controller.

Abstract: Techniques described herein relate to a method for provisioning workflows with data transformation services. The method may include receiving, by a platform controller associated with a first domain, workflow information associated with a portion of a workflow to be deployed in a device ecosystem, where the portion of the workflow includes a first subportion of the workflow; identifying an output intent associated with data of the first subportion of the workflow; making a first determination that the output intent is associated with a data transformation of the data; making a second determination that the first domain is able to perform the data transformation; and establishing data transformation services using resources of the first domain; and initiating performance of the first subportion of the workflow, where executing the first subportion of the workflow includes executing the data transformation services.

Abstract: Techniques described herein relate to a method for deploying workflows with data management services.

Abstract: Techniques described herein relate to a method for optimizing workflow execution. The method may include receiving an event notification at a service controller, wherein the event notification is associated with a workflow deployed in a device ecosystem; performing, based on receiving the event notification, a workflow reconfiguration action comprising: providing a first workflow portion to a first platform controller in a first domain in the device ecosystem; transmitting a workflow reconfiguration action notification to a second platform controller in a second domain of the device ecosystem; identifying, by the second platform controller and in response to receiving the workflow reconfiguration action notification, a data transfer optimization action associated with data to be transmitted from the second domain to the first domain and used during execution of the first workflow portion; and transmitting the data from the second domain to the first domain using the data transfer optimization action.

Abstract: Techniques described herein relate to a method for deploying workflows. The method may include receiving, by a platform controller of a domain, a workflow portion and workflow constraints from a service controller of a federated controller, provisioning, by the platform controller, a set of devices in the domain to the workflow portion, executing the workflow portion in the domain using the set of devices, monitoring the executing of the workflow portion to determine an occupancy of the workflow portion, making a determination that the occupancy of the workflow portion exceeds an available capacity of the domain, providing, based on the determination, the occupancy to the service controller, and in response to the occupancy, performing, by the service controller, a remediation action.

Abstract: Techniques described herein relate to a method for deploying workflows with data management services.

Abstract: A method includes after being allocated to a composed information handling system of the composed information handling systems: monitoring, by a system control processor manager, health of computing resources of the composed information handling system, making a determination that a computing resource of the computing resources is in a compromised state, based on the determination, identifying a second computing resource currently allocated to a second composed information handling system, wherein the composed information handling system is associated with a first priority and the second composed information handling system is associated with a second priority, wherein the first priority is higher than the second priority, and replacing, after the deallocating, the compute resource with the second compute resource in the composed information handling system.

Abstract: Systems and methods are provided for dynamically reallocating resources during run-time execution of workloads in a distributed accelerator-as-a-service computing system to increase workload execution performance and resource utilization. A workload is executed in the distributed accelerator-as-a-service computing system using an initial set of resources allocated to the executing workload, wherein the allocated resources include accelerator resources (e.g., physical and/or virtual accelerator resources). The performance of the executing workload is monitored to detect a bottleneck condition which causes a decrease in the performance of the executing workload. In response to detecting the bottleneck condition, another set of resources is reallocated to the executing workload, which is determined to reduce or eliminate the bottleneck condition.

Abstract: A system, computer program product, and computer-executable method for managing flash devices within a data storage environment utilized by an application of one or more applications, wherein the application accesses the managed flash devices through a pool of flash storage provided by the data storage, the system, computer program product, and computer-executable method comprising receiving a data I/O from the application, analyzing the data I/O directed toward the pool of flash storage in relation to the flash devices, and managing the flash devices based on data I/Os directed toward the pool of flash storage by the application.

Abstract: Techniques are provided to decouple data pipeline tasks from an execution flow of a high-performance computing task (e.g., distributed deep model training) in a distributed computing system. For example, a method includes receiving a client request to provision resources for executing a computing job, provisioning accelerator resources of one or more accelerator server nodes in the distributed computing system to perform tasks associated with an execution flow of the computing job, and provisioning a logical nodes within the distributed computing system to compose a data flow pipeline which is configured to perform data flow operations associated with the computing job for providing data to the provisioned accelerator resources to perform the tasks associated with the execution flow of the computing job. The data flow operations include, e.g., data storage input/output operations, data pre-processing operations, and data staging operations, which are decoupled from the execution flow of the computing job.