Abstract: Systems and methods for allocating computing resources within a distributed computing system are disclosed. Computing resources such as CPUs, GPUs, network cards, and memory are allocated to jobs submitted to the system by a scheduler. System configuration and interconnectivity information is gathered by a mapper and used to create a graph. Resource allocation is optimized based on one or more quality of service (QoS) levels determined for the job. Job performance characterization, affinity models, computer resource power consumption, and policies may also be used to optimize the allocation of computing resources.

Abstract: A system and method for ranking computing resources in a distributed computing marketplace is disclosed. Ranking may be based on the performance factors that the system predicts will have the greatest impact on the particular application the user plans to run. A performance database stores historical performance data for applications that have been executed on multiple different computer systems. The database is checked to see if the application, or one similar, has already been run on any of the computing systems participating in the distributed computing marketplace. If so, the existing performance data is used to predict which performance factors will have the greatest impact on the application. Those factors are then used to rank the available computing systems options for the user.

Abstract: Systems, methods, and apparatuses are disclosed for implementation and management of a network of computing clusters and interfaces. In various examples, a dynamic supercomputing resource marketplace system includes a cluster network having one or more interconnected computing clusters. The dynamic supercomputing resource marketplace system includes a user interface system and/or an application program interface system for enabling a user to access the computing clusters. Advantageously, the dynamic supercomputing resource marketplace system facilitates increased utilization of computing clusters.

Abstract: A method for configuring replicas in a distributed computing system is disclosed. In one example embodiment, a plurality of replicas with associated bootstrap modules may be created. The same bootstrap module code may be used for each replica, thereby simplifying configuration. Using the bootstrap module, the replicas may automatically configure themselves and self-assign a role for a set of predetermined roles such as master and worker. The bootstrap module may check a predetermined location such as a shared network folder for earlier registration entries and then self-select based on the remaining available roles. The bootstrap module may also store its own registration entry to inform subsequent replicas of the role and network address for the current replica so that they may self-configure correctly.

Abstract: Systems and methods for building applications by automatically incorporating application performance data into the application build process are disclosed. By capturing build settings and performance data from prior applications being executed on different computing systems such as bare metal and virtualized cloud instances, a performance database may be maintained and used to predict build settings that improve application performance (e.g., on a specific computing system or computing system configuration).

Abstract: Systems, methods, and apparatuses are disclosed for implementation and management of a network of computing clusters and interfaces. In various embodiment, a dynamic supercomputing resource marketplace system can include a cluster network having one or more interconnected computing clusters. The dynamic supercomputing resource marketplace system also can include a user interface system or an application program interface system for enabling a user to access the computing clusters. Advantageously, the dynamic supercomputing resource marketplace system can be used to facilitate increased utilization of computing clusters.

Abstract: Systems and methods for building applications by automatically incorporating application performance data into the application build process are disclosed. By capturing build settings and performance data from prior applications being executed on different computing systems such as bare metal and virtualized cloud instances, a performance database may be maintained and used to predict build settings that improve application performance (e.g., on a specific computing system or computing system configuration).

Abstract: Systems, methods and apparatuses are disclosed for implementation and management of a dynamic compute and application marketplace. The dynamic computer marketplace system can coordinate access to one or more other computing resources, including on-premises computing resources, external (or off-premises) computing resources or a combination thereof. In various embodiments, the dynamic computer marketplace system advantageously can be used to facilitate inter-provider migration, transparent pricing, and/or competitive pricing, among other things.

Abstract: Examples related to providing cloud-based computational services to a requesting entity having an infrastructure management agent. Revenue accounting corresponding to a first time period is performed for computational services utilized by the requesting entity. If revenue has changed over the first period of time beyond a pre-selected threshold amount, the infrastructure management agent adjusts quality of service (QoS) parameters or modifies billing discounts for the cloud-based computational services.

Abstract: A system and method for recommending computing resources for processing jobs in a distributed computing environment with multiple heterogeneous computing resources are disclosed. Training applications or jobs are executed and measured on different computing resources and on different configurations of the computing resources to establish a database of performance metrics. A matrix of application features and computing resource features is created and populated with performance data. Machine learning may be used to create and update multiple recommendation engines based on the matrix that are cross-validated and merged to form final performance estimators. The performance estimators are applied to new applications and determine which existing applications are most similar and which resources to recommend.

Abstract: Systems and methods for allocating computing resources within a distributed computing system are disclosed. Computing resources such as CPUs, GPUs, network cards, and memory are allocated to jobs submitted to the system by a scheduler. System configuration and interconnectivity information is gathered by a mapper and used to create a graph. Resource allocation is optimized based on one or more quality of service (QoS) levels determined for the job. Job performance characterization, affinity models, computer resource power consumption, and policies may also be used to optimize the allocation of computing resources.

Abstract: A system and method for processing application performance using application phase differentiation and detection is disclosed. Phase detection may be accomplished in a number of different ways, including by using a deterministic algorithm that looks for changes in the computing resource utilization patterns (as detected in the performance data collected). Machine learning (ML) and neural networks (e.g. sparse auto encoder SAE) may also be used. Performance data is aggregated according to phase and stored in a database along with additional application and computing system information. This database may then be used to find similar applications for performance prediction.

Abstract: Systems and methods for allocating computing resources within a distributed computing system are disclosed. Computing resources such as CPUs, GPUs, network cards, and memory are allocated to jobs submitted to the system by a scheduler. System configuration and interconnectivity information is gathered by a mapper and used to create a graph. Resource allocation is optimized based on one or more quality of service (QoS) levels determined for the job. Job performance characterization, affinity models, computer resource power consumption, and policies may also be used to optimize the allocation of computing resources.

Abstract: Systems and apparatuses for provisioning computer services or resources and methods for making and using the same. In one embodiment, an exemplary method for performing an iterative search can include selecting a service from a group of available services for adjustment. An application associated with the selected service can be run, and an amount of resources consumed while the application is run can be captured. A provision level for the selected service, a provision type for the selected service or both can be adjusted based upon the captured amount of resources consumed. The method then can determine whether provision levels of the available services, provision types of the available services or both require further adjustment to be most performant. The approaches described herein advantageously can be applied, for example, to “right-size” or “scale” multiple resources.

Abstract: A system and method for efficiently creating and managing application instances in distributed computing systems is disclosed. Controls are presented for specifying an application for instantiation, a data file for use with the application, and a destination for results from the application. Application resources and topology may be recommended to the user based on prior application execution, and CPU, GPU, and interconnect parameters such as bandwidth and latency. The controls may enable to user to customize the recommendations prior to automated instantiation based on the user's needs, such as whether the application is to be run in batch mode or interactive mode.

Abstract: A method for configuring replicas in a distributed computing system is disclosed. In one example embodiment, a plurality of replicas with associated bootstrap modules may be created. The same bootstrap module code may be used for each replica, thereby simplifying configuration. Using the bootstrap module, the replicas may automatically configure themselves and self-assign a role for a set of predetermined roles such as master and worker. The bootstrap module may check a predetermined location such as a shared network folder for earlier registration entries and then self-select based on the remaining available roles. The bootstrap module may also store its own registration entry to inform subsequent replicas of the role and network address for the current replica so that they may self-configure correctly.

Abstract: An improved multi-level scheduling system and method are disclosed. In one embodiment, the system comprises a coarse scheduler to allocate sets of computing resources at a first level and a set of fine grain schedulers configured to schedule at a second level, wherein the second level comprises individual computing resources within each set of computing resources. The fine grain scheduler may be configured to communicate with the coarse scheduler and monitor performance and utilization of the individual computing resources. The fine grain schedulers may also be configured to implement a different set of allocation rules than the coarse scheduler and request additional sets of resources from the coarse scheduler based on current and predicted utilization of the individual computing resources.

Abstract: A system and method for ranking computing resources in a distributed computing marketplace is disclosed. Ranking may be based on the performance factors that the system predicts will have the greatest impact on the particular application the user plans to run. A performance database stores historical performance data for applications that have been executed on multiple different computer systems. The database is checked to see if the application, or one similar, has already been run on any of the computing systems participating in the distributed computing marketplace. If so, the existing performance data is used to predict which performance factors will have the greatest impact on the application. Those factors are then used to rank the available computing systems options for the user.

Abstract: A system and method for job-based bidding for application processing work is disclosed. The system may include interfaces for submitting requests for bids (RFBs) and corresponding offers. The system may allow pricing based on the type of application and the quantity and type of work primitive to be processed, and it may use prior captured performance data to calculate estimated per unit of work costs that can be translated to different system types based on their capabilities. This per unit of work cost may assist providers in making offers on the RFBs. Recommended job requirements may also be generated. Once an offer is accepted, the system may configure and dispatch the job to the appropriate provider computing queue(s).

Abstract: Systems and methods for estimating the scalability of applications in high performance computing and distributed computing environments and for configuring applications based on those estimates are disclosed. A model is disclosed that provides an estimate of the scalability behavior of an application based on basic parameters and a small number of runs on bare metal and cloud systems. The system may also be configured to use the estimated performance to recommend optimal configurations based on different policies, including best performance, lowest cost, and best performance per cost.