Abstract: Techniques for composing an application resource model are disclosed.

Abstract: Techniques for determining feasibility of a set of one or more operator partitioning constraints are provided. The techniques include receiving one or more sets of operator partitioning constraints, wherein each set of one or more constraints define one or more desired conditions for grouping together of operators into partitions and placing partitions on hosts, wherein each operator is embodied as software that performs a particular function, processing each set of one or more operator partitioning constraints to determine feasibility of each set of one or more operator partitioning constraints, creating and outputting one or more candidate partitions and one or more host placements for each set of feasible partitioning constraints, and creating and outputting a certificate of infeasibility for each set of infeasible partitioning constraints, wherein the certificate of infeasibility outlines one or more reasons for infeasibility.

Abstract: Techniques for composing an application resource model are disclosed.

Abstract: Techniques for composing an application resource model in a data stream processing system are disclosed. The application resource model may be used to understand what resources will be consumed by an application when executed by the data stream processing system. For example, a method for composing an application resource model for a data stream processing system comprises the following steps. One or more operator-level metrics are obtained from an execution of a data stream processing application in accordance with a first configuration. The application is executed by one or more nodes of the data stream processing system, and the application is comprised of one or more processing elements that are comprised of one or more operators. One or more operator-level resource functions are generated based on the obtained one or more operator-level metrics. A processing element-level resource function is generated based on the one or more generated operator-level resource functions.

Abstract: Techniques for composing an application resource model in a data stream processing system are disclosed. The application resource model may be used to understand what resources will be consumed by an application when executed by the data stream processing system. For example, a method for composing an application resource model for a data stream processing system comprises the following steps. One or more operator-level metrics are obtained from an execution of a data stream processing application in accordance with a first configuration. The application is executed by one or more nodes of the data stream processing system, and the application is comprised of one or more processing elements that are comprised of one or more operators. One or more operator-level resource functions are generated based on the obtained one or more operator-level metrics. A processing element-level resource function is generated based on the one or more generated operator-level resource functions.

Abstract: Systems and methods are provided for optimizing the performance and/or allocation of constrained resources in a dynamic computing environment using adaptive regulatory control methods. For example, systems and methods for providing constrained optimization in a dynamic computing system implement model-based adaptive (self-tuning) regulatory control schemes that are designed to handle the system dynamics and which take into consideration control costs (such as the overheads of changing resource allocations and performance degradation due to transient load imbalances) to find an optimal solution. To facilitate practical application, a dual control architecture is disclosed which combines a heuristic fixed step control process that is implemented when there is no valid system model for model-based control. A system model is continually obtained and validated during run-time to adapt control parameters to variations in system dynamics.