Abstract: The invention is a method and apparatus for on-demand resource planning for unified messaging services. In one embodiment, multiple clients are served by a single system, and existing system resources are allocated among all clients in a manner that optimizes system output and service provider profit without the need to increase system resources. In one embodiment, resource allocation and job scheduling are guided by individual service level agreements between the service provider and the clients that dictate minimum service levels that must be achieved by the system. Jobs are processed in a manner that at least meets the specified service levels, and the benefit or profit derived by the service provider is maximized by prioritizing incoming job requests within the parameters of the specified service levels while meeting the specified service levels. Thus, operation and hardware costs remain substantially unchanged, while system output and profit are maximized.

Abstract: Identifying traffic patterns to web sites based on templates that characterize the arrival of traffic to the web sites is provided. Based on these templates, determinations are made as to which web sites should be co-located so as to optimize resource allocation. Web sites whose templates are complimentary, i.e. a first web site having a peak in arrival traffic at time t1 and a second web site that has a trough in arrival traffic at time t1, are designated as being candidates for co-location. In addition, the templates identified for the traffic patterns of web sites are used to determine thresholds for offloading traffic to other servers. These thresholds include a first threshold at which offloading should be performed, a second threshold that takes into consideration the lead time needed to begin offloading, and a third threshold that takes into consideration a lag time needed to stop offloading of traffic.

Abstract: A failure recovery framework to be used in cooperative data stream processing is provided that can be used in a large-scale stream data analysis environment. Failure recovery supports a plurality of independent distributed sites, each having its own local administration and goals. The distributed sites cooperate in an inter-site back-up mechanism to provide for system recovery from a variety of failures within the system. Failure recovery is both automatic and timely through cooperation among sites. Back-up sites associated with a given primary site are identified. These sites are used to identify failures within the primary site including failures of applications running on the nodes of the primary site. The failed applications are reinstated on one or more nodes within the back-up sites using job management instances local to the back-up sites in combination with previously stored state information and data values for the failed applications.

Abstract: Arrangements and methods for developing a software toolkit that can be used to design or obtain parameters for a sensor network. High-level guidelines on the basic relations between sensor network parameters like number of sensors, degree of quantization at each sensor, and the distortion requirements, based on a deep analysis on two basic coding possibilities (multiplexed point-to-point, distributed) are contemplated. By evaluating tradeoffs among the various parameters, an optimization framework to obtain the most cost-effective design with required quantization capabilities pertaining to given distortion criterion is provided.

Abstract: A system and method for resource allocation includes, in a network having nodes and links, injecting units of flow for at least one commodity at a source corresponding to the at least one commodity. At each node, queue heights, associated with the at least one commodity, are balanced for queues associated with each of one or more outgoing paths associated with that node. An amount of commodity flow is pushed across a link toward a sink, where the amount of commodity flow is constrained by a capacity constraint. Flow that reached the sink is absorbed by draining the queues.

Abstract: A system, method and computer program product for allocating computing resources to process a plurality of data streams. A system for allocating resources to process a plurality of data streams. The system includes, but is not limited to: a memory device and a processor being connected to the memory device. The system receives at least one query from a user. The system obtains at least one sub-query associated with the at least one query. The system identifies at least one data stream associated with the at least one sub-query. The system computes at least one probability that the at least one sub-query is true. The system assigns the computing resources to process the data streams according to the computed probability.

Abstract: A method to estimate demand of a market for a service is disclosed. The method includes selecting a set of feasible service offerings to offer for sale to the market from a set of candidate service offerings, observing a response of the market to the set of feasible service offerings offered for sale, and estimating a demand of the market for the service based upon the observed response. Each service offering of the set of feasible service offerings and the set of candidate service offerings is defined by a price and a service level.

Abstract: Methods and systems are provided for optimally trading off replication overhead and consistency levels in distributed data replication where nodes are organized in a hierarchy. The root node has the original data that need to be replicated at all other nodes, and the replicated copies have a freshness threshold that must be satisfied. The data are propagated through periodic updates in the hierarchy. Each node periodically sends data to its child nodes. Given the freshness threshold, an algorithm and its distributed protocol can determine the optimal update period for each link of the hierarchy such that the freshness threshold is satisfied for every node and the overall replication overhead is minimized. The systems and methods can be used in any scenario where replicated data have consistency requirements, such as in a replicate overlay assisted resource discovery system.

Abstract: Methods and apparatus operating in a stream processing network perform load shedding and dynamic resource allocation so as to meet a pre-determined utility criterion. Load shedding is envisioned as an admission control problem encompassing source nodes admitting workflows into the stream processing network. A primal-dual approach is used to decompose the admission control and resource allocation problems. The admission control operates as a push-and-pull process with sources pushing workflows into the stream processing network and sinks pulling processed workflows from the network. A virtual queue is maintained at each node to account for both queue backlogs and credits from sinks. Nodes of the stream processing network maintain shadow prices for each of the workflows and share congestion information with neighbor nodes.

Abstract: Methods and apparatus operating in a stream processing network perform load shedding and dynamic resource allocation so as to meet a pre-determined utility criterion. Load shedding is envisioned as an admission control problem encompassing source nodes admitting workflows into the stream processing network. A primal-dual approach is used to decompose the admission control and resource allocation problems. The admission control operates as a push-and-pull process with sources pushing workflows into the stream processing network and sinks pulling processed workflows from the network. A virtual queue is maintained at each node to account for both queue backlogs and credits from sinks. Nodes of the stream processing network maintain shadow prices for each of the workflows and share congestion information with neighbor nodes.

Abstract: Apparatus and methods for classifying web sites are provided. With the apparatus and methods, traffic data is obtained for a plurality of web sites. This patterns, or templates, for each web site are generated based on this traffic data and the patterns are clustered into classes of web sites using a clustering algorithm. The clusters, or classes, are then profiled to generate a template for each class. The template for each class is generated by first shifting the patterns for each web site that is part of the class to compensate for effects like time zone differences, if any, and then identifying a pattern that is most similar to all of the patterns in the class. Once the template for each class is generated, this template is then used with traffic data from a new web site to classify the new web site into one of the existing classes.

Abstract: Methods and system for dynamic reallocation of data processing resources for efficient processing of sensor data in a distributed network is provided. The methods and system include determining a data transmission cost ft; determining a data processing cost fp; determining a data storage cost fs; and determining a data query Q which minimizes f(ft+fp+fs) for a system of networked data processing resources.

Abstract: Methods and apparatus operating in a stream processing network perform load shedding and dynamic resource allocation so as to meet a pre-determined utility criterion. Load shedding is envisioned as an admission control problem encompassing source nodes admitting workflows into the stream processing network. A primal-dual approach is used to decompose the admission control and resource allocation problems. The admission control operates as a push-and-pull process with sources pushing workflows into the stream processing network and sinks pulling processed workflows from the network. A virtual queue is maintained at each node to account for both queue backlogs and credits from sinks. Nodes of the stream processing network maintain shadow prices for each of the workflows and share congestion information with neighbor nodes.

Abstract: Techniques are provided for determining a capacity allocation in a multi-tiered computing system. In one aspect of the invention, a technique for designing capacity allocation for a multi-tiered computing system, each tier of the computing system having one or more computing devices (e.g., servers), comprises the following steps/operations. Input parameters relating to capacity allocation are obtained. A capacity allocation is computed based on at least a portion of the input parameters, the capacity allocation being computable such that one or more end-to-end performance measures are substantially satisfied for multiple service classes. The capacity allocation computation may minimize a cost function.

Abstract: A method to estimate demand of a market for a service is disclosed. The method includes selecting a set of feasible service offerings to offer for sale to the market from a set of candidate service offerings, observing a response of the market to the set of feasible service offerings offered for sale, and estimating a demand of the market for the service based upon the observed response. Each service offering of the set of feasible service offerings and the set of candidate service offerings is defined by a price and a service level.

Abstract: The invention is a method and apparatus for on-demand resource planning for unified messaging services. In one embodiment, multiple clients are served by a single system, and existing system resources are allocated among all clients in a manner that optimizes system output and service provider profit without the need to increase system resources. In one embodiment, resource allocation and job scheduling are guided by individual service level agreements between the service provider and the clients that dictate minimum service levels that must be achieved by the system. Jobs are processed in a manner that at least meets the specified service levels, and the benefit or profit derived by the service provider is maximized by prioritizing incoming job requests within the parameters of the specified service levels while meeting the specified service levels. Thus, operation and hardware costs remain substantially unchanged, while system output and profit are maximized.

Abstract: Methods and systems are provided for optimally trading off replication overhead and consistency levels in distributed data replication where nodes are organized in a hierarchy. The root node has the original data that need to be replicated at all other nodes, and the replicated copies have a freshness threshold that must be satisfied. The data are propagated through periodic updates in the hierarchy. Each node periodically sends data to its child nodes. Given the freshness threshold, an algorithm and its distributed protocol can determine the optimal update period for each link of the hierarchy such that the freshness threshold is satisfied for every node and the overall replication overhead is minimized. The systems and methods can be used in any scenario where replicated data have consistency requirements, such as in a replicate overlay assisted resource discovery system.

Abstract: In one embodiment, the present invention is a method and apparatus for automatic system parameter configuration for performance improvement. One embodiment of the inventive method involves formulating a black box optimization problem, and solving the optimization problem using an enhanced smart hill climbing method. The smart hill climbing method includes both a global and a more precise local search to identify an optimal solution. In one embodiment, one or both of the global and local searches employs a weighted Latin Hypercube Sampling method in combination with importance sampling techniques to yield improved search results.

Abstract: Methods and apparatus operating in a stream processing network perform load shedding and dynamic resource allocation so as to meet a pre-determined utility criterion. Load shedding is envisioned as an admission control problem encompassing source nodes admitting workflows into the stream processing network. A primal-dual approach is used to decompose the admission control and resource allocation problems. The admission control operates as a push-and-pull process with sources pushing workflows into the stream processing network and sinks pulling processed workflows from the network. A virtual queue is maintained at each node to account for both queue backlogs and credits from sinks. Nodes of the stream processing network maintain shadow prices for each of the workflows and share congestion information with neighbor nodes.

Abstract: The invention is a method and apparatus for on-demand resource planning for unified messaging services. In one embodiment, multiple clients are served by a single system, and existing system resources are allocated among all clients in a manner that optimizes system output and service provider profit without the need to increase system resources. In one embodiment, resource allocation and job scheduling are guided by individual service level agreements between the service provider and the clients that dictate minimum service levels that must be achieved by the system. Jobs are processed in a manner that at least meets the specified service levels, and the benefit or profit derived by the service provider is maximized by prioritizing incoming job requests within the parameters of the specified service levels while meeting the specified service levels. Thus, operation and hardware costs remain substantially unchanged, while system output and profit are maximized.