Abstract: Identifying traffic patterns to web sites based on templates that characterize the arrival of traffic to the web sites is provided. Based on these templates, determinations are made as to which web sites should be co-located so as to optimize resource allocation. Web sites whose templates are complimentary, i.e. a first web site having a peak in arrival traffic at time t1 and a second web site that has a trough in arrival traffic at time t1, are designated as being candidates for co-location. In addition, the templates identified for the traffic patterns of web sites are used to determine thresholds for offloading traffic to other servers. These thresholds include a first threshold at which offloading should be performed, a second threshold that takes into consideration the lead time needed to begin offloading, and a third threshold that takes into consideration a lag time needed to stop offloading of traffic.

Abstract: Apparatus and methods for classifying web sites are provided. With the apparatus and methods, traffic data is obtained for a plurality of web sites. This patterns, or templates, for each web site are generated based on this traffic data and the patterns are clustered into classes of web sites using a clustering algorithm. The clusters, or classes, are then profiled to generate a template for each class. The template for each class is generated by first shifting the patterns for each web site that is part of the class to compensate for effects like time zone differences, if any, and then identifying a pattern that is most similar to all of the patterns in the class. Once the template for each class is generated, this template is then used with traffic data from a new web site to classify the new web site into one of the existing classes.

Abstract: A Web crawler data collection method is provided for collecting information associated with a plurality of queries, which is used to calculate estimates of return probabilities, clicking probabilities and incorrect response probabilities. The estimated return probabilities relate to a probability that a search engine will return a particular Web page in a particular position of a particular query result page. The estimated clicking probabilities relate to a frequency with which a client selects a returned Web page in a particular position of a particular query result. The estimated incorrect response probabilities relate to the probability that a query to a stale version of a particular Web page yields an incorrect or vacuous response. Further, information may be collected regarding the characteristics and update time distributions of a plurality of Web pages.

Abstract: Mathematical means and methods are used within the context of mathematical models of a workforce evolution to address key issues in workforce design and planning. Examples of such mathematical means and methods are (but not limited to) fluid-flow models and diffusion-process models. In each case, these mathematical models characterize the workforce evolution over time as a function of dynamic workforce events, such as new hires, terminations, resignations, retirements, promotions and transfers, and dynamic workforce topology, policy, or scenario, such as the viable paths from one workforce resource state to another workforce resource state.

Abstract: Mathematical means and methods are used within the context of mathematical models of a workforce evolution to address key issues in workforce design and planning. Examples of such mathematical means and methods are (but not limited to) fluid-flow models and diffusion-process models. In each case, these mathematical models characterize the workforce evolution over time as a function of dynamic workforce events, such as new hires, terminations, resignations, retirements, promotions and transfers, and dynamic workforce topology, such as the viable paths from one workforce resource state to another workforce resource state.

Abstract: A methodology for business process analysis and optimization. This enables firms to analyze business processes using stochastic processing network models to estimate process key performance indicators. Based on these indicators, alternate process models can be further developed and analyzed, in order to optimize business objectives. The analysis methodology can be used for business process design (at design time) and for business process management (at run time).

Abstract: Apparatus and methods for identifying traffic patterns to web sites based on templates that characterize the arrival of traffic to the web sites are provided. Based on these templates, determinations are made as to which web sites should be co-located so as to optimize resource allocation. Specifically, web sites whose templates are complimentary, i.e. a first web site having a peak in arrival traffic at time t1 and a second web site that has a trough in arrival traffic at time t1, are designated as being candidates for co-location. In addition, the present invention uses the templates identified for the traffic patterns of web sites to determine thresholds for offloading traffic to other servers. These thresholds include a first threshold at which offloading should be performed, a second threshold that takes into consideration the lead time needed to begin offloading, and a third threshold that takes into consideration a lag time needed to stop all offloading of traffic to the other servers.

Abstract: A technique is provided for efficient search engine crawling. First, optimal crawling frequencies, as well as the theoretically optimal times to crawl each Web page, are determined. This is performed under an extremely general distribution model of Web page updates, one which includes both stochastic and generalized deterministic update patterns. Techniques from the theory of resource allocation problems which are extraordinarily computationally efficient, crucial for practicality because the size of the problem in the Web environment is immense. The second part employs these frequencies and ideal crawl times as input, creating an optimal achievable schedule for crawlers. The solution, based on network flow theory, is exact and highly efficient as well.

Abstract: A method is provided for Web crawler data collection. The method includes the step of collecting information associated with a plurality of queries, the information related to results of the queries and/or responses to the queries. Estimates of return probabilities, clicking probabilities and incorrect response probabilities are then calculated at least in part based on the collected information. The estimated return probabilitues relate to a probability that a search engine will return a particular Web page in a particular position of a particular query result page. The estimated clicking probabilities relate to a frequency with which a client selects a returned Web page in a particular position of a particular query result. The estimated incorrect response probabilities relate to the probability that a query to a stale version of a particular Web page yields an incorrect or vacuous response.

Abstract: Apparatus and methods for classifying web sites are provided. With the apparatus and methods, traffic data is obtained for a plurality of web sites. This patterns, or templates, for each web site are generated based on this traffic data and the patterns are clustered into classes of web sites using a clustering algorithm. The clusters, or classes, are then profiled to generate a template for each class. The template for each class is generated by first shifting the patterns for each web site that is part of the class to compensate for effects like time zone differences, if any, and then identifying a pattern that is most similar to all of the patterns in the class. Once the template for each class is generated, this template is then used with traffic data from a new web site to classify the new web site into one of the existing classes.

Abstract: Apparatus and methods for identifying traffic patterns to web sites based on templates that characterize the arrival of traffic to the web sites are provided. Based on these templates, determinations are made as to which web sites should be co-located so as to optimize resource allocation. Specifically, web sites whose templates are complimentary, i.e. a first web site having a peak in arrival traffic at time t1 and a second web site that has a trough in arrival traffic at time t1, are designated as being candidates for co-location. In addition, the present invention uses the templates identified for the traffic patterns of web sites to determine thresholds for offloading traffic to other servers. These thresholds include a first threshold at which offloading should be performed, a second threshold that takes into consideration the lead time needed to begin offloading, and a third threshold that takes into consideration a lag time needed to stop all offloading of traffic to the other servers.

Abstract: The invention provides the mechanism, hereinafter referred to as Flexible Dynamic Partitioning (FDP), to allocate and reallocate resources among scheduling schemes of many types for multicomputing environments. Resources can include, but are not limited to, processors, disks and communications connections. Partitioning of resources can be initiated by both application and system triggers. Once dynamic partitioning is triggered, FDP allows a partition to invoke a set of resource allocation functions associated with its partition. The reallocation function performs a set of resource matchings and determine the necessary resource movement among partitions.

Abstract: A system and method whereby a gang scheduling entity assembles a schedule of application tasks to be run on a plurality of distributed parallel processors for pre-set time intervals. The scheduling information, including an ordered set of applications and the length of the time interval, are provided to each of the node level schedulers for the relevant processors, and the node level schedulers assure that the tasks of the applications are executed, with context switching between tasks being controlled at the node level. Tasks from multiple applications can be scheduled simultaneously, with time sharing and space sharing being utilized in the schedule. The inventive system is dynamically scalable, with the gang scheduling entity restructuring the matrix schedule should dynamic re-partitioning result in addition or deletion of processors.

Abstract: A method of scheduling jobs to be executed by a resource in a computer system wherein the jobs are grouped in “classes.” The job classes vying for the resource's attention are arranged in a hierarchy. Each job class has a time-function value that controls when the job class is selected by the resource if processing time becomes available. Within a particular level of the hierarchy, scheduling priorities are defined by one or more time-based functions, each of which may be constant or dynamically varying. When constant time-based functions are used, each job class has a schedule value that remains fixed with time. When dynamic time-based functions are used, job class “time-function values” are modified to alter the timing by which the job class(es) acquire the resource.