Abstract: A prediction method for estimating the running time of WAND queries executed on a Web search engine which includes an off-line component using the Discrete Fourier Transform to models the index as a collection of signals to obtain characteristic vectors for query terms and an on-line feed-forward neural network with back-propagation to estimate the time required to process the incoming queries. The DFT is used to obtain values for six characteristics of the posting lists associated with the query terms. These characteristics are used to train a neuronal network which is used to predict the query execution time.

Abstract: An asynchronous parallel simulation protocol useful to simulate events in a stream processing platform. The invention is a windowing scheme which is suitable for distributed stream computing platforms containing facilities for supporting fully asynchronous processing elements and downstream event flows. It is a realization of the concept of a BSP superstep based oracle simulator that enables using statistics from the recent past to conduct the actual optimistic simulation at low rate of straggler messages. The invention is capable of achieving good statistic agreement with results from sequential simulations of the same models.

Abstract: The stream processing engine uses the Actor programming paradigm for defining the application in terms of a graph built with processing elements (PEs) that use a hash based partitioning of data, where events (key, value) are pushed towards the next element in the operator, and in case of an overloaded PE the method changes to a Producer/Consumer Model where new workers pull events from a buffer queue in order to release the amount of traffic in the overloaded PE. The programmer defines a sequential version of the PE and other parallel version that recovers the events from a buffer and, if the operator is stateless sends the result to the next PE, or if the operator is stateful sends the result to an aggregator PE before moving to the next stage of the pipeline process. Strategies for triggering changes in the graph are defined in an administrator module to provide the right amount of elasticity and load balance in the distributed stream processing engine using queues analysis of the monitoring module.