Abstract: The present invention relates to methods, systems, and computer readable media comprising instructions for identifying needy queries for which additional responsive content is needed. The method of the present invention comprises receiving a query comprising one or more terms and retrieving one or more content items identified as responsive to the query, the one or more content items ranked according to one or more ranking techniques. A score is generated for the one or more ranked content items identified as responsive to the query. A determination is thereafter made as to whether the query is needy based upon a comparison of the one or more scores associated with the one or more content items identified as responsive to the query and a needy query score threshold.

Abstract: An improved system and method for crawl ordering of a web crawler by impact upon search results of a search engine is provided. Content-independent features of uncrawled web pages may be obtained, and the impact of uncrawled web pages may be estimated for queries of a workload using the content-independent features. The impact of uncrawled web pages may be estimated for queries by computing an expected impact score for uncrawled web pages that match needy queries. Query sketches may be created for a subset of the queries by computing an expected impact score for crawled web pages and uncrawled web pages matching the queries. Web pages may then be selected to fetch using a combined query-based estimate and query-independent estimate of the impact of fetching the web pages on search query results.

Abstract: An improved system and method for matching objects using a cluster-dependent multi-armed bandit is provided. The matching may be performed by using a multi-armed bandit where the arms of the bandit may be dependent. In an embodiment, a set of objects segmented into a plurality of clusters of dependent objects may be received, and then a two step policy may be employed by a multi-armed bandit by first running over clusters of arms to select a cluster, and then secondly picking a particular arm inside the selected cluster. The multi-armed bandit may exploit dependencies among the arms to efficiently support exploration of a large number of arms. Various embodiments may include policies for discounted rewards and policies for undiscounted reward. These policies may consider each cluster in isolation during processing, and consequently may dramatically reduce the size of a large state space for finding a solution.

Abstract: An improved system and method for matching objects belonging to hierarchies is provided and an optimal matching between two feature spaces organized as taxonomies may be learned. The matching may be performed through a multi-level exploration of the hierarchical feature spaces by using multi-armed bandits where the arms of the bandit may be dependent due to the structure induced by the taxonomies. Upon the arrival of an object assigned to the first taxonomy, multi-armed bandits may be run at multiple levels of the taxonomies to select an object assigned to the second taxonomy. Then shrinkage estimation may be performed in a Bayesian framework to exploit dependencies among the arms by estimating payoff probabilities from a beta-binomial model to update payoff probabilities for matching objects from the taxonomies.