Abstract: An example system includes a processor to receive a pseudo-relevance set including top results form a search engine in response to transmitting a set of concatenated messages of a dialog. The processor can execute a first fixed point operation on the pseudo-relevance set to generate weighted terms. The processor can also execute a second fixed point operation on a message graph including nodes with a heaviness based on the weighted terms.

Abstract: A computer-implemented method, computerized apparatus and computer program product for query performance prediction, the method comprising: obtaining a result list comprising a listing of documents retrieved from a collection in response to a query; obtaining for each of the listed documents in the result list a score indicating a measure of the document's relevance to the query; sampling the result list to obtain a plurality of sub-lists each of which comprising a listing of documents subsumed by the result list; for each of the plurality of sub-lists, analyzing scores of the documents listed therein to obtain a sample performance estimator; and estimating performance of the result list based on the sample performance estimator of each of the plurality of sub-lists.

Abstract: An exemplary method includes: determining a pool of documents, wherein each document is within at least one of a plurality of lists, each of the lists results from executing a query on a corpus, and the corpus comprises at least the pool of documents; determining a first ranking of documents within the pool of documents based at least in part on first scores computed for respective documents within the pool; estimating relevance to the specified query at least of respective documents within the first ranking, wherein the relevance is estimated without user feedback regarding the relevance; and determining a second ranking of documents within the pool based at least in part on second scores computed at least for respective documents within the first ranking, wherein the second score for a given document is computed based at least in part on the estimated relevance of at least the given document.

Abstract: Techniques are disclosed for query performance prediction (QPP) in the fusion-based retrieval setting. Symmetric list similarity measures used in traditional QPP techniques do not properly account for relevance-dependent aspects of the relationship between a given (base) reference list generated using an information retrieval technique and a final fused list generated using a fusion technique, as such a relationship is actually asymmetric. Embodiments more properly model the asymmetric relationship of reference and fused lists using an asymmetric co-relevance model that estimates, assuming a reference list contains relevant information, the odds that the fused list will be observed. In particular, the asymmetric co-relevance between a reference list and a fused list may be determined by adjusting a symmetric co-relevance of the reference list and the fused list using an odds ratio between the symmetric co-relevance of the reference list and the fused list to the reference list's own relevance.

Abstract: A method including: Obtaining multiple conversation texts, one text per conversation, wherein each of the multiple conversation texts comprises: multiple messages authored by multiple parties, and a reference to an electronic document that provides resolution of a problem that is common to all the conversations. Calculating an importance score for each of the multiple messages of all the conversation texts. Clustering the multiple messages of all the conversation texts into multiple bins. Calculating an aggregated importance score for each of the multiple bins, based on the importance scores of the messages contained in the respective bin. Enriching (a) the electronic document, or (b) a record of the electronic document in an index of electronic documents, with at least some of the multiple bins and their aggregated importance scores, wherein the at least some of the multiple bins are added as fields to the electronic document or to the record.

Abstract: Embodiments of the present systems and methods may provide techniques that provide improved information retrieval. For example, a method may comprise receiving, at the computer system, a query to retrieve a document from a corpus of documents, retrieving, at the computer system, a plurality of documents from the corpus of documents using a plurality of retrieval methods, each retrieval method generating a ranked list of retrieved documents and a score for each document, fusing, at the computer system, the generated ranked list of retrieved documents to form an aggregated ranked list of retrieved documents by re-scoring, at the computer system, the plurality of documents according to its passage scores, with respect to the query and associating, at the computer system, a given document and its maximal passage using relevance information induced from the plurality of ranked lists.

Abstract: A method including: Obtaining multiple conversation texts, one text per conversation, wherein each of the multiple conversation texts comprises: multiple messages authored by multiple parties, and a reference to an electronic document that provides resolution of a problem that is common to all the conversations. Calculating an importance score for each of the multiple messages of all the conversation texts. Clustering the multiple messages of all the conversation texts into multiple bins. Calculating an aggregated importance score for each of the multiple bins, based on the importance scores of the messages contained in the respective bin. Enriching (a) the electronic document, or (b) a record of the electronic document in an index of electronic documents, with at least some of the multiple bins and their aggregated importance scores, wherein the at least some of the multiple bins are added as fields to the electronic document or to the record.

Abstract: An example system includes a processor to receive a pseudo-relevance set including top results form a search engine in response to transmitting a set of concatenated messages of a dialog. The processor can execute a first fixed point operation on the pseudo-relevance set to generate weighted terms. The processor can also execute a second fixed point operation on a message graph including nodes with a heaviness based on the weighted terms.

Abstract: A method for automated selection of a search result ranker comprising: providing a set of queries; for each of said queries, receiving, from a search engine, a plurality of relevancy score sets, wherein each relevancy score set is associated with search results found in a corpus of electronic documents using each of a plurality of computerized search result rankers; calculating a difficulty score for each of said queries relative to all other queries in the set, based on said plurality of relevancy score sets associated with said query; calculating a quality score for each of said search result rankers based on said plurality of relevancy score sets associated with said search result ranker, wherein each of said plurality of relevancy score sets is weighed according to the difficulty score of its associated query; and selecting one of said search rankers based on said quality score.

Abstract: Methods and systems for generating and evaluating fused query lists. A query on a corpus of documents is evaluated using a plurality of retrieval methods and a ranked list for each of the plurality of retrieval methods is obtained. A plurality of fused ranked lists is sampled, each fusing said ranked lists for said plurality of retrieval methods, and the sampled fused ranked lists are sorted. In an unsupervised manner, an objective comprising a likelihood that a fused ranked list, fusing said ranked lists for each of said plurality of retrieval methods, is relevant to a query and a relevance event, is optimized to optimize the sampling, until convergence is achieved. Documents of the fused ranked list are determined based on the optimization.

Abstract: Embodiments of the present systems and methods may provide techniques that provide improved information retrieval. For example, a method may comprise receiving, at the computer system, a query to retrieve a document from a corpus of documents, retrieving, at the computer system, a plurality of documents from the corpus of documents using a plurality of retrieval methods, each retrieval method generating a ranked list of retrieved documents and a score for each document, fusing, at the computer system, the generated ranked list of retrieved documents to form an aggregated ranked list of retrieved documents by re-scoring, at the computer system, the plurality of documents according to its passage scores, with respect to the query and associating, at the computer system, a given document and its maximal passage using relevance information induced from the plurality of ranked lists.

Abstract: A computer implemented method for estimating quality of document retrieval comprising: retrieving from a corpus of documents stored on at least one storage a plurality of digital documents which comply with a document retrieval query according to a retrieval model; computing a plurality of retrieval scores each calculated for one of the plurality of digital documents using a relevance function scoring a relevance of one of the retrieved plurality of digital documents to the query; computing a calibrated weighted product model (WPM) estimator by calculating a combination of the plurality of retrieval scores weighted according to a plurality of retrieval features of the corpus and/or the query and/or a document, wherein the plurality of retrieval features are weighted according to a relative importance; and using the calibrated WPM estimator to score the plurality of digital documents' relevance to the query.

Abstract: Techniques are disclosed for query performance prediction (QPP) in the fusion-based retrieval setting. Symmetric list similarity measures used in traditional QPP techniques do not properly account for relevance-dependent aspects of the relationship between a given (base) reference list generated using an information retrieval technique and a final fused list generated using a fusion technique, as such a relationship is actually asymmetric. Embodiments more properly model the asymmetric relationship of reference and fused lists using an asymmetric co-relevance model that estimates, assuming a reference list contains relevant information, the odds that the fused list will be observed. In particular, the asymmetric co-relevance between a reference list and a fused list may be determined by adjusting a symmetric co-relevance of the reference list and the fused list using an odds ratio between the symmetric co-relevance of the reference list and the fused list to the reference list's own relevance.

Abstract: Methods and systems for generating and evaluating fused query lists. A query on a corpus of documents is evaluated using a plurality of retrieval methods and a ranked list for each of the plurality of retrieval methods is obtained. A plurality of fused ranked lists is sampled, each fusing said ranked lists for said plurality of retrieval methods, and the sampled fused ranked lists are sorted. In an unsupervised manner, an objective comprising a likelihood that a fused ranked list, fusing said ranked lists for each of said plurality of retrieval methods, is relevant to a query and a relevance event, is optimized to optimize the sampling, until convergence is achieved. Documents of the fused ranked list are determined based on the optimization.

Abstract: An exemplary method includes: determining a pool of documents, wherein each document is within at least one of a plurality of lists, each of the lists results from executing a query on a corpus, and the corpus comprises at least the pool of documents; determining a first ranking of documents within the pool of documents based at least in part on first scores computed for respective documents within the pool; estimating relevance to the specified query at least of respective documents within the first ranking, wherein the relevance is estimated without user feedback regarding the relevance; and determining a second ranking of documents within the pool based at least in part on second scores computed at least for respective documents within the first ranking, wherein the second score for a given document is computed based at least in part on the estimated relevance of at least the given document.

Abstract: A method for automated selection of a search result ranker comprising: providing a set of queries; for each of said queries, receiving, from a search engine, a plurality of relevancy score sets, wherein each relevancy score set is associated with search results found in a corpus of electronic documents using each of a plurality of computerized search result rankers; calculating a difficulty score for each of said queries relative to all other queries in the set, based on said plurality of relevancy score sets associated with said query; calculating a quality score for each of said search result rankers based on said plurality of relevancy score sets associated with said search result ranker, wherein each of said plurality of relevancy score sets is weighed according to the difficulty score of its associated query; and selecting one of said search rankers based on said quality score.

Abstract: A computer implemented method for estimating quality of document retrieval comprising: retrieving from a corpus of documents stored on at least one storage a plurality of digital documents which comply with a document retrieval query according to a retrieval model; computing a plurality of retrieval scores each calculated for one of the plurality of digital documents using a relevance function scoring a relevance of one of the retrieved plurality of digital documents to the query; computing a calibrated weighted product model (WPM) estimator by calculating a combination of the plurality of retrieval scores weighted according to a plurality of retrieval features of the corpus and/or the query and/or a document, wherein the plurality of retrieval features are weighted according to a relative importance; and using the calibrated WPM estimator to score the plurality of digital documents' relevance to the query.

Abstract: A computer-implemented method, computerized apparatus and computer program product for query performance prediction, the method comprising: obtaining a result list comprising a listing of documents retrieved from a collection in response to a query; obtaining for each of the listed documents in the result list a score indicating a measure of the document's relevance to the query; sampling the result list to obtain a plurality of sub-lists each of which comprising a listing of documents subsumed by the result list; for each of the plurality of sub-lists, analyzing scores of the documents listed therein to obtain a sample performance estimator; and estimating performance of the result list based on the sample performance estimator of each of the plurality of sub-lists.

Abstract: A method, including submitting, to a search engine, a first query including, and receiving, in response to the first query, a first list including first results, each of the first results having a respective first ranking. Keywords are derived from the first query, and for each keyword, a respective second query is submitted to the search engine, the respective second query including the first query term and the derived keyword. In response to each of the respective second queries, a respective second list including respective second results is received, each of the respective second results having a second ranking and a corresponding first result, and for each given second result, one or more stability scores are computed based on the second ranking of the given second result and the first ranking of the corresponding first result. The second results are ranked based on their respective one or more stability scores.

Abstract: A method, apparatus and computer program product for estimating group expertise, the method comprising: executing a query against a knowledge base to retrieve at least one document; retrieving at least one entity associated with the at least one document; assigning at least one relevancy score to the at least one entity; obtaining a filtered list by filtering the at least one entity to contain only entities appearing in a predetermined collection; and assessing findability of the query based on the at least one entity and the relevancy score.