Abstract: Systems and methods for regularly updating portions of a merged index are provided. Initially, upon receiving an indication that modifications have occurred to content of web-based documents, dynamic update of index (DUI) objects that identify the documents and expose the modified content are composed by ascertaining relative positions of the modified content within the documents, and packaging identifiers of the documents, the relative positions, and metadata underlying the modified content into a message. The DUI objects are applied to an overloading index that maintains structured records of recent modifications. In particular, portions of the overloading index are targeted utilizing the document identifiers and the relative positions specified by the DUI object, thereby updating the targeted portions within the overloading index corresponding to the modified content without rewriting the entire overloading index.

Abstract: Systems and methods for regularly updating portions of a merged index are provided. Initially, upon receiving an indication that modifications have occurred to content of web-based documents, dynamic update of index (DUI) objects that identify the documents and expose the modified content are composed by ascertaining relative positions of the modified content within the documents, and packaging identifiers of the documents, the relative positions, and metadata underlying the modified content into a message. The DUI objects are applied to an overloading index that maintains structured records of recent modifications. In particular, portions of the overloading index are targeted utilizing the document identifiers and the relative positions specified by the DUI object, thereby updating the targeted portions within the overloading index corresponding to the modified content without rewriting the entire overloading index.

Abstract: Systems and methods for regularly updating portions of a merged index are provided. Initially, upon receiving an indication that modifications have occurred to content of web-based documents, dynamic update of index (DUI) objects that identify the documents and expose the modified content are composed by ascertaining relative positions of the modified content within the documents, and packaging identifiers of the documents, the relative positions, and metadata underlying the modified content into a message. The DUI objects are applied to an overloading index that maintains structured records of recent modifications. In particular, portions of the overloading index are targeted utilizing the document identifiers and the relative positions specified by the DUI object, thereby updating the targeted portions within the overloading index corresponding to the modified content without rewriting the entire overloading index.

Abstract: A search engine can provide referencing information as context for a particular search result when an excerpt from the search result, comprising at least some similar elements to the user's query, is not generated. Referencing information can include one or more anchor texts having similarity to at least some elements of the user's query, the anchor texts being used by referencing pages to link to the page returned as a search result. User selection of the anchor text can enable the user to visit a referencing page using that anchor text to link to the page returned as a search result, and having a high static rank.

Abstract: A search engine can provide referencing information as context for a particular search result when an excerpt from the search result, comprising at least some similar elements to the user's query, is not generated. Referencing information can include one or more anchor texts having similarity to at least some elements of the user's query, the anchor texts being used by referencing pages to link to the page returned as a search result. User selection of the anchor text can enable the user to visit a referencing page using that anchor text to link to the page returned as a search result, and having a high static rank.

Abstract: A search engine system is described herein that provides an alternative data path for collecting results provided by index servers. The alternative data path collects the results in a direct and asynchronous manner; this is in contrast to a synchronous path used to deliver search results to end users via one or more aggregator modules. An analysis system can use the alternative data path to collect a large amount of richly descriptive information regarding the performance of the search engine system, circumventing bottlenecks and other constraints that would otherwise be imposed by the synchronous data path. The analysis system can analyze the information collected from the index servers to improve the performance of the search engine system.

Abstract: Methods, systems, and media are provided for an optimized search engine index. The optimized index is formed by merging small lower level indexes of fresh documents together into a hierarchical cluster of multiple higher level indexes. The optimized index of fresh documents is formed via a single threaded process, while a fresh index serving platform concurrently serves fresh queries. The hierarchy of higher level indexes is formed by merging lower and/or higher level indexes with similar expiration times together. Therefore, as some indexes expire, the remaining un-expired indexes can be re-used and merged with new incoming indexes. The single threaded process provides fast serving of fresh documents, while also providing time to integrate the fresh indexes into a long term primary search engine index, prior to expiring.

Abstract: Systems and methods are provided for regularly updating a web index with new or updated content, such as meta words or meta streams, for a particular web page address, such as a URL. Web page addresses and associated updated information, such as meta words, meta streams, values, and locations in the web index for those meta words are received. In order to update a web index, which is used by search engines to search web documents, a document identification is retrieved and associated with the updated information. As information in the web index is stored by document identification and not by web page addresses, the document identification may replace the web page address. Each meta word received is matched with corresponding document identifications and associated updated information, which creates an inverted format of the information. The web index may now be updated and stored by the system.

Abstract: Search macros suggestions are provided to refine a user's search. When a search query is received from an end user, one or more search macros are determined to be relevant to the search query. The search macros are then provided to the end user as suggestions for refining the user's search. In some instances, the end user may chose to select one of the suggested search macros. A search is then performed using the search query and the selected search macro to provide search results to the end user that may be more relevant to the user's search.

Abstract: Methods, systems, and media are provided for identifying and clustering queries that are rising in popularity. Resultant clustered queries can be compared to other stored queries using textual and temporal correlations. Fresh indices containing information and results from recently crawled content sources are searched to obtain the most recent query activity. Historical indices are also searched to obtain temporally correlated information and results that match the clustered query stream. A weighted average acceleration of a spike can be calculated to distinguish between a legitimate spike and a non-legitimate spike. Legitimate clusters are combined with other stored clusters and presented as grouped content results to a user output device.

Abstract: Computational units of any task may run in different silos. In an embodiment, a search query may be evaluated efficiently on a non-uniform memory architecture (NUMA) machine, by assigning separate chunks of the index to separate memories. In a NUMA machine, each socket has an attached memory. The latency time is low or high, depending on whether a processor accesses data in its attached memory or a different memory. Copies of an index manager program, which compares a query to an index, run separately on different processors in a NUMA machine. Each instance of the index manager compares the query to the index chunk in the memory attached to the processor on which that instance is running. Thus, each instance of the index manager may compare a query to a particular portion of the index using low-latency accesses, thereby increasing the efficiency of the search.

Abstract: Systems and methods for regularly updating portions of a merged index are provided. Initially, upon receiving an indication that modifications have occurred to content of web-based documents, dynamic update of index (DUI) objects that identify the documents and expose the modified content are composed by ascertaining relative positions of the modified content within the documents, and packaging identifiers of the documents, the relative positions, and metadata underlying the modified content into a message. The DUI objects are applied to an overloading index that maintains structured records of recent modifications. In particular, portions of the overloading index are targeted utilizing the document identifiers and the relative positions specified by the DUI object, thereby updating the targeted portions within the overloading index corresponding to the modified content without rewriting the entire overloading index.

Abstract: A search engine system is described herein that provides an alternative data path for collecting results provided by index servers. The alternative data path collects the results in a direct and asynchronous manner; this is in contrast to a synchronous path used to deliver search results to end users via one or more aggregator modules. An analysis system can use the alternative data path to collect a large amount of richly descriptive information regarding the performance of the search engine system, circumventing bottlenecks and other constraints that would otherwise be imposed by the synchronous data path. The analysis system can analyze the information collected from the index servers to improve the performance of the search engine system.

Abstract: Methods, systems, and media are provided for an optimized search engine index. The optimized index is formed by merging small lower level indexes of fresh documents together into a hierarchical cluster of multiple higher level indexes. The optimized index of fresh documents is formed via a single threaded process, while a fresh index serving platform concurrently serves fresh queries. The hierarchy of higher level indexes is formed by merging lower and/or higher level indexes with similar expiration times together. Therefore, as some indexes expire, the remaining un-expired indexes can be re-used and merged with new incoming indexes. The single threaded process provides fast serving of fresh documents, while also providing time to integrate the fresh indexes into a long term primary search engine index, prior to expiring.

Abstract: Systems and methods for regularly updating portions of a merged index are provided. Initially, upon receiving an indication that modifications have occurred to content of web-based documents, dynamic update of index (DUI) objects that identify the documents and expose the modified content are composed by ascertaining relative positions of the modified content within the documents, and packaging identifiers of the documents, the relative positions, and metadata underlying the modified content into a message. The DUI objects are applied to an overloading index that maintains structured records of recent modifications. In particular, portions of the overloading index are targeted utilizing the document identifiers and the relative positions specified by the DUI object, thereby updating the targeted portions within the overloading index corresponding to the modified content without rewriting the entire overloading index.

Abstract: Systems, methods, computer-readable media, and graphical user interfaces for presenting search results having collapsed domains are provided. A search result obtaining module obtains search results based upon a received query. Upon obtaining the search results, search results having the same domain are associated with one another. Thereafter, search result clusters of associated search results are formed. In some embodiments, the search result clusters may be formatted to include desired search result cluster attributes. The search result clusters are presented such that two or more associated search results form a single cluster of search results rather than being presented individually. In some embodiments, an option to view more search results with the same domain may be provided.

Abstract: Systems and methods for performing an updating process to an in-memory index are provided. Upon receiving notice of document modifications covered by an inverted index associated with a search engine, in the form of an update file, a representation of the modification is published onto various index serving machines. Each index serving machine receiving the update file determines if the modifications are applicable to the index serving machine. If an index serving machine determines that it contains mapping information corresponding to the modified documents, the index serving machine utilizes the update file and associated mapping information to update an in-memory index. In embodiments, the in-memory index is used to provide results to user queries in tandem with the inverted index. In some embodiments, an extra in-memory index is maintained that is revised with constantly incoming metadata updates and the existing in-memory index is periodically swapped with the revised in-memory index.

Abstract: Methods, systems, and media are provided for identifying and clustering queries that are rising in popularity. Resultant clustered queries can be compared to other stored queries using textual and temporal correlations. Fresh indices containing information and results from recently crawled content sources are searched to obtain the most recent query activity. Historical indices are also searched to obtain temporally correlated information and results that match the clustered query stream. A weighted average acceleration of a spike can be calculated to distinguish between a legitimate spike and a non-legitimate spike. Legitimate clusters are combined with other stored clusters and presented as grouped content results to a user output device.

Abstract: Methods, systems, and media are provided for synchronizing information across multiple environments of a synchronization system. A search query is received into a frontend infrastructure of a first synchronization environment. The frontend infrastructure checks a local cache manager to see if results already exist for the search query. If existing results are not found, then one or more backend search engines of the first synchronization environment are utilized for the search query. The search results from the backend search engines are saved into the local cache manager of the first synchronization environment. A cache sync notification is created to identify the contents and location of the actual saved results. The cache sync notification is saved in a cache synchronization service located within the first synchronization environment, and broadcast to all other synchronization environments within the synchronization system. The actual results can be retrieved from any other synchronization environment.

Abstract: The embodiments contemplate a system and method for obtaining related results for a portion of a query and for generating an updated set of queries for a cache of a server. Other queries beginning with the portion of the query may be identified and obtained from a data structure that includes a server cache and a set of common queries. Once the other queries are identified, results for the other queries are obtained from the server cache or from a back-end database. A set of common queries, which may include deleted and additional queries, may be utilized to generate the updated set of queries for the server. Both missing queries and deleted queries that may belong to the server based on an assignment function are inserted into a queue, which is later delivered to the cache of the server. The transfer may occur during a low-activity or idle state.