Abstract: Method for determining a webpage importance, including receiving web browsing behavior data of one or more users; creating a model of the web browsing behavior data; calculating a stationary probability distribution of the model; and correlating the stationary probability distribution to the webpage importance.

Abstract: Method for creating a graph representing web browsing behavior, including receiving web browsing behavior data from one or more web browsers; adding a node on the graph for each web page listed in the web browsing behavior data; adding a first link connecting two or more nodes on the graph, wherein the first link representing a hyperlink for accessing a webpage; calculating an amount of time in which each web page is being accessed; determining a number of units of time in the calculated amount of time; adding one or more virtual nodes to the graph based on the number of units of time; and adding a second link connecting two or more virtual nodes on the graph, wherein the second link representing a virtual hyperlink for accessing a webpage.

Abstract: A method and system for introducing spam into a search engine for testing purposes is provided. An active spam testing system receives from a tester a specification of spam that is to be introduced into the search engine for testing purposes. The testing system may then generate auxiliary data structures for storing indications of the spam that is to be introduced. A search engine has original data structures that may include a content index and a link data structure. The testing system stores the indications of the spam in the auxiliary data structures so that use of the search engine for non-testing purposes is not affected. When the search engine is used for testing purposes, the search engine generates search results based on a combination of the original data structures and the auxiliary data structures.

Abstract: A method and system for determining temporal importance of documents having links between documents based on a temporal analysis of the links is provided. A temporal ranking system collects link information or snapshots indicating the links between documents at various snapshot times. The temporal ranking system calculates a current temporal importance of a document by factoring in the current importance of the document derived from the current snapshot (i.e., with the latest snapshot time) and the historical importance of the document derived from the past snapshots. To calculate the current temporal importance of a web page, the temporal ranking system aggregates the importance of the web page for each snapshot.

Abstract: Methods and systems for identifying black frames within a sequence of frames are provided. In one embodiment, the detection system detects black frames within a sequence of frames by fully decoding base frames and then partially decoding non-black, non-base frames in a way that ensures the blackness of each frame can be determined. The detection system decodes base frames before decoding dependent frames, which is referred to as processing frames in reverse order of dependency since a frame is processed before the frames that depend on it are processed. In another embodiment, the detection system determines the blackness of frames within a sequence of frames by processing the frames in order of their dependency and following chains of block dependency to decode and determine the blackness of blocks.

Abstract: A method and system for determining a ranking of web sites based on an aggregation of rankings of the web pages within the web sites is provided. A ranking system identifies for each web site a stationary distribution of a stochastic complement of the transition probabilities between web pages of the web site. The ranking system then calculates transition probabilities between web sites based on the web page transition probabilities weighted by the stationary distribution of the stochastic complements. The ranking system then calculates the stationary distribution of the transition probabilities of the web sites to represent a ranking of the web sites.

Abstract: A community mining system analyzes objects of different types and relationships between the objects of different types to identify communities. The relationships between the objects have an associated time. The community mining system extracts various features related to objects of a designated type from the relationships between objects of different types that represent the evolution of the features over time. The community mining system collects training data that indicates extracted features associated with members of the communities. The community mining system then classifies an object of the designated type as being within the community based on closeness of the features of the object to the features of the training data.

Abstract: A method and system for generating a search result for a query of hierarchically organized documents based on retrieval of subtrees that are key resources for topic distillation is provided. The retrieval system may identify documents relevant to a query using conventional searching techniques. The retrieval system then calculates a subtree feature for subtrees that have an identified document as their root. After the retrieval system calculates the subtree feature for the subtrees, the retrieval system may generate a subtree relevance score for each subtree based on its subtree feature. The retrieval system may then order the identified documents based on their corresponding subtree relevances.

Abstract: A method and system is provided for calculating importance of documents based on transition probabilities from a source document to a target document based on looking ahead to information content of target documents of the source document. A look-ahead importance system generates transition probabilities of transitioning between any pair of source and target documents based on analysis of links to target documents of the source document. The system may calculate the transition probabilities based on the number of links on documents a look-ahead distance away. The system then solves for the stationary probabilities of the transition probabilities. The stationary probabilities represent the importance of the documents.

Abstract: A method and system for ranking importance of vertices of a directed graph based on links between the vertices and some prior knowledge of importance of the vertices is provided. A ranking system inputs an indication of the vertices along with an indication of the links between the vertices as the directed graph. The ranking system generates a transition-probability matrix that represents the probability of transitioning from vertex to vertex. The ranking system then generates a ranking of the vertices based on the links between the vertices represented by the stationary distribution of the transition-probability matrix that is minimally perturbed to satisfy the prior knowledge, which may be a partial ranking of the vertices.

Abstract: Embodiments for caching and accessing Directed Acyclic Graph (DAG) data to and from a computing device of a DAG distributed execution engine during the processing of an iterative algorithm. In accordance with one embodiment, a method includes processing a first subgraph of the plurality of subgraphs from the distributed storage system in the computing device. The first subgraph being processed with associated input values in the computing device to generate first output values in an iteration. The method further includes storing a second subgraph in a cache of the device. The second subgraph being a duplicate of the first subgraph. Moreover, the method also includes processing the second subgraph with the first output values to generate second output values if the device is to process the first subgraph in each of one or more subsequent iterations.

Abstract: A method and system for generating a ranking function to rank the relevance of documents to a query is provided. The ranking system learns a ranking function from training data that includes queries, resultant documents, and relevance of each document to its query. The ranking system learns a ranking function using the training data by weighting incorrect rankings of relevant documents more heavily than the incorrect rankings of not relevant documents so that more emphasis is placed on correctly ranking relevant documents. The ranking system may also learn a ranking function using the training data by normalizing the contribution of each query to the ranking function so that it is independent of the number of relevant documents of each query.

Abstract: An anti-spam tool works with a web browser to detect spam webpages locally on a client machine. The anti-spam tool can be implemented either as a plug-in module or an integral part of the browser, and manifested as a toolbar. The tool can perform an anti-spam action whenever a webpage is accessed through the browser, and does not require direct involvement of a search engine. A spam detection module installed on the computing device determines whether a webpage being accessed or whether a link contained in the webpage being accessed is spam, by comparing the URL of the webpage or the link with a spam list. The spam list can be downloaded from a remote search engine server, stored locally and updated from time to time. A two-level indexing technique is also introduced to improve the efficiency of the anti-spam tool's use of the spam list.

Abstract: A method and system is provided for calculating importance of documents based on transition probabilities from a source document to a target document based on looking ahead to information content of target documents of the source document. A look-ahead importance system generates transition probabilities of transitioning between any pair of source and target documents based on analysis of links to target documents of the source document. The system may calculate the transition probabilities based on the number of links on documents a look-ahead distance away. The system then solves for the stationary probabilities of the transition probabilities. The stationary probabilities represent the importance of the documents.

Abstract: A method and system for generating a search result for a query of hierarchically organized documents based on retrieval of subtrees that are key resources for topic distillation is provided. The retrieval system may identify documents relevant to a query using conventional searching techniques. The retrieval system then calculates a subtree feature for subtrees that have an identified document as their root. After the retrieval system calculates the subtree feature for the subtrees, the retrieval system may generate a subtree relevance score for each subtree based on its subtree feature. The retrieval system may then order the identified documents based on their corresponding subtree relevances.

Abstract: An anti-spam technique for protecting search engine ranking is based on mining search engine optimization (SEO) forums. The anti-spam technique collects webpages such as SEO forum posts from a list of suspect spam websites, and extracts suspicious link exchange URLs and corresponding link formation from the collected webpages. A search engine ranking penalty is then applied to the suspicious link exchange URLs. The penalty is at least partially determined by the link information associated with the respective suspicious link exchange URL. To detect more suspicious link exchange URLs, the technique may propagate one or more levels from a seed set of suspicious link exchange URLs generated by mining SEO forums.

Abstract: This disclosure describes various exemplary methods, computer program products, and systems for selecting features for ranking in information retrieval. This disclosure describes calculating importance scores for features, measuring similarity scores between two features, selecting features that maximizes total importance scores of the features and minimizes total similarity scores between the features. Also, the disclosure includes selecting features for ranking that solves an optimization problem. Thus, this disclosure identifies relevant features by removing noisy and redundant features and speeds up a process of model training.

Abstract: An importance system calculates the importance of pages using a conditional Markov random walk model rather than a conventional Markov random walk model. The importance system calculates the importance of pages factoring in the importance of sites that contain those pages. The importance system may factor in the importance of sites based on the strength of the correlation of the importance of a page to the importance of a site. The strength of the correlation may be based upon the depth of the page within the site. The importance system may iteratively calculate the importance of the pages using “conditional” transition probabilities. During each iteration, the importance system may recalculate the conditional transition probabilities based on the importance of sites that are derived from the recalculated importance of pages during the iteration.

Abstract: A method and system for determining similarity or correlation between categories of a hierarchical taxonomy for documents by combining heterogeneous similarity metrics is provided. A correlation system uses both a taxonomy distance metric and a term space distance metric to represent the similarity between categories. The correlation system finds a new distance metric for categories that factors in both the taxonomy distance metric and the term space distance metric. The new distance metric can then be used by classifiers to more accurately represent the correlation between categories.

Abstract: Procedures for learning and ranking items in a listwise manner are discussed. A listwise methodology may consider a ranked list, of individual items, as a specific permutation of the items being ranked. In implementations, a listwise loss function may be used in ranking items. A listwise loss function may be a metric which reflects the departure or disorder from an exemplary ranking for one or more sample listwise rankings used in learning. In this manner, the loss function may approximate the exemplary ranking for the plurality of items being ranked.