Abstract: An example of a method includes determining features of a first type for a web page of a plurality of web pages. The method also includes electronically determining a plurality of rules for an attribute of the first web page, wherein the plurality of rules are determined based on features of the first type. The method also includes electronically identifying a first rule, from the plurality of rules, which satisfies a first predefined criterion. The first predefined criteria include at least one of a first threshold for a precision parameter, a second threshold for a support parameter, a third threshold for a distance parameter and a fourth threshold for a recall parameter. The method further includes storing the first rule to enable extraction of value of the attribute from a second web page.

Abstract: A set of clusters associated with a plurality of web pages is received. A first data set and a second data set are generated by applying a first rule and the second rule, respectively, to web pages of a first cluster of the set of clusters. The second rule is substituted for the first rule responsive to having an acceptable extraction accuracy when applied to the first cluster. The extraction accuracy of the second rule is determined by comparing attributes of the second data set to attributes of the first data set.

Abstract: An example of a method includes determining features of a first type for a web page of a plurality of web pages. The method also includes electronically determining a plurality of rules for an attribute of the first web page, wherein the plurality of rules are determined based on features of the first type. The method also includes electronically identifying a first rule, from the plurality of rules, which satisfies a first predefined criterion. The first predefined criteria include at least one of a first threshold for a precision parameter, a second threshold for a support parameter, a third threshold for a distance parameter and a fourth threshold for a recall parameter. The method further includes storing the first rule to enable extraction of value of the attribute from a second web page.

Abstract: An example of a method includes generating an attributed extensible markup language path (XPath) for an annotated entity in a web page. The method further includes determining a first node that satisfy the attributed XPath in the web page and is annotated. The method also includes identifying an attribute property that satisfies predefined criteria in the web page while traversing from the first node to a root node, the attribute property comprising an attribute value and an attribute name. Moreover, the method includes populating the attributed XPath with the attribute property that satisfies predefined criteria. The method also includes filtering the attributed XPath to generate a robust XPath, and extracting content from multiple web pages based on the robust XPath.

Abstract: Embodiments of methods, apparatuses, or systems relating to identifying previously annotated web page information are disclosed.

Abstract: Techniques for high precision multi entity extraction are provided. A wrapper that represents a generalized structure of a set of training web pages is accessed. The wrapper includes one or more annotations that indicate a set of attributes that are included in each of a plurality of records. Record boundaries are determined based on nodes included in the wrapper, where the record boundaries delimit the plurality of records within any training page of the set of training web pages. The wrapper is modified to include one or more boundary nodes, where the one or more boundary nodes indicate the record boundaries of the plurality of records within the set of training web pages. Multiple records are extracted from a web page, where extracting the multiple records comprises detecting record completions based at least on the wrapper and on a document object model (DOM) representation of the web page.

Abstract: A template or wrapper tree for a document such as a web page is generalized from the bottom up (from leaf toward root of a logical tree structure of the template). At a given level in the tree, sub-trees are clustered and the clustered sub-trees are generalized, and the process is repeated at a next higher level in the tree, resulting in a generalized template or wrapper tree. This can be done by generating a nested pattern regular expression based on the sub-tree clusters, merging sub-trees based on the nested pattern regular expression, and then replacing sub-trees in a tree-based regular expression of the template or wrapper at the given level with the merged sub-trees. This process is repeated at a next higher level of the tree (progressing from leaf towards root) until the wrapper or tree-based regular expression that represents the template is fully generalized.

Abstract: A template or wrapper tree for a document such as a web page is generalized from the bottom up (from leaf toward root of a logical tree structure of the template). At a given level in the tree, sub-trees are clustered and the clustered sub-trees are generalized, and the process is repeated at a next higher level in the tree, resulting in a generalized template or wrapper tree. This can be done by generating a nested pattern regular expression based on the sub-tree clusters, merging sub-trees based on the nested pattern regular expression, and then replacing sub-trees in a tree-based regular expression of the template or wrapper at the given level with the merged sub-trees. This process is repeated at a next higher level of the tree (progressing from leaf towards root) until the wrapper or tree-based regular expression that represents the template is fully generalized.

Abstract: A template or wrapper tree for a document such as a web page is generalized from the bottom up (from leaf toward root of a logical tree structure of the template). At a given level in the tree, sub-trees are clustered and the clustered sub-trees are generalized, and the process is repeated at a next higher level in the tree, resulting in a generalized template or wrapper tree. This can be done by generating a nested pattern regular expression based on the sub-tree clusters, merging sub-trees based on the nested pattern regular expression, and then replacing sub-trees in a tree-based regular expression of the template or wrapper at the given level with the merged subtrees. This process is repeated at a next higher level of the tree (progressing from leaf towards root) until the wrapper or tree-based regular expression that represents the template is fully generalized.

Abstract: Techniques are disclosed herein for extracting attributes from documents such as web pages. A structure of a training document is compared with a structure of a template to determine a template-node that structurally corresponds to a training-document node that has been annotated with an attribute. Filters can be learned by analyzing characteristics that the attribute possesses in the training document. To extract information for the attribute from a new document, first a set of candidate nodes in a new document are determined by determining which nodes in the new document structurally map to the template node. The filters are applied to eliminate false positives from the candidate nodes. Information can then be extracted from the new document, based on remaining candidate nodes. Even if incremental changes are made to the structure of new documents, nodes that posses the attributes can still be reliably identified.