Abstract: A method and system for autonomously downloading and indexing Hidden Web pages from Websites includes the steps of selecting a query term and issuing a query to a site-specific search interface containing Hidden Web pages. A results index is then acquired and the Hidden Web pages are downloaded from the results index. A plurality of potential query terms are then identified from the downloaded Hidden Web pages. The efficiency of each potential query term is then estimated and a next query term is selected from the plurality of potential query terms, wherein the next selected query term has the greatest efficiency. The next selected query term is then issued to the site-specific search interface using the next query term. The process is repeated until all or most of the Hidden Web pages are discovered.

Abstract: A method and system is provided for autonomously downloading and indexing Hidden Web pages from Websites having site-specific search interfaces. The method may be implemented using a crawler program or the like to autonomously cull Hidden Web content. The method includes the steps of selecting a query term and issuing a query to a site-specific search interface containing Hidden Web pages. A results index is then acquired and the Hidden Web pages are downloaded from the results index. A plurality of potential query terms are then identified from the downloaded Hidden Web pages. The efficiency of each potential query term is then estimated and a next query term is selected from the plurality of potential query terms, wherein the next selected query term has the greatest efficiency. The next selected query term is then issued to the site-specific search interface using the next query term. The process is repeated until all or most of the Hidden Web pages are discovered.

Abstract: The pages in a network of linked pages are ranked based on the quality of the pages. Page quality is obtained by determining the change over time of the link structure of the page, which is obtained by determining the link structure of the page at different periods of time by taking multiple snapshots of the link structure of the network. The link structures are approximated by their PageRanks, page quality being determined by the formula: Q ? ( p ) ? D · ? ? ? ? PR ? ( p ) PR ? ( p ) + PR ? ( p ) where Q(p) is the quality of the page, PR(p) is the current PageRank of the page, ?PR(p) is the change over time in the PageRank of the page, and D is a constant that determines the relative weight of the terms ?PR(p)/PR(p) and PR(p).

Abstract: A system and method for executing a regular expression (regex) query against a large data repository such as the World Wide Web includes an index engine that constructs multigram indices based on regex. A run time then receives a regex query and accesses the indices to return a set of potentially matching pages, which are then efficiently and quickly searched for matches to the regex query.

Abstract: A system and method for executing a regular expression (regex) query against a large data repository such as the World Wide Web includes an index engine that constructs multigram indices based on regex. A run time then receives a regex query and accesses the indices to return a set of potentially matching pages, which are then efficiently and quickly searched for matches to the regex query.

Abstract: A method and apparatus are defined for assigning keywords to media objects in files. The media objects include image, video and audio, for example. Various criteria are used for assigning the keywords, including measuring visual distance, measuring syntactical distance, detecting regular patterns in a table and detecting groups of images.