Abstract: A system, apparatus, and method are disclosed to provide an enhanced search index for searchable content by limited capability devices, such as mobile phones. A URL analyzer determines clusters of mobile web page URLs and corresponding tokens. A landing page identifier evaluates cluster URLs and the tokens to identify landing page URLs for mobile web pages likely to be relevant to an associated search term. A mapper associates the landing page URLs with related domain hosts that include conventional web pages generally not accessible to limited capability devices. A category tagger associates the landing page URLs with categories of conventional web pages that share a content topic. The landing page URLs, and corresponding tokens, domain host information, and categories are added to the search index. A submitted search term is evaluated relative to the tokens, domain host information, and categories to determine relevant mobile web landing pages as search results.

Abstract: A system, apparatus, and method are disclosed for providing a single source for searchable content by mobile devices. Mobile user agents crawl host sites to determine whether there is content that may be viewed by any of the mobile user agents. If files are identified as viewable for a given mobile user agent, the site is tagged as crawlable. A best mobile user agent may also be identified for that site, based on number of files viewable, response time, file mobileness, or the like. The sites may be discovered by instrumentation of URLs with mobile device keywords, performing a backlink analysis to discover interlinking of mobile device related sites, exploring URLs of a known host name for potential starting points for mobile device sites, or employing a list of known popular conventional sites with a mobile user agent for possible redirections to a potential mobile device web site.

Abstract: An automated form filler and script executor is integrated with a web browser engine, which is communicatively coupled to a web crawler, thereby enabling the crawler to identify dynamic web content based on submission of forms completed by the form filler. The crawler is capable of identifying web pages containing forms that require submission, and JavaScript code that requires execution, respectively, for requesting dynamic web content from a server. The form filler systematically completes the form based on various combinations of search parameter values provided by the web page for requesting dynamic content. Duplicate forms are detected, so that the crawler does not unnecessarily re-process forms that are similar to forms that have already been processed. The crawler may also determine which JavaScript links in a page are relevant for execution, so as to avoid unnecessary execution of irrelevant JavaScript links.

Abstract: An automated form filler and script executor is integrated with a web browser engine, which is communicatively coupled to a web crawler, thereby enabling the crawler to identify dynamic web content based on submission of forms completed by the form filler. The crawler is capable of identifying web pages containing forms that require submission, and JavaScript code that requires execution, respectively, for requesting dynamic web content from a server. The form filler systematically completes the form based on various combinations of search parameter values provided by the web page for requesting dynamic content. Duplicate forms are detected, so that the crawler does not unnecessarily re-process forms that are similar to forms that have already been processed. The crawler may also determine which JavaScript links in a page are relevant for execution, so as to avoid unnecessary execution of irrelevant JavaScript links.

Abstract: An automated form filler and script executor is integrated with a web browser engine, which is communicatively coupled to a web crawler, thereby enabling the crawler to identify dynamic web content based on submission of forms completed by the form filler. The crawler is capable of identifying web pages containing forms that require submission, and JavaScript code that requires execution, respectively, for requesting dynamic web content from a server. The crawler passes a representation of such web pages to the browser engine. The form filler systematically completes the form based on various combinations of search parameter values provided by the web page for requesting dynamic content. Request messages are constructed by the browser engine and passed to the crawler for submission to the server. The dynamic content, received by the crawler from the server in response to the request, can be indexed according to conventional search engine indexing techniques.

Abstract: Automatically determining values for fields in an electronic document. In one embodiment, an intelligent form filler automatically fills in at least some of the fields based on as set of rules associated with a domain. A particular set of domain rules may have class definitions that define how to classify a field for that domain and group definitions that define how to group fields. The domain rules also describe how values can be determined for the fields, based on the classifications, groupings, and other factors. In one embodiment, the intelligent form filler submits more than one form such that different combinations of values are submitted. The values that were used to fill in the form(s) may be provided to an extraction tool, which use the values to facilitate extraction of information from a document returned in response to submitting the form.

Abstract: A system, apparatus, and method are disclosed to identify and tag documents that are mobile documents in that they are compatible with limited capability devices, such as mobile phones. A website hosting a document is checked to determine whether the website considers the document to be a mobile document. The document is also for indications that the document is a mobile document. The indications include a content type, a document type, and/or markup tags that are consistent with a mobile document. A URL for the document is also checked for parameters indicating a mobile document. The above information is used to determine one or more categories of mobile devices that could display or otherwise process the document. A confidence level is determined indicating a degree of confidence that the document is a mobile document. The information is used for searching documents for those that are likely to be mobile documents.

Abstract: A system, apparatus, and method are disclosed to provide an enhanced search index for searchable content by limited capability devices, such as mobile phones. A URL analyzer determines clusters of mobile web page URLs and corresponding tokens. A landing page identifier evaluates cluster URLs and the tokens to identify landing page URLs for mobile web pages likely to be relevant to an associated search term. A mapper associates the landing page URLs with related domain hosts that include conventional web pages generally not accessible to limited capability devices. A category tagger associates the landing page URLs with categories of conventional web pages that share a content topic. The landing page URLs, and corresponding tokens, domain host information, and categories are added to the search index. A submitted search term is evaluated relative to the tokens, domain host information, and categories to determine relevant mobile web landing pages as search results.

Abstract: A system, apparatus, and method are disclosed for providing a single source for searchable content by mobile devices. Mobile user agents crawl host sites to determine whether there is content that may be viewed by any of the mobile user agents. If files are identified as viewable for a given mobile user agent, the site is tagged as crawlable. A best mobile user agent may also be identified for that site, based on number of files viewable, response time, file mobileness, or the like. The sites may be discovered by instrumentation of URLs with mobile device keywords, performing a backlink analysis to discover interlinking of mobile device related sites, exploring URLs of a known host name for potential starting points for mobile device sites, or employing a list of known popular conventional sites with a mobile user agent for possible redirections to a potential mobile device web site.