Abstract: A real-time communication architecture establishes a continuous connection between an enterprise network and a communication management system. The connection is continuously held open allowing mobile devices real-time access to enterprise email systems. The real-time communication architecture can support an entire enterprise email system or individual email users. The foregoing and other objects, features and advantages of the invention will become more readily apparent from the following detailed description of a preferred embodiment of the invention which proceeds with reference to the accompanying drawings.

Abstract: A method for performing activities on a web site is disclosed. A user's browsing activities on a web site are captured. The user's browsing activities includes affixing labels to web pages and filling out forms. The captured activities are analyzed for patterns. Rules for performing activities on a web site are generated based on the patterns. Further activities are performed on the web site according to the rules and content from the web site is fetched. The fetched content is used in various web service applications, including crawlers.

Abstract: Automated crawling of page links associated with a site domain that was previously crawled involves computing the dynamicity of a site based on totals of continuous dead links, live links and/or prerequisite pages encountered while crawling page links corresponding to the site. The degree to which links are crawled is optimized based on the dynamicity of the site. Some pages require that another particular page (i.e., a prerequisite page) is retrieved from the host prior to retrieving a given page, e.g., so that the prerequisite page can set a cookie. Prerequisite pages are determined based on stored information about pages that were retrieved, during a previous crawl, prior to retrieving a page. Prerequisite pages are identified to a search system so that when a user clicks on the URL for the page, the request is redirected to the prerequisite page to set the cookie appropriately.

Abstract: An automated form filler and script executor is integrated with a web browser engine, which is communicatively coupled to a web crawler, thereby enabling the crawler to identify dynamic web content based on submission of forms completed by the form filler. The crawler is capable of identifying web pages containing forms that require submission, and JavaScript code that requires execution, respectively, for requesting dynamic web content from a server. The form filler systematically completes the form based on various combinations of search parameter values provided by the web page for requesting dynamic content. Duplicate forms are detected, so that the crawler does not unnecessarily re-process forms that are similar to forms that have already been processed. The crawler may also determine which JavaScript links in a page are relevant for execution, so as to avoid unnecessary execution of irrelevant JavaScript links.

Abstract: An automated form filler and script executor is integrated with a web browser engine, which is communicatively coupled to a web crawler, thereby enabling the crawler to identify dynamic web content based on submission of forms completed by the form filler. The crawler is capable of identifying web pages containing forms that require submission, and JavaScript code that requires execution, respectively, for requesting dynamic web content from a server. The crawler passes a representation of such web pages to the browser engine. The form filler systematically completes the form based on various combinations of search parameter values provided by the web page for requesting dynamic content. Request messages are constructed by the browser engine and passed to the crawler for submission to the server. The dynamic content, received by the crawler from the server in response to the request, can be indexed according to conventional search engine indexing techniques.

Abstract: Techniques for correcting miscategorized features excerpted from web pages are provided. For each of several categories and several pages on a particular web site, a separate feature may be excerpted from that page and associated with that page in relation to that category. Often, many of the “high confidence” features that have been associated with the same category are found to be associated with similar characteristics regardless of the pages from which those features were excerpted. Thus, a set of category characteristics, which are often found associated with the “high confidence” features in a particular category, may be determined. For each page, a candidate feature that is associated with the set of category characteristics may be identified in that page. If, in relation to the particular category, a feature other than the candidate feature is associated with that page, then that other feature may be replaced by the candidate feature.

Abstract: A method for providing feedback to a web crawler is provided. A content processor determines whether a crawled web page is useful to an application. This determination is passed to a learning module. The learning module analyzes crawled web pages and the determinations of usefulness made by the content processor and generates rules for crawling more useful web pages and less non-useful web pages. The learning module provides these rules to the crawler, which applies them in making crawling decisions. Rules expire after a period of time. Paths from a web site's main web page to useful web pages are preserved. A token tree is constructed to facilitate the generation of rules.

Abstract: A method for performing activities on a web site is disclosed. A user's browsing activities on a web site are captured. The user's browsing activities includes affixing labels to web pages and filling out forms. The captured activities are analyzed for patterns. Rules for performing activities on a web site are generated based on the patterns. Further activities are performed on the web site according to the rules and content from the web site is fetched. The fetched content is used in various web service applications, including crawlers.

Abstract: Automatically determining values for fields in an electronic document. In one embodiment, an intelligent form filler automatically fills in at least some of the fields based on as set of rules associated with a domain. A particular set of domain rules may have class definitions that define how to classify a field for that domain and group definitions that define how to group fields. The domain rules also describe how values can be determined for the fields, based on the classifications, groupings, and other factors. In one embodiment, the intelligent form filler submits more than one form such that different combinations of values are submitted. The values that were used to fill in the form(s) may be provided to an extraction tool, which use the values to facilitate extraction of information from a document returned in response to submitting the form.

Abstract: Techniques for correcting miscategorized features excerpted from web pages are provided. For each of several categories and several pages on a particular web site, a separate feature may be excerpted from that page and associated with that page in relation to that category. Often, many of the “high confidence” features that have been associated with the same category are found to be associated with similar characteristics regardless of the pages from which those features were excerpted. Thus, a set of category characteristics, which are often found associated with the “high confidence” features in a particular category, may be determined. For each page, a candidate feature that is associated with the set of category characteristics may be identified in that page. If, in relation to the particular category, a feature other than the candidate feature is associated with that page, then that other feature may be replaced by the candidate feature.

Abstract: A real-time communication architecture establishes a continuous connection between an enterprise network and a communication management system. The connection is continuously held open allowing mobile devices real-time access to enterprise email systems. The real-time communication architecture can support an entire enterprise email system or individual email users. The foregoing and other objects, features and advantages of the invention will become more readily apparent from the following detailed description of a preferred embodiment of the invention which proceeds with reference to the accompanying drawings.

Abstract: Automated crawling of page links associated with a site domain that was previously crawled involves computing the dynamicity of a site based on totals of continuous dead links, live links and/or prerequisite pages encountered while crawling page links corresponding to the site. The degree to which links are crawled is optimized based on the dynamicity of the site. Some pages require that another particular page (i.e., a prerequisite page) is retrieved from the host prior to retrieving a given page, e.g., so that the prerequisite page can set a cookie. Prerequisite pages are determined based on stored information about pages that were retrieved, during a previous crawl, prior to retrieving a page. Prerequisite pages are identified to a search system so that when a user clicks on the URL for the page, the request is redirected to the prerequisite page to set the cookie appropriately.

Abstract: A real-time communication architecture establishes a continuous connection between an enterprise network and a communication management system. The connection is continuously held open allowing mobile devices real-time access to enterprise email systems. The real-time communication architecture can support an entire enterprise email system or individual email users. The foregoing and other objects, features and advantages of the invention will become more readily apparent from the following detailed description of a preferred embodiment of the invention which proceeds with reference to the accompanying drawings.

Abstract: A real-time communication architecture establishes a continuous connection between an enterprise network and a communication management system. The connection is continuously held open allowing mobile devices real-time access to enterprise email systems. The real-time communication architecture can support an entire enterprise email system or individual email users. The foregoing and other objects, features and advantages of the invention will become more readily apparent from the following detailed description of a preferred embodiment of the invention which proceeds with reference to the accompanying drawings.