Abstract: A link referral system includes a classification arrangement for classifying Web pages that the system retrieves over the World Wide Web. In assigning a link in a Web page to a particular classification, the classification arrangement originally establishes the classification using the name for the particular section of the Web page in which the link is located. Using statistical analysis of the occurrence of related links within sections of a number of diverse Web pages, some of which are named sections, the system creates classifications of links under both named classifications as well as anonymous classifications of links that co-occur with a high degree of probability. To minimize errors in the statistical analysis, the classification arrangement determines whether different World Wide Web addresses (so-called "URL's) which are used to access the Web pages in fact identify the same Web page.

Abstract: A system receives an information stream comprising the textual information whose semantic content is to be determined, divides the information stream into a series of elements and classifies each element into one of a plurality of predetermined classifications. The system includes a token generator and a token classifier. The token generator receives the textual information stream, parses the stream to identifies the respective elements, identifies for each element one of a plurality of element types, and generates a token identifying the element type for each element. At least some of the tokens also include a pointer pointing to the actual information associated with the element. The token classifier receives the tokens and classifies them in order.