Abstract: The subject invention provides for a feedback loop system and method that facilitate classifying items in connection with spam prevention in server and/or client-based architectures. The invention makes uses of a machine-learning approach as applied to spam filters, and in particular, randomly samples incoming email messages so that examples of both legitimate and junk/spam mail are obtained to generate sets of training data. Users which are identified as spam-fighters are asked to vote on whether a selection of their incoming email messages is individually either legitimate mail or junk mail. A database stores the properties for each mail and voting transaction such as user information, message properties and content summary, and polling results for each message to generate training data for machine learning systems. The machine learning systems facilitate creating improved spam filter(s) that are trained to recognize both legitimate mail and spam mail and to distinguish between them.

Abstract: The present invention relates to a system and methodology to facilitate automated error correction of user input data via an analysis of the input data in accordance with an automatically generated and filtered database of processed structural groupings or formulations selected and filtered from past user activities. The filtered database provides a relevant foundation of potential phrases, topics, symbols, speech and/or colloquial structures of interest to users—which are automatically determined from previous user activity, and employed to facilitate automated error checking in accordance with the user's current input, command and/or request for information.

Abstract: Distribution displays for categories are provided which illuminate the distribution of continuous attributes over all cases in a category, and which provide a histogram of the population of the different states of categorical attributes. An array of such displays by attribute (in one dimension) and category (in another dimension) may be provided. Category diagram displays are also provided for visualizing the different categories, and their distributions, populations, and similarities. These are displayed through different shading of nodes and edges representing categories and the relationship between two categories, and through proximity of nodes.

Abstract: The subject invention provides for an intelligent quarantining system and method that facilitates detecting and preventing spam. In particular, the invention employs a machine learning filter specifically trained using origination features such as an IP address as well as destination feature such as a URL. Moreover, the system and method involve training a plurality of filters using specific feature data for each filter. The filters are trained independently each other, thus one feature may not unduly influence another feature in determining whether a message is spam. Because multiple filters are trained and available to scan messages either individually or in combination (at least two filters), the filtering or spam detection process can be generalized to new messages having slightly modified features (e.g., IP address). The invention also involves locating the appropriate IP addresses or URLs in a message as well as guiding filters to weigh origination or destination features more than text-based features.

Abstract: The subject invention provides for an advanced and robust system and method that facilitates detecting spam. The system and method include components as well as other operations which enhance or promote finding characteristics that are difficult or the spammer to avoid and finding characteristics in non-spam that are difficult for spammers to duplicate. Exemplary characteristics include examining origination features in pairs analyzing character and/or number sequences, strings, and sub-strings, detecting various entropy levels of one or more character sequences, strings and/or sub-strings as well as analyzing message and/or feature sizes.

Abstract: Architecture for detecting and removing obfuscating clutter from the subject and/or body of a message, e.g., e-mail, prior to filtering of the message, to identify junk messages commonly referred to as spam. The technique utilizes the powerful features built into an HTML rendering engine to strip the HTML instructions for all non-substantive aspects of the message. Pre-processing includes pre-rendering of the message into a final format, which final format is that which is displayed by the rendering engine to the user. The final format message is then converted to a text-only format to remove graphics, color, non-text decoration, and spacing that cannot be rendered as ASCII-style or Unicode-style characters. The result is essentially to reduce each message to its common denominator essentials so that the junk mail filter can view each message on an equal basis.

Abstract: The subject invention provides for an intelligent quarantining system and method that facilitates a more robust classification system in connection with spam prevention. The invention involves holding back some messages that appear to be questionable, suspicious, or untrustworthy from classification (as spam or good). In particular, the filter lacks information about these messages and thus classification is temporarily delayed. This provides more time for a filter update to arrive with a more accurate classification. The suspicious messages can be quarantined for a determined time period to allow more data to be collected regarding these messages. A number of factors can be employed to determine whether messages are more likely to be flagged for further analysis. User feedback by way of a feedback loop system can also be utilized to facilitate classification of the messages. After some time period, classification of the messages can be resumed.

Abstract: The present invention provides for generating inputs that can be provided to a message classification module to facilitate more reliable classification of electronic messages, such as, for example, as unwanted and/or unsolicited. In one embodiment, a sending messaging server provides an appropriate response to address verification data thereby indicating a reduced likelihood of the sending messaging server using a forged network address. In another embodiment, it is determined if a messaging server is authorized to send electronic messages for a domain. In yet another embodiment, electronic message transmission policies adhered to by a domain are identified. In yet a further embodiment, a sending computer system expends computational resources to solve a computational puzzle and includes an answer document in an electronic message. A receiving computer system receives the electronic message and verifies the answer document.

Abstract: The present invention provides for generating inputs that can be provided to a message classification module to facilitate more reliable classification of electronic messages, such as, for example, as unwanted and/or unsolicited. In one embodiment, a sending messaging server provides an appropriate response to address verification data thereby indicating a reduced likelihood of the sending messaging server using a forged network address. In another embodiment, it is determined if a messaging server is authorized to send electronic messages for a domain. In yet another embodiment, electronic message transmission policies adhered to by a domain are identified. In yet a further embodiment, a sending computer system expends computational resources to solve a computational puzzle and includes an answer document in an electronic message. A receiving computer system receives the electronic message and verifies the answer document.

Abstract: The subject invention provides for a feedback loop system and method that facilitate classifying items in connection with spam prevention in server and/or client-based architectures. The invention makes uses of a machine-learning approach as applied to spam filters, and in particular, randomly samples incoming email messages so that examples of both legitimate and junk/spam mail are obtained to generate sets of training data. Users which are identified as spam-fighters are asked to vote on whether a selection of their incoming email messages is individually either legitimate mail or junk mail. A database stores the properties for each mail and voting transaction such as user information, message properties and content summary, and polling results for each message to generate training data for machine learning systems. The machine learning systems facilitate creating improved spam filter(s) that are trained to recognize both legitimate mail and spam mail and to distinguish between them.

Abstract: The invention relates to a system for filtering messages—the system includes a seed filter having associated therewith a false positive rate and a false negative rate. A new filter is also provided for filtering the messages, the new filter is evaluated according to the false positive rate and the false negative rate of the seed filter, the data used to determine the false positive rate and the false negative rate of the seed filter are utilized to determine a new false positive rate and a new false negative rate of the new filter as a function of threshold. The new filter is employed in lieu of the seed filter if a threshold exists for the new filter such that the new false positive rate and new false negative rate are together considered better than the false positive and the false negative rate of the seed filter.

Abstract: A model-based system and method for global optimization that utilizes quantum mechanics in order to approximate the global minimum of a given problem (e.g., mathematical function). A quantum mechanical particle with a sufficiently large mass has a ground state solution to the Schrödinger Equation which is localized to the global minimum of the energy field, or potential, it experiences. A given function is modeled as a potential, and a quantum mechanical particle with a sufficiently large mass is placed in the potential. The ground state of the particle is determined, and the probability density function of the ground state of the particle is calculated. The peak of the probability density function is localized to the global minimum of the potential.

Abstract: Disclosed is a system for approximating conditional probabilities using an annotated decision tree where predictor values that did not exist in training data for the system are tracked, stored, and referenced to determine if statistical aggregation should be invoked. Further disclosed is a system for storing statistics for deriving a non-leaf probability corresponding to predictor values, and a system for aggregating such statistics to approximate conditional probabilities.

Abstract: The present invention relates to a system and methodology to facilitate automated error correction of user input data via an analysis of the input data in accordance with an automatically generated and filtered database of processed structural groupings or formulations selected and filtered from past user activities. The filtered database provides a relevant foundation of potential phrases, topics, symbols, speech and/or colloquial structures of interest to users—which are automatically determined from previous user activity, and employed to facilitate automated error checking in accordance with the user's current input, command and/or request for information.

Abstract: A system and method facilitating detection of unsolicited e-mail message(s) with challenges is provided. The invention includes an e-mail component and a challenge component. The system can receive e-mail message(s) and associated probabilities that the e-mail message(s) are spam. Based, at least in part, upon the associated probability, the system can send a challenge to a sender of an e-mail message. The challenge can be an embedded code, computational challenge, human challenge and/or micropayment request. Based, at least in part, upon a response to the challenge (or lack of response), the challenge component can modify the associated probability and/or delete the e-mail message.

Abstract: Distribution displays for categories are provided which illuminate the distribution of continuous attributes over all cases in a category, and which provide a histogram of the population of the different states of categorical attributes. An array of such displays by attribute (in one dimension) and category (in another dimension) may be provided. Category diagram displays are also provided for visualizing the different categories, and their distributions, populations, and similarities. These are displayed through different shading of nodes and edges representing categories and the relationship between two categories, and through proximity of nodes.

Abstract: Determination as to whether a variable is numeric or non-numeric. In one embodiment, a variable is input having a plurality of values, where each value has a count. The variable is determined to be numeric or non-numeric by assessing closeness of counts for adjacent values of the variable. Whether the variable is numeric or non-numeric is then output.

Abstract: Visualization of high-dimensional data sets is disclosed, particularly the display of a network model for a data set. The network, such as a dependency or a Bayesian network, has a number of nodes having dependencies thereamong. The network can be displayed items and connections, corresponding to nodes and dependencies, respectively. Selection of a particular item in one embodiment results in the display of the local distribution associated with the node for the item. In one embodiment, only a predetermined number of the items are shown, such as only the items representing the most popular nodes. Furthermore, in one embodiment, in response to receiving a user input, a sub-set of the connections is displayed, proportional to the user input.

Abstract: An architecture for automated data analysis. In one embodiment, a computerized system comprising an automated problem formulation layer, a first learning engine, and a second learning engine. The automated problem formulation layer receives a data set. The data set has a plurality of records, where each record has a value for each of a plurality of raw transactional variables. The layer abstracts the raw transactional variables into cooked transactional variables. The first learning engine generates a model for the cooked transactional variables, while the second learning engine generates a model for the raw transactional variables.

Abstract: A method and a system for abstracting cooked variables from raw variables. In one embodiment, a data set that has a plurality of records is input into a system, where each record has a value for each of a plurality of raw transactional variables. These variables are organized into a hierarchy of nodes. The raw transactional variables are abstracted into a lesser number of cooked transactional variables, and the cooked transactional variables are output.