Abstract: A method for loading data into a datamart. The method includes identifying business data in a first format from a real business process and identifying a desired second format. The method includes designing a transformation algorithm that transforms the business data in the first format into the second format and implementing the transformation algorithm in computer executable code. The method includes running the computer executable code on business data in the first format and generating business data in the second format and storing the business data in the second format in a datamart.

Abstract: A method for processing semi-structured data. The method includes receiving semi-structured data into a first format from a real business process. Preferably, the semi-structured data are machine generated. The method includes tokenizing the semi-structured data into a second format and storing the semi-structured data in the second format into one or more memories and clustering the tokenized data to form a plurality of clusters. The method also includes identifying a selected low frequency term in each of the clusters, and processing at least two of the clusters and the associated selected low frequency terms to form a single template for the at least two of the clusters. In a preferred embodiment, the method replaces the selected low frequency term with a wild card character.

Abstract: A method for estimating the performance of a statistical classifier. The method includes inputting a first set of business data in a first format from a real business process and storing the first set of business data in the first format into memory. The method applying a statistical classifier to the first set of business data and recording its classification decisions and obtaining a labeling that contains the correct decision for each data item. The method includes computing a weight for each data item that reflects its true frequency and computing a performance measure of the statistical classifier based on the weights that reflect true frequency. The method also displays the performance measure to a user.

Abstract: A method for detecting change in business data using a statistical classifier process. The method includes inputting a first set of business data in a first format from a real business process from a first data source and storing the first set of business data into one or more memories. The method also includes inputting a second set of business data in a second format from a real business process from a second data source and storing the second set of business data into one or more memories. The method forms a statistical classifier by inputting the first set of business data into a learning process associating with the statistical classifier that processes business the data in the first format. The method stores the classifier into the one or more memories, the classifier being associated with the first set of data in the first format and processes the data from the first data source in the statistical classifier to derive a first result.

Abstract: A method for processing semi-structured data. The method includes receiving semi-structured data into a first format from a real business process. Preferably, the semi-structured data are machine generated. The method includes tokenizing the semi-structured data into a second format and storing the semi-structured data in the second format into one or more memories and clustering the tokenized data to form a plurality of clusters. The method also includes identifying a selected low frequency term in each of the clusters, and processing at least two of the clusters and the associated selected low frequency terms to form a single template for the at least two of the clusters. In a preferred embodiment, the method replaces the selected low frequency term with a wild card character.

Abstract: A method for loading data into a datamart. The method includes identifying business data in a first format from a real business process and identifying a desired second format. The method includes designing a transformation algorithm that transforms the business data in the first format into the second format and implementing the transformation algorithm in computer executable code. The method includes running the computer executable code on business data in the first format and generating business data in the second format and storing the business data in the second format in a datamart.

Abstract: A system and method for the efficient creation of training data for automatic classification.

Abstract: A method for estimating the performance of a statistical classifier. The method includes inputting a first set of business data in a first format from a real business process and storing the first set of business data in the first format into memory. The method applying a statistical classifier to the first set of business data and recording its classification decisions and obtaining a labeling that contains the correct decision for each data item. The method includes computing a weight for each data item that reflects its true frequency and computing a performance measure of the statistical classifier based on the weights that reflect true frequency. The method also displays the performance measure to a user.