Abstract: A grid-based approach for enterprise-scale data mining that leverages database technology for I/O parallelism and on-demand compute servers for compute parallelism in the statistical computations is described. By enterprise-scale, we mean the highly-automated use of data mining in vertical business applications, where the data is stored on one or more relational database systems, and where a distributed architecture comprising of high-performance compute servers or a network of low-cost, commodity processors, is used to improve application performance, provide better quality data mining models, and for overall workload management. The approach relies on an algorithmic decomposition of the data mining kernel on the data and compute grids, which provides a simple way to exploit the parallelism on the respective grids, while minimizing the data transfer between them.

Abstract: A new method is used to model the class probability from data that is based on a novel multiplicative adjustment of the class probability by a plurality of items of evidence induced from training data. The optimal adjustment factors from each item of evidence can be determined by several techniques, a preferred embodiment thereof being the method of maximum likelihood. The evidence induced from the data can be any function of the feature variables, the simplest of which are the individual feature variables themselves. The adjustment factor of an item of evidence Ej is given by the ratio of the conditional probability P(C|Ej) of the class C given Ej to the prior class probability P(C), exponentiated by a parameter aj. The method provides a new and useful way to aggregate probabilistic evidence so that the final model output exhibits a low error rate for classification, and also gives a superior lift curve when distinguishing between any one class and the remaining classes.

Abstract: A new method is used to model the class probability from data that is based on a novel multiplicative adjustment of the class probability by a plurality of items of evidence induced from training data. The optimal adjustment factors from each item of evidence can be determined by several techniques, a preferred embodiment thereof being the method of maximum likelihood. The evidence induced from the data can be any function of the feature variables, the simplest of which are the individual feature variables themselves. The adjustment factor of an item of evidence E1 is given by the ratio of the conditional probability P(C|E1) of the class C given E1 to the prior class probability P(C), exponentiated by a parameter a1. The method provides a new and useful way to aggregate probabilistic evidence so that the final model Output exhibits a low error rate for classification, and also gives a superior lift curve when distinguishing between any one class and the remaining classes.

Abstract: The present invention generally relates to computer databases and, more particularly, to data mining and knowledge discovery. The invention specifically relates to a method for constructing segmentation-based predictive models, such as decision-tree classifiers, wherein data records are partitioned into a plurality of segments and separate predictive models are constructed for each segment. The present invention contemplates a computerized method for automatically building segmentation-based predictive models that substantially improves upon the modeling capabilities of decision trees and related technologies, and that automatically produces models that are competitive with, if not better than, those produced by data analysts and applied statisticians using traditional, labor-intensive statistical techniques. The invention achieves these properties by performing segmentation and multivariate statistical modeling within each segment simultaneously.

Abstract: When a customer is in the process of filling a market basket for purchase on an Internet commerce site, a method makes prioritized recommendation of items so as to maximize the likelihood that the customer will add to the basket those items that are in the list with higher priorities. The method separately considers in turn preferences due to a current set of items in the market basket and also preferences due to a new choice independent of what is in the market basket. In this way, the method recognizes that not all items in the market basket are selected because of their affinity with some other item already in the basket. The two preferences are estimated separately from training data and combined in proper proportions to obtain an overall preference for item not yet in the market basket.

Abstract: The computational cost of many statistical modeling algorithms is affected by the input/output (I/O) cost of accessing out-of-core training data. This is an important challenge for emerging data mining applications, where the amount of training data can be potentially enormous. A heuristic approach to this problem is described. This approach is based on constructing a simple probability model from the large training data set, and using this model to generate simulated pseudo data for some aspects of the statistical modeling procedure. This approach is illustrated in the context of building a Naive Bayes probability model with feature selection. Here, the usual algorithms would require numerous data scans over the massive training data set, but our heuristic obtains models of comparable accuracy with just two data scans.