Abstract: A method for optimizing the construction of a KStore includes providing a data set having a plurality of records with a plurality of fields, providing an optimization structure from a sample of the data set and analyzing the optimization structure to determine a field order for the records contained in the data set. A KStore is provided from the data set in accordance with the field order. The sample of the data set can include the entire data set, a random sample of the data set or a predetermined number of records from the data set. The data set has an initial order having earlier records and later records and the sample can include a plurality of the earlier records. The sample of the data set includes a user determined number of records from the data set. The optimization structure can be a KStore.

Abstract: In learning for an interlocking trees datastore or KStore, the process is made more efficient by noting the (n-level) address within the KStore during the learning of each particle. In a pre-particle stream of data, which may be organized within or before the Learn Engine prior to this, “Marks” and “References” are inserted. Each Mark identifies where any number of References may start the learning process, enabling the avoidance of re-learning redundant data. Thus, in a field record data set, the redundant data fields (or even partial fields) can be skipped over and only the new data learned. The Marks and References are removed before processing into a particle stream. When particles are learned the K Engine returns the n-level address or pointer(s) which the Learn Engine uses to associate with the relevant Reference(s). The system can be implemented in hardware if desired to speed processing. No limit to the distribution or numbers of KStores, Learn Engines being used or K Engines being used is indicated.