Abstract: A method is disclosed that places data-intensive subprocesses in close physical and logical proximity to the facility responsible for storing the data, so that high efficiencies at reduced cost are achieved. In one specific example, new computer programs, termed adjuncts, are added and placed in a logical partition on a storage facility so that they can be invoked using appropriate commands issued on the I/O channel. Further, programs or changes are added to existing programs on the host machine, wherein such programs or changes discover the function extensions and invoke them to perform data processing.

Abstract: A method, apparatus and program storage device readable by a computer tangibly embodying a program of instructions executable by the computer is provided for reorganization of database data. The computer database reorganization method reorganizes one set of database data blocks at a time, allowing concurrent data manipulation. Method identifies a set of data blocks for reorganization in a sliding peephole mode, re-orders the set of data blocks and replaces the original set of data blocks with the re-ordered set of data blocks. The method include an overlapping peephole method, which chooses, for each set of data block to be reorganized, a next succeeding set of data blocks plus an overlap segment, wherein the overlap segment includes a set of empty pages other than intentionally specified free pages, and the overlap segment is a subset of the preceding set of data blocks.

Abstract: A method is disclosed that places data-intensive subprocesses in close physical and logical proximity to the facility responsible for storing the data, so that high efficiencies at reduced cost are achieved. In one specific example, new computer programs, termed adjuncts, are added and placed in a logical partition on a storage facility so that they can be invoked using appropriate commands issued on the I/O channel. Further, programs or changes are added to existing programs on the host machine, wherein such programs or changes discover the function extensions and invoke them to perform data processing.

Abstract: A method, apparatus and article of manufacture is provided for loading data from a remote data source record by record, in a computer system network connecting a source site and a target site via a database connection communication line. The source site has at least one data source and a Distributed Relational Database Architecture (DRDA) software server having multi-database access to DBMSs. The target site requests data loading from the source site DRDA via a block of Structured Query Language (SQL) statements. Data are transported record by record via the database connection communication line according to the DRDA communication protocol, wherein the target site loads records concurrently with the unloading of records in the source site. The data loading may also be performed in a pipeline manner, loading data records in multiple partitions with a plurality of parallel streams, pointed to by a plurality of data source partition cursors.

Abstract: The invention relates to a system and method for predicting the elapsed time required to execute a database utility command. A base of historical data is maintained containing information relating to the elapsed time required for previous database utility commands to execute. The elapsed time for a database utility command provides a consolidated measurement of all factors that affect processing time, including amount and type of data stored as well as changes within the computer system environment. Upper and lower control limits are calculated using statistical process control techniques to predict database utility command execution times. The technique may be used to validate and/or edit database utility command files containing multiple database utility commands, based upon whether the commands contained within are likely to fully execute within a specified batch window.

Abstract: A method, apparatus, and article of manufacture of a computer-implemented parallel database loading system. The optimum number of tasks to be processed by the system is determined by identifying the memory constraints of the system, by identifying available processing capabilities, and by determining a number of load and sort processes to be started in parallel based on the identified memory constraints and processing capabilities. Optimizing the number of load and sort processes increases overall system processing speed.

Abstract: A technique for loading data into a data store connected to a computer. Under control of a main process, multiple agent load processes are started for loading data in parallel. The main process awaits receipt of a checkpoint signal from each agent load process. Then, upon receiving the checkpoint signal from each load process, the main process performs a checkpoint.

Abstract: The invention relates to a system and method for predicting the elapsed time required to execute a database utility command. A base of historical data is maintained containing information relating to the elapsed time required for previous database utility commands to execute. The elapsed time for a database utility command provides a consolidated measurement of all factors that affect processing time, including amount and type of data stored as well as changes within the computer system environment. Upper and lower control limits are calculated using statistical process control techniques to predict database utility command execution times. The technique may be used to validate and/or edit database utility command files containing multiple database utility commands, based upon whether the commands contained within are likely to fully execute within a specified batch window.

Abstract: A reorganizer utility performs an online reorganization of a database that provides applications with concurrent access to data during an iterative process of log apply operations to a shadow version of the data. The reorganizer continues this iterative process of log apply operations until an estimate of an elapsed time necessary to complete a last iteration is less than or equal to a user-specified maximum length of time that read-only (RO) access may be provided to the data. When this threshold is reached, the reorganizer performs the last iteration, which includes draining all applications making modifications to the data; allowing read-only access to the data, performing log apply operations against the shadow version of the data, and then switching all of the applications to the shadow data.

Abstract: A technique for identifying changes in a data store connected to a computer. Initially, one or more interval changes are measured. Each interval change indicates an amount of change in the data store at an interval. Next, a data store change is estimated that indicates an amount of change in the data store across all of the intervals using each interval change.

Abstract: A method, apparatus and article of manufacture is provided for loading data from a remote data source record by record, in a computer system network connecting a source site and a target site via a database connection communication line. The source site has at least one data source and a Distributed Relational Database Architecture (DRDA) software server having multi-database access to DBMSs. The target site requests data loading from the source site DRDA via a block of Structured Query Language (SQL) statements. Data are transported record by record via the database connection communication line according to the DRDA communication protocol, wherein the target site loads records concurrently with the unloading of records in the source site. The data loading may also be performed in a pipeline manner, loading data records in multiple partitions with a plurality of parallel streams, pointed to by a plurality of data source partition cursors.

Abstract: A method, apparatus, and article of manufacture for a computer-implemented building indexes system. Indexes are built for a database that is stored in a data storage device coupled to a computer. An amount of available memory is determined. An amount of memory for use in transmitting data between extract, sort, and index build tasks is determined. Then, a number of sort tasks to be used to build indexes is determined based on the determined amount of available memory, the determined amount of memory for use in transmitting data between tasks, and task memory requirements.

Abstract: A method, apparatus, and article of manufacture for a computer implemented rebalancing system. Partitioned data is rebalanced in a database stored on a data storage device connected to a computer. Range values are redefined for each partition. Next, the data is reordered into the redefined ranges for the partitions.

Abstract: A method, apparatus, and article of manufacture for a computer implemented image copying system for identifying modifications in a database in a computer. The database contains data objects and is stored on a primary data storage device connected to the computer. A data object identifier is generated for each data object as the data object is modified. An image identifier is generated for an image copy of the database when the image copy is updated, wherein the image copy is stored on a secondary data storage device. When one or more data objects are to be copied from the primary data storage device to the secondary data storage device, the data objects to be copied are determined by comparing the data object identifier of each data object to the image identifier.

Abstract: A method, apparatus, and article of manufacture for a computer-implemented repartitioning system. Data is repartitioned in a database stored on a data storage device connected to a computer. First, it is detected that a partitioning scheme for the data has been altered. Next, partitions that would be affected by the altered partitioning scheme are identified. Then, the identified partitions are reorganized based on the altered partitioning scheme.

Abstract: A method, apparatus, and article of manufacture for a computer implemented recovery system for restoring a database in a computer. The database contains objects and is stored on a primary data storage device connected to the computer. Objects of different types in the database are copied from the primary data storage device to a secondary data storage device. Modifications to the objects are logged in a log file. A recovery indicator is received that indicates that recovery of the objects in the database is required. The objects are copied from the secondary data storage device to the database on the primary data storage device. Modifications in the log file are applied to the copied objects during one pass through the log file.

Abstract: A primary database's log records are used to update a second database that has a different encoding scheme than the log record and the primary database. This encoding scheme, for example, may involve compression, encryption, or another data transformation. First, a stale encoded data record from the second database is identified. This data record is one whose counterpart in the primary database has been updated in the primary database and where update has been logged. The stale data record is modified to reflect its contents as encoded by the first encoding scheme rather than the second. Then, the logged change is applied to update the modified record. The updated record is then decoded and subsequently re-encoded according to the second encoding scheme. This record is then stored in the second database replacing the stale second data record.

Abstract: A multiprocessing system forms a data structure, such as by loading reorganizing, or recovering, while concurrently collecting various statistics about the data structure. The data structure may comprise tables and/or indices, for example. A first processing unit forms the data structure by assimilating data from one or more data sources into data rows, storing the rows in a buffer, and copying the rows from the buffer to the data structure. Concurrently with the forming step, the same or a second processing unit retrieves the rows from the buffer and applies a predetermined analysis to the rows to formulate statistics regarding the data structure.

Abstract: A method, apparatus, and article of manufacture for a computer implemented recover/build index system. The recover/build index system builds a database index for a database file by scanning partitions of the database file in parallel to retrieve key values and their associated record identifier (rid) values. The recover/build index system then sorts the scanned key/rid values for each partition in parallel. Next, the recover/build index system performs one or more merges on the sorted key/rid values from all of the partitions to generate a single key/rid value stream. Finally, the recover/build index system builds the index using the single key/rid value stream.