Abstract: Provided is a method, system and program for storing data for later retrieval and for transfer within a storage hierarchy. A data storage subsystem stores both individual user files and also managed files, each managed file comprising an aggregation of multiple user files. After receiving user files from a client station, the subsystem stores user files in a retrieval storage pool to facilitate file retrieval. Criteria are subsequently applied to the user files stored in the retrieval storage pool to delete low or lower priority files from the retrieval storage pool. In addition, user files are aggregated in managed files for transfer to another location within the storage hierarchy.

Abstract: Provided is a method, system and program for storing data for later retrieval and for transfer within a storage hierarchy. A data storage subsystem stores both individual user files and also managed files, each managed file comprising an aggregation of multiple user files. After receiving user files from a client station, the subsystem stores user files in a retrieval storage pool to facilitate file retrieval. Criteria are subsequently applied to the user files stored in the retrieval storage pool to delete low or lower priority files from the retrieval storage pool. In addition, user files are aggregated in managed files for transfer to another location within the storage hierarchy.

Abstract: Provided is a method, system and program for storing data for later retrieval and for transfer within a storage hierarchy. A data storage subsystem stores both individual user files and also managed files, each managed file comprising an aggregation of multiple user files. After receiving user files from a client station, the subsystem stores user files in a retrieval storage pool to facilitate file retrieval. Criteria are subsequently applied to the user files stored in the retrieval storage pool to delete low or lower priority files from the retrieval storage pool. In addition, user files are aggregated in managed files for transfer to another location within the storage hierarchy.

Abstract: Provided are a method, system, and an article of manufacture, wherein resources corresponding to at least one copy pool are acquired, and wherein the at least one copy pool has been defined for a first primary storage pool of a storage hierarchy. The acquired resources are retained, in response to determining that data cannot be written to the first primary storage pool. The data is written to the at least one copy pool, in response to writing the data to a second primary storage pool of the storage hierarchy.

Abstract: Provided are a method, system, and program for performing a data transfer operation with respect to a source and target storage devices in a network, wherein storage devices in the network are assigned temporary network addresses. Device information is maintained and includes for each identified device a fixed address that does not change and a temporary network address. A data transfer request is received to transfer data between a source and target devices. The network is queried to determine changes to the temporary network addresses of the storage devices in response to receiving the data transfer request and the device information is updated to include any changed determined temporary network addresses for the storage devices. A data transfer command is constructed to transfer data between the source and target storage devices, wherein the data transfer command includes the temporary network addresses of the source and target devices in the device information.

Abstract: An apparatus, system, and method for managing and formatting data in an autonomous data transfer operation are provided. An initialization module is configured to prepare metadata corresponding to a data source. A loader loads autonomous operation instructions corresponding to the data source into a first location and loads autonomous operation instructions corresponding to the metadata into a second location. An assembler selectively assembles, according to a set of formatting rules, autonomous operation instructions from the first location and the second location into a set of autonomous operation instructions. By assembling an ordered set of autonomous operation instructions, a third party autonomous operation may effectively insert metadata into and remove metadata from a contiguous data stream of a data source without altering the data.

Abstract: An apparatus, system, and method for managing and formatting data in an autonomous data transfer operation are provided. An initialization module is configured to prepare metadata corresponding to a data source. A loader loads autonomous operation instructions corresponding to the data source into a first location and loads autonomous operation instructions corresponding to the metadata into a second location. An assembler selectively assembles, according to a set of formatting rules, autonomous operation instructions from the first location and the second location into a set of autonomous operation instructions. By assembling an ordered set of autonomous operation instructions, a third party autonomous operation may effectively insert metadata into and remove metadata from a contiguous data stream of a data source without altering the data.

Abstract: Provided are a system, method, and article of manufacture wherein at least two primary storage pools are configured, wherein a first primary storage pool is configured to indicate a second primary storage pool as a next storage pool in a storage hierarchy that comprises the first primary storage pool and the second primary storage pool. The first primary storage pool of the storage hierarchy is copied to a copy storage pool, wherein the first primary storage pool and the copy storage pool are included as parameters of a single command. The second primary storage pool is copied to the copy storage pool, in response to determining that the second primary storage pool is the next storage pool for the first primary storage pool in the storage hierarchy.

Abstract: An apparatus, system, and method are disclosed for concurrent storage pool migration and backup. An association module associates at least one copy pool with a second storage pool. A migration module concurrently migrates at least one data file from a first storage pool to the second storage pool and copies the at least one data file to each copy pool associated with the second copy pool that does not already store an instance of the at least one data file. In one embodiment, the migration module further concurrently migrates each data file that the second storage pool cannot receive to a third storage pool.

Abstract: An apparatus, system, and method are disclosed for concurrent storage to an active data file storage pool, copy pool, and next pool. A copy module copies one or more data files from a client to a storage pool that is configured to store active data files. In addition, the copy module may concurrently copy the data files to a copy pool that is configured to backup the storage pool and stores active and inactive data files. The copy module further concurrently copies the data files to a next pool that is configured to offload one or more data files from the storage pool and may store active and inactive data files. In one embodiment, a migration module migrates one or more inactive data files from the storage pool to the next pool by deleting the inactive data files from the storage pool.

Abstract: Provided are a method, system, and program for managing retention of stored objects. A modification request is received with respect to a stored object. A determination is made as to whether a retention protection mechanism is set and a storage policy associated with the stored object is processed to determine whether the stored object has expired according to the storage policy in response to determining that the retention protection mechanism is set. The modification request is allowed to proceed in response to determining that the stored object has expired.

Abstract: A method for generating file copies with minimal mounting and positioning of storage volumes involves receiving a request to generate file copies specifying file selection criteria, identifying matching files meeting the selection criteria, locating the matching files on their storage volumes, and copying the files to a copy set. Determination of file copying order is optimized by placing greater emphasis on relative storage locations of matching files than on the order in which their copies are requested. Once a storage volume is mounted, all files stored on that volume are matched against the pending requests including future requests and if found to be the subject of a request, are copied from the mounted volume in optimal position-sensitive order. Copying from secondary storage is deferred until all copying from primary storage is complete.

Abstract: An apparatus, system, and method for managing and formatting data in an autonomous data transfer operation are provided. An initialization module is configured to prepare metadata corresponding to a data source. A loader loads autonomous operation instructions corresponding to the data source into a first location and loads autonomous operation instructions corresponding to the metadata into a second location. An assembler selectively assembles, according to a set of formatting rules, autonomous operation instructions from the first location and the second location into a set of autonomous operation instructions. By assembling an ordered set of autonomous operation instructions, a third party autonomous operation may effectively insert metadata into and remove metadata from a contiguous data stream of a data source without altering the data.

Abstract: An apparatus, system, and method for managing and formatting data in an autonomous data transfer operation are provided. An initialization module is configured to prepare metadata corresponding to a data source. A loader loads autonomous operation instructions corresponding to the data source into a first location and loads autonomous operation instructions corresponding to the metadata into a second location. An assembler selectively assembles, according to a set of formatting rules, autonomous operation instructions from the first location and the second location into a set of autonomous operation instructions. By assembling an ordered set of autonomous operation instructions, a third party autonomous operation may effectively insert metadata into and remove metadata from a contiguous data stream of a data source without altering the data.

Abstract: Provided are a method, system, and program for performing a data transfer operation with respect to a source and target storage devices in a network, wherein storage devices in the network are assigned temporary network addresses. Device information is maintained and includes for each identified device a fixed address that does not change and a temporary network address. A data transfer request is received to transfer data between a source and target devices. The network is queried to determine changes to the temporary network addresses of the storage devices in response to receiving the data transfer request and the device information is updated to include any changed determined temporary network addresses for the storage devices. A data transfer command is constructed to transfer data between the source and target storage devices, wherein the data transfer command includes the temporary network addresses of the source and target devices in the device information.

Abstract: Storage space allocation is performed in a data storage subsystem, where the amount of storage space to be allocated is determined adaptively in consideration of past data storage requests. First, a request is received from one of the client stations for the subsystem to store a first data item. In response to the request, the subsystem determines whether a storage size estimate for data storage transactions of the first client has been established. If so, the storage size estimate is retrieved. Otherwise, if no storage size estimate has been established, a default storage size estimate is established for data storage transactions of the first client. The estimated amount of storage space is then allocated in the data storage subsystem. After receiving the first data item from the first client station, it is stored in the storage subsystem where it occupies an actual amount of storage space. The estimated and actual amounts of storage space are compared.

Abstract: A data storage subsystem employs managed files, each comprising one or an aggregation of multiple constituent user files, to reduce file management overhead costs. After receiving user files from a client station, the subsystem creates a contiguous managed file by aggregating selected ones of the received user files according to certain predetermined criteria. Managed file creation and use are transparent to the client stations. Eventually, unused or "deleted-file space" arises as individual user files are deleted from their respective managed files. "Reclamation" is triggered when the amount of deleted-file space in a prescribed storage area (e.g. device, volume, managed file, etc.) satisfies certain criteria, e.g. poor data storage efficiency. Reclamation is performed to regain wasted space between managed files, and also space that occurs within managed files due to deletion of some but not all constituent user files. Reclamation is applied to the prescribed storage area one managed file at a time.

Abstract: Files are stored along with embedded codes containing information useful in reconstructing a failed database that normally contains information about the files. Multiple user files are stored in the form of a managed file, identified by a managed filename. Storage of the managed file includes a number of sub-steps, among them the generation and storage of a first frame marker including a recognition pattern, the managed filename, a next-frame locator, and a sequence number distinguishing the first frame marker from other frame markers. Next, a self-describing header is generated and stored adjacent to the first frame marker. A first one of the user files is stored adjacent to the self-describing header. The self-describing header and following user file are exclusively associated with each other, and the header contains certain information about the adjacent user file.