Abstract: Systems, methods, and computer products for communicating between a client and server by identifying and separating file data streams within a file are provided. The method comprises indicating the type of file data stream to be transmitted and transmitting the indicated file data stream. The transmitted file data stream is compared with a stored data stream. A non-redundant file data stream is stored based upon the outcome of the comparison. The transmitted file data stream and stored data stream may be compared according to a deduplication table based on data stream profiles.

Abstract: Provided are a method, system, and program for managing storage objects. Storage object information is provided for each storage object, wherein the storage object information indicates at least one of a plurality of groups associated with the storage object, and wherein each storage object is capable of being associated with a plurality of groups and different group types. Group information is provided for each group indicating a group identifier, storage objects associated with the group, and a group type.

Abstract: Systems, methods, and computer products for separating file data streams for improved progressive incremental processing are provided. The method comprises identifying data in a file, prior to transmission of the data in a data stream to a second system, such that the second system can distinguish first data from second data in the file. The method further comprises processing the first and second data to determine whether the first or second data has changed, and transmitting the first data to the second system in response to determining the first data has changed, so that the first data is stored in a data storage medium in association with third and fourth data previously stored without replacing the third data and fourth data, wherein the third and fourth data comprise older versions of the first and second data, respectively.

Abstract: Provided are techniques for data deduplication. A chunk of data and a mapping of boundaries between file data and meta data in the chunk of data are received. The mapping is used to split the chunk of data into a file data stream and a meta data stream and to store file data from the file data stream in a first file and to store meta data from the meta data stream in a second file, wherein the first file and the second file are separate files. The file data in the first file is deduplicated.

Abstract: Provided are a system and article of manufacture for merging metadata on files in a backup storage. A query is received for information on files from a file system for a client node. A determination is made of metadata for files in a backup storage for the client node stored external to backup sets. At least one backup set in the backup storage for the client node is processed to determine metadata for files included in at least one backup set. The determined metadata for files in the backup storage and for files included in at least one backup set is merged to form merged metadata. The query is executed against the merged metadata to determine files whose metadata satisfies the query. Results including information from the merged metadata on the determined files are returned.

Abstract: Provided are techniques for data deduplication. A chunk of data and a mapping of boundaries between file data and meta data in the chunk of data are received. The mapping is used to split the chunk of data into a file data stream and a meta data stream and to store file data from the file data stream in a first file and to store meta data from the meta data stream in a second file, wherein the first file and the second file are separate files. The file data in the first file is deduplicated.

Abstract: Provided are techniques for data deduplication. A chunk of data and a mapping of boundaries between file data and meta data in the chunk of data are received. The mapping is used to split the chunk of data into a file data stream and a meta data stream and to store file data from the file data stream in a first file and to store meta data from the meta data stream in a second file, wherein the first file and the second file are separate files. The file data in the first file is deduplicated.

Abstract: Systems, methods, and computer products for separating file data streams for improved progressive incremental processing are provided. The method comprises identifying data in a file, prior to transmission of the data in a data stream to a second system, such that the second system can distinguish first data from second data in the file. The method further comprises processing the first and second data to determine whether the first or second data has changed, and transmitting the first data to the second system in response to determining the first data has changed, so that the first data is stored in a data storage medium in association with third and fourth data previously stored without replacing the third data and fourth data, wherein the third and fourth data comprise older versions of the first and second data, respectively.

Abstract: Systems, methods, and computer products for separating file data streams for improved progressive incremental processing are provided. The method comprises identifying data in a file, prior to transmission of the data in a data stream to a second system, such that the second system can distinguish first data from second data in the file. The method further comprises processing the first and second data to determine whether the first or second data has changed, and transmitting the first data to the second system in response to determining the first data has changed, so that the first data is stored in a data storage medium in association with third and fourth data previously stored without replacing the third data and fourth data, wherein the third and fourth data comprise older versions of the first and second data, respectively.

Abstract: A system, apparatus, and process creates a table of contents (TOC), including one or more table of contents (TOC) entries, to manage data in a hierarchical storage management system. Each TOC entry contains metadata describing the contents and attributes of a data object within an image, which is an aggregation of multiple data objects into a single object for storage management purposes. The TOC is stored in a storage hierarchy, such as magnetic disk, for fast access of and efficient operation on the aggregated TOC entries. The system, apparatus, and process also provide for aggregating the TOC entries from one or more TOCs into a TOC set in the storage management server database. The TOC set may be manipulated and queried in order to find a particular data object or image referenced by a TOC entry.

Abstract: Provided are a method, system, and program for transferring data between system and storage in a shared buffer. An application requests a buffer from a component. The component allocates a buffer and returns to the application a first offset in the buffer for application data and a second offset in the buffer for a first header. The application writes the application data at the first offset in the buffer; and writes the first header at the second offset in the buffer. A second header is written to a third offset in the buffer.

Abstract: Systems, methods, and computer products for separating file data streams for improved progressive incremental processing are provided. The method comprises identifying data in a file, prior to transmission of the data in a data stream to a second system, such that the second system can distinguish first data from second data in the file. The method further comprises processing the first and second data to determine whether the first or second data has changed, and transmitting the first data to the second system in response to determining the first data has changed, so that the first data is stored in a data storage medium in association with third and fourth data previously stored without replacing the third data and fourth data, wherein the third and fourth data comprise older versions of the first and second data, respectively.

Abstract: Systems, methods, and computer products for communicating between a client and server by identifying and separating file data streams within a file are provided. The method comprises indicating the type of file data stream to be transmitted and transmitting the indicated file data stream. The transmitted file data stream is compared with a stored data stream. A non-redundant file data stream is stored based upon the outcome of the comparison. The transmitted file data stream and stored data stream may be compared according to a deduplication table based on data stream profiles.

Abstract: Provided are techniques for accessing information. A partial key index is created, wherein the partial key index includes one or more partial key index (PKI) entries, wherein each PKI entry identifies a portion of a metadata object, wherein the metadata object includes metadata entries, each of which corresponds to an associated data object. The partial key index is used to locate one or more PKI entries, wherein each of the one or more PKI entries corresponds to one or more data objects in one of the one or more composite objects. Each of the one or more PKI entries is used to retrieve location information of metadata in a metadata object for each of the one or more composite objects. The metadata from each metadata object is loaded into a database using the location information.

Abstract: Provided are techniques for data deduplication. A chunk of data and a mapping of boundaries between file data and meta data in the chunk of data are received. The mapping is used to split the chunk of data into a file data stream and a meta data stream and to store file data from the file data stream in a first file and to store meta data from the meta data stream in a second file, wherein the first file and the second file are separate files. The file data in the first file is deduplicated.

Abstract: Provided are a system and article of manufacture for merging metadata on files in a backup storage. A query is received for information on files from a file system for a client node. A determination is made of metadata for files in a backup storage for the client node stored external to backup sets. At least one backup set in the backup storage for the client node is processed to determine metadata for files included in at least one backup set. The determined metadata for files in the backup storage and for files included in at least one backup set is merged to form merged metadata. The query is executed against the merged metadata to determine files whose metadata satisfies the query. Results including information from the merged metadata on the determined files are returned.

Abstract: Provided is a method for merging metadata on files in a backup storage. A query is received for information on files from a file system for a client node. A determination is made of metadata for files in a backup storage for the client node stored external to backup sets. At least one backup set in the backup storage for the client node is processed to determine metadata for files included in at least one backup set. The determined metadata for files in the backup storage and for files included in at least one backup set is merged to form merged metadata. The query is executed against the merged metadata to determine files whose metadata satisfies the query. Results including information from the merged metadata on the determined files are returned.

Abstract: A system, apparatus, and process creates a table of contents (TOC), including one or more table of contents (TOC) entries, to manage data in a hierarchical storage management system. Each TOC entry contains metadata describing the contents and attributes of a data object within an image, which is an aggregation of multiple data objects into a single object for storage management purposes. The TOC is stored in a storage hierarchy, such as magnetic disk, for fast access of and efficient operation on the aggregated TOC entries. The system, apparatus, and process also provide for aggregating the TOC entries from one or more TOCs into a TOC set in the storage management server database. The TOC set may be manipulated and queried in order to find a particular data object or image referenced by a TOC entry.

Abstract: A system, apparatus, and process creates a table of contents (TOC), including one or more table of contents (TOC) entries, to manage data in a hierarchical storage management system. Each TOC entry contains metadata describing the contents and attributes of a data object within an image, which is an aggregation of multiple data objects into a single object for storage management purposes. The TOC is stored in a storage hierarchy, such as magnetic disk, for fast access of and efficient operation on the aggregated TOC entries. The system, apparatus, and process also provide for aggregating the TOC entries from one or more TOCs into a TOC set in the storage management server database. The TOC set may be manipulated and queried in order to find a particular data object or image referenced by a TOC entry.

Abstract: Provided are a method, system, and program for transferring data between system and storage in a shared buffer. An application requests a buffer from a component. The component allocates a buffer and returns to the application a first offset in the buffer for application data and a second offset in the buffer for a first header. The application writes the application data at the first offset in the buffer; and writes the first header at the second offset in the buffer. A second header is written to a third offset in the buffer.