Abstract: A chunk index has information on chunks in a storage space referenced in objects in the storage space. The chunk index includes a reference count for each chunk indicating a number of objects in which the chunk is referenced and a reference measurement representing a level of data object references to the chunk. One chunk is selected to remove from the storage space based on a criteria applied to the reference measurements of chunks having reference counts indicating that the chunks are not referenced in one object in the storage space.

Abstract: Provided are a method, system, and article of manufacture, wherein a data structure corresponding to a set of client nodes selected from a plurality of client nodes is generated. Objects from the selected set of client nodes are stored in the data structure. A determination is made that an object corresponding to a client node of the selected set of client nodes has to be stored. An additional determination is made as to whether the object has already been stored in the data structure by any client node of the selected set of client nodes. The object is stored in the data structure, in response to determining that the object has not already been stored in the data structure by any client node of the selected set of client nodes.

Abstract: The present invention provides for an enhanced method and system for assuring integrity of deduplicated data objects stored within a storage system. A digital signature of the data object is generated to determine if the data object reassembled from a deduplicated state is identical to its pre-deduplication state. In one embodiment, generating the object signature of a data object before deduplication comprises generating an object signature from intermediate hash values computed from a hash function operating on each data chunk within the data object, the hash function also used to determine duplicate data chunks. In an alternative embodiment, generating the object signature of a data object before deduplication comprises generating an object signature on a portion of each data chunk of the data object.

Abstract: One aspect of the present invention includes enabling data chunks to be shared among different storage pools within a storage management system, according the use of deduplication and storage information kept at the system level, and applied with policy-based rules that define the scope of deduplication. In one embodiment, the parameters of performing deduplication are defined within the policy, particularly which of the plurality of storage pools allow deduplication to which other pools. Accordingly, a data object may be linked to deduplicated data chunks existent within other storage pools, and the transfer of a data object may occur by simply creating references to existing data chunks in other pools provided the policy allows the pool to reference chunks in these other pools. Additionally, a group of storage pools may be defined within the policy to perform a common set of deduplication activities across all pools within the group.

Abstract: In one aspect of the present description, in connection with storing a first deduplicated data object in a primary storage pool, described operations include determining the duration of time that the first data object has resided in the primary storage pool, and comparing the determined duration of time to a predetermined time interval. In addition, described operations include, after the determined duration of time meets or exceeds the predetermined time interval, determining if the first data object has an extent referenced by another data object, and determining whether to move the first data object from the primary storage pool to a secondary storage pool as a function of whether the first data object has an extent referenced by another data object after the determined duration of time meets or exceeds the predetermined time interval. Other features and aspects may be realized, depending upon the particular application.

Abstract: In one aspect of the present description, in connection with storing a first deduplicated data object in a primary storage pool, described operations include determining the duration of time that the first data object has resided in the primary storage pool, and comparing the determined duration of time to a predetermined time interval. In addition, described operations include, after the determined duration of time meets or exceeds the predetermined time interval, determining if the first data object has an extent referenced by another data object, and determining whether to move the first data object from the primary storage pool to a secondary storage pool as a function of whether the first data object has an extent referenced by another data object after the determined duration of time meets or exceeds the predetermined time interval. Other features and aspects may be realized, depending upon the particular application.

Abstract: During execution of an existing scheduling computer program on a client node, an update computer program and a self-describing automatic installation package are downloaded to the client node from a logical depot node implemented on an existing management server. Therefore, advantageously, no physical depot node or other additional computing device is needed for the client node to update itself. Execution of the update computer program is spawned on the client node from the existing scheduling computer program. As such, the update computer program inherits root access to the client node and security credentials to the management server from the scheduling computer program—advantageously, then, a user does not have to perform any laborious configuration of the client node in order to update the node. The client node ultimately updates itself using the self-describing automatic installation package, which includes all the information needed for the client node to update itself.

Abstract: The various embodiments of the present invention include techniques for securing the use of data deduplication activities occurring in a source-deduplicating storage management system. These techniques are intended to prevent fake data backup, target data contamination, and data spoofing attacks initiated by a source. In one embodiment, one technique includes limiting chunk querying to authorized users. Another technique provides detection of attacks and unauthorized access to keys within the target system. Additional techniques include the combination of validating the existence of data from the source by validating the data chunk, validating a data sample of the data chunk, or validating a hash value of the data chunk. A further embodiment involves the use of policies to provide authorization levels for chunk sharing and linking within the target. These techniques separately and in combination provide a comprehensive strategy to avoid unauthorized access to data within the target storage system.

Abstract: Provided are a computer program product, system, and method for encrypting data objects to back-up to a server. A client private key is intended to be maintained only by the client. A data object of chunks to store at the server is generated. A first portion of the chunks in the data object is encrypted with the client private key and the first portion of the chunks in the data object encrypted with the client private key are sent to the server to store. A second portion of the chunks in the data object not encrypted with the client private key are sent to the server to store.

Abstract: Provided are a computer program product, system, and method for encrypting data objects to back-up to a server. A client private key is intended to be maintained only by the client. A data object of chunks to store at the server is generated. A first portion of the chunks in the data object is encrypted with the client private key and the first portion of the chunks in the data object encrypted with the client private key are sent to the server to store. A second portion of the chunks in the data object not encrypted with the client private key are sent to the server to store.

Abstract: Data objects are replicated from a source storage managed by a source server to a target storage managed by a target server. A source list is built of objects at the source server to replicate to the target server. The target server is queried to obtain a target list of objects at the target server. A replication list is built indicating objects on the source list not included on the target list to transfer to the target server. For each object in the replication list, data for the object not already at the target storage is sent to the target server and metadata on the object is sent to the target server to cause the target server to include the metadata in an entry for the object in a target server replication database. An entry for the object is added to a source server replication database.

Abstract: A chunk index has information on chunks in a storage space referenced in objects in the storage space. The chunk index includes a reference count for each chunk indicating a number of objects in which the chunk is referenced and a reference measurement representing a level of data object references to the chunk. One chunk is selected to remove from the storage space based on a criteria applied to the reference measurements of chunks having reference counts indicating that the chunks are not referenced in one object in the storage space.

Abstract: A chunk index has information on chunks in a storage space referenced in objects in the storage space. The chunk index includes a reference count for each chunk indicating a number of objects in which the chunk is referenced and a reference measurement representing a level of data object references to the chunk. One chunk is selected to remove from the storage space based on a criteria applied to the reference measurements of chunks having reference counts indicating that the chunks are not referenced in one object in the storage space.

Abstract: Data objects are replicated from a source storage managed by a source server to a target storage managed by a target server. A source list is built of objects at the source server to replicate to the target server. The target server is queried to obtain a target list of objects at the target server. A replication list is built indicating objects on the source list not included on the target list to transfer to the target server. For each object in the replication list, data for the object not already at the target storage is sent to the target server and metadata on the object is sent to the target server to cause the target server to include the metadata in an entry for the object in a target server replication database. An entry for the object is added to a source server replication database.

Abstract: A source server maintains a replication rule specifying a condition for a replication attribute and a replication action to take if the condition with respect to the replication attribute is satisfied, wherein the replication action indicates to include or exclude the object having an attribute value for the replication attribute that satisfies the condition. For each of the objects, the replication rule is applied by determining an attribute value of the object corresponding to the replication attribute in the replication rule and determining whether the determined attribute value satisfies the condition for the replication attribute defined in the determined replication rule. The replication action on the object in response to determining that the determined attribute value satisfies the condition for the replication attribute.

Abstract: Various techniques for improving the performance of restoring deduplicated data files from a server to a client within a storage management system are disclosed. In one embodiment, a chunk index is maintained on the client that tracks the chunks remaining on the client for each data file that is stored to and restored from the storage server. When a specific file is selected for restore from the storage server to the client, the client determines if any local copies of this specific file's chunks are stored in files already existing on the client data store. The file is then reconstructed from a combination of these local copies of the file chunks and chunks retrieved from the storage server. Therefore, only chunks that are not stored or are inaccessible to the client are retrieved from the server, reducing server-side processing requirements and the bandwidth required for data restore operations.

Abstract: A process is disclosed by which data is securely deleted in a transactionally consistent manner. This may be accomplished by committing a preparation transaction for a data object within a system managing the data object in order to return the system to an initial condition if necessary, attempting to commit an execution transaction with the data object only after committing the preparation transaction, and securely deleting any portion of the data object necessary to return the system to the initial condition if committing the execution transaction fails and to change the system to a completed condition only if committing the execution transaction succeeds. In a delete or move transaction an existing backup object may be assigned a new logically deleted state such that if the delete or move transaction fails, the data will be made accessible again.

Abstract: Provided are computer program product, system, and method for restoring deduplicated data objects from sequential backup devices. A server stores data objects of extents having deduplicated data in the at least one sequential backup device. The server receives from a client a request for data objects. The server determines extents stored in the at least one sequential backup device for the requested data objects. The server or client sorts the extents according to an order in which they are stored in the at least one sequential backup device to generate a sort list. The server retrieves the extents from the at least one sequential backup device according to the order in the sort list to access the extents sequentially from the sequential backup device in the order in which they were stored. The server returns the retrieved extents to the client and the client reconstructs the requested data objects from the received extents.

Abstract: One aspect of the invention is a method for providing real-time feedback regarding the effect of applying a policy definition used for management in a computing system. An example of the method includes receiving the policy definition, and accessing stored information regarding at least one managed entity in the computing system. This example also includes applying the policy definition to the information regarding the at least one managed entity. This example further includes outputting information providing real-time feedback regarding the effect of applying the policy definition to the information regarding the at least one managed entity. Another aspect of the invention is a method for defining a policy used for management in a computing system.

Abstract: One aspect of the invention is a method for providing real-time feedback regarding the effect of applying a policy definition used for management in a computing system. An example of the method includes receiving the policy definition, and accessing stored information regarding at least one managed entity in the computing system. This example also includes applying the policy definition to the information regarding the at least one managed entity. This example further includes outputting information providing real-time feedback regarding the effect of applying the policy definition to the information regarding the at least one managed entity. Another aspect of the invention is a method for defining a policy used for management in a computing system.