Abstract: Techniques for deduplicating a data stream with checksum data embedded therein are described. According to one embodiment, a first data stream is received from a client having a plurality of data regions and a plurality of checksums for verifying integrity of the data regions embedded therein, where the first data stream represents a file or a directory of one or more files of a file system associated with the client. In response the first data stream with the checksums removed is deduplicated into a plurality of deduplicated chunks.

Abstract: A system for storing files comprises a processor and a memory. The processor is configured to break a file into one or more segments; store the one or more segments in a first storage unit; and add metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit. The memory is coupled to the processor and configured to provide the processor with instructions.

Abstract: Techniques for committing data objects to be immutable are described herein. According to one embodiment, in response to a request received through an interface of a storage system, the request being associated with a data object stored in the storage system, it is determined whether the data object should be committed to be immutable. The data object is committed to be immutable if it is determined that the data object should be committed to be immutable. Thereafter, an action associated with the request is performed, where the action is performed dependent upon whether the data object is immutable. Other methods and apparatuses are also described.

Abstract: According to one embodiment, recovering file systems by restoring partitions is described. In one example, a method includes discovering an error in a storage partition of a file storage system, the storage partition including identification metadata to identify itself to the file system, finding a backup storage partition containing data and identification metadata corresponding to that of the error storage partition, restoring the error storage partition using the backup storage partition, and using the restored identification metadata to identify the restored storage partition to the file system.

Abstract: A method and apparatus to exercise and manage a related set of power managed storage devices is described. In one example a method includes receiving a request to allow a transition of the set of storage devices from a first power state to a second power state, determining a wear level for each device in the set, comparing the determined wear levels to thresholds, testing data integrity for each device in the set, and if the wear level does not exceed the threshold and the data integrity tests are passed, then commanding the requested power state transition.

Abstract: Techniques for data migration of a storage system are described herein. According to one embodiment, a candidate index is generated to store information identifying a plurality of segments associated with one or more candidate files of a source storage tier to be migrated to a target storage tier of a storage system. Segments of the one or more candidate files are then migrated from the source storage tier to the target storage tier according to a sequential order, and based on the candidate index.

Abstract: Techniques for data migration of a storage system are described herein. According to one embodiment, a list of files stored in a source storage tier is identified as candidate files to be migrated to a target storage tier based on a migration policy. One of a regular data movement method and a bulk data movement method is selected based on a characteristic of the candidate files. The candidate files are then migrated from the source storage tier to the target storage tier using the selected data movement method.

Abstract: Techniques for efficiently indexing and searching similar data are described herein. According to one embodiment, in response to a query for one or more terms received from a client, a query index is accessed to retrieve a list of one or more super files. Each super file is associated with a group of similar files. Each super file includes terms and/or sequences of terms obtained from the associated group of similar files. Thereafter, the super files representing groups of similar files are presented to the client, where each of the super files includes at least one of the queried terms. Other methods and apparatuses are also described.

Abstract: A method, article of manufacture, and apparatus for transferring data from a source tier to a target tier. In some embodiments, this may include applying a policy to an object in the source tier, determining the target tier based on the policy, and moving the object from the source tier to the target tier based on the policy. In some embodiments, determining the target tier based on the policy may include determining a higher target tier. In some embodiments, determining the target tier based on the policy may include determining a lower target tier.

Abstract: A method and apparatus to control and manage a power state in a related set of storage devices is described. In one example a method includes, determining an idleness measure at the file system, the idleness measure indicating availability requirements of the device set, and setting an idle state based on the idleness measure. The method also includes receiving the idle state setting at the storage subsystem, determining whether the idle state setting is different from a current state of the device set, determining whether to change the idle state of the device set if the idle state setting is different from the current state of the device set, the determining being based on information about the device set. The method also includes commanding the device set to change the current idle state to correspond to the idle state setting based on determining whether to change the idle state.

Abstract: According to one embodiment, in response to a request to write a prime segment of a file system of a storage system having a plurality of storage units, one or more of the storage units are identified based on a prime segment write-map (PSWM). The PSWM includes information indicating which of the storage units to which a next prime should be written. The prime segment is then written in the one or more storage units identified from the PSWM, without writing the prime segment to a remainder of the storage units. The prime segment represents at least a portion of a prime that contains metadata representing a consistent point of data stored in the file system.

Abstract: Techniques for managing data objects of a data storage system are described herein. According to one embodiment, a hash function is generated for a plurality of data objects by analyzing each of the data objects stored in the data storage system. For each of the data objects, a hash operation is performed on the data object using the hash function, generating a hash value. The hash value is associated with a predetermined attribute of the data object, such that the predetermined attribute of the data object is uniquely identified using the hash function subsequently in response to a request for accessing the predetermined attribute of the data object. The data object is then stored at a persistent storage location of the data storage system, wherein the persistent storage location is identifiable based on a hash value obtained from the hash function.

Abstract: A computer-implemented proxy server local to a backup target system performs a backup method for a system supporting a plurality of virtual machines. The backup method includes receiving an initialization of a connection over a network from a backup server identifying the system to backup to the backup target system, establishing a communication connection over the network with the system to backup including identifying an interface for accessing data of the system to backup, receiving an image of the system to backup from the backup server, retrieving a data set corresponding to the image from the system to backup over the network, and writing the data set to a local file in the backup target system.

Abstract: A system for storing files comprises a processor and a memory. The processor is configured to break a file into one or more segments; store the one or more segments in a first storage unit; and add metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit.

Abstract: Techniques for providing access to data objects within another data object are described herein. In one embodiment, a compound object including multiple data objects is received and metadata is extracted for a data object from the compound object, where the metadata includes a layout of the data object in view of the compound object. Subsequently, access to one or more of the data objects within the compound object is provided based on the extracted metadata without using an application associated with the compound object. Other methods and apparatuses are also described.

Abstract: According to one embodiment, a large scale data storage system with fault tolerance is described. In one example, a system includes a plurality of storage partitions, each partition including a portion of a large scale data store, and metadata corresponding to the portion of the large scale data store. At least one partition includes namespace metadata for the large scale data store including namespace data for a portion of the large scale data store outside the at least one partition.

Abstract: Techniques for providing storage for electronic records are described herein. According to one embodiment, a command is received from a client through an interface of a storage system. An approval is received from an authorization agent associated with the storage system for the received command. In response to the approval received from the authorization agent for the received command, an operation associated with the received command is performed. Other methods and apparatuses are also described.

Abstract: Techniques for virtual machine incremental backup are described herein. According to one embodiment, a request for an incremental backing up a virtual machine (VM) is received at a storage system, the request identifying a requested VM disk image associated with a consistent state of the VM. The storage system determines a difference between the requested VM disk image and a previous VM disk image representing a previous VM backup. The changes between the requested VM disk image and a previous VM disk image are then transmitted to a target backup storage system.

Abstract: Techniques for virtual machine full backup are described herein. According to one embodiment, in response to a request to back up a virtual machine (VM) of a client, a request of VM backup is sent out. A consistent state of the VM is then identified via a VM application program interface (VM API). Subsequently a request is sent to a storage system associated with the client to ask for VM disk image associated with the consistent state of the VM to a target backup storage system.

Abstract: A system for storing files comprises a processor and a memory. The processor is configured to break a file into one or more segments; store the one or more segments in a first storage unit; and add metadata to the first storage unit so that the file can be accessed independent of a second storage unit, wherein a single namespace enables access for files stored in the first storage unit and the second storage unit. The memory is coupled to the processor and configured to provide the processor with instructions.