Abstract: A method and system of optimizing the memory usage and performance of data deduplication storage systems includes organizing the metadata of data blocks needed by deduplicating storage systems. A three level hierarchy is used. Level 1 stores the metadata on disk along with the user data. Level 2 uses low latency storage (e.g. RAM and Solid State Disks) to cache the on-disk meta data for faster direct access. Level 3 organizes the fingerprints using a Trie and is entirely resident in RAM. Thus, the search, to determine whether a data block is unique or not and a candidate for transfer, can be more efficiency executed and to ensure that the meta data is transactionally secure.

Abstract: A method and system of optimizing the performance, capacity and data redundancy in a storage system by defining the LBA range on each storage element managing that corresponding range (slice) on each device using a data protection method optimized for the performance and level of data protection required. The creation of such a storage array along with the replacement of failed elements and the expansion of the capacity through the addition of additional elements is managed is an automatic and transparent manner.

Abstract: A method and system is provided for copying data between two deduplicating storage systems. A list of unique fingerprints is compiled from the data which will be sent. This list is transmitted to the receiving system during a preliminary data exchange called the preamble. The receiving system replies with a second list which contains the unique fingerprints of the data which either needs to be sent or can be omitted. Which list depends on the size of the list where the smaller list is sent for efficiency and lower bandwidth consumption. A reference list of duplicate blocks being sent is retained on the receiving system until the copy operation is complete. This reference list is used to protect blocks on the receiving system by deferring deletions until the incoming hallow block can reference the duplicate block on the receiver to confirm that is on the target system and should not be deleted.

Abstract: The present invention is a method for the allocation of data on physical media by a file system that eliminates duplicate data. Efficient searches are employed using a unique algorithm when a compare on hash is used to achieve realtime operation of the file system. The in memory feature of the invention allows the search to be performed in constant time. Also, the on disk representation of search structures enables the present invention to maintain these critical search structures in a highly efficient, self-consistent and resilient manner.

Abstract: The present invention provides a method for the allocation of data on physical media by a file system which optimizes power consumption. The present invention improves power efficiency of computing devices using file systems by allocating data in a way that allows devices to power down when they are not in use. This is carried out by using an allocation group method to divide the storage system into groups of one or more disk drives. The allocator fills the first group entirely before moving to the second group. The allocator globally allocates blocks within each group. Groups are typically sized to meet the average I/O bandwidth requirement of the application. The method also dynamically increases the number of active allocation groups based on I/O load. This is done by computing the average number of write requests per time period.

Abstract: A method and system of optimizing the memory usage and performance of data deduplication storage systems includes organizing the metadata of data blocks needed by deduplicating storage systems. A three level hierarchy is used. Level 1 stores the metadata on disk along with the user data. Level 2 uses low latency storage (e.g. RAM and Solid State Disks) to cache the on-disk meta data for faster direct access. Level 3 organizes the fingerprints using a Trie and is entirely resident in RAM. Thus, the search, to determine whether a data block is unique or not and a candidate for transfer, can be more efficiency executed and to ensure that the meta data is transactionally secure.

Abstract: A method and system is provided for copying data between two deduplicating storage systems. A list of unique fingerprints is compiled from the data which will be sent. This list is transmitted to the receiving system during a preliminary data exchange called the preamble. The receiving system replies with a second list which contains the unique fingerprints of the data which either needs to be sent or can be omitted. Which list depends on the size of the list where the smaller list is sent for efficiency and lower bandwidth consumption. A reference list of duplicate blocks being sent is retained on the receiving system until the copy operation is complete. This reference list is used to protect blocks on the receiving system by deferring deletions until the incoming hallow block can reference the duplicate block on the receiver to confirm that is on the target system and should not be deleted.

Abstract: The present invention is a method for the allocation of data on physical media by a file system that eliminates duplicate data. Efficient searches are employed using a unique algorithm when a compare on hash is used to achieve realtime operation of the file system. The in memory feature of the invention allows the search to be performed in constant time. Also, the on disk representation of search structures enables the present invention to maintain these critical search structures in a highly efficient, self-consistent and resilient manner.

Abstract: The present invention provides a method for the allocation of data on physical media by a file system which optimizes power consumption. The present invention improves power efficiency of computing devices using file systems by allocating data in a way that allows devices to power down when they are not in use. This is carried out by using an allocation group method to divide the storage system into groups of one or more disk drives. The allocator fills the first group entirely before moving to the second group. The allocator globally allocates blocks within each group. Groups are typically sized to meet the average I/O bandwidth requirement of the application. The method also dynamically increases the number of active allocation groups based on I/O load. This is done by computing the average number of write requests per time period.