Abstract: Described is a process for encrypting and compressing data for storage by computing systems. A data stream received by a computing system is separated into data chunks for storage on a storage device using a data layout format. Each data chunk is examined and if possible, compressed to produce a compressed data chunk. Whether or not the data chunk can be compressed, the data chunk is encrypted and stored on the storage device in addition to metadata that describes the data chunk. When reading the stored data chunk from the storage device, the metadata identifies the starting location of the stored data chunk and provides metadata information to the computing system for decryption and decompression processing.

Abstract: A system and method for “on-the-fly” de-duplication of data before storing the data in a storage system is provided. A data de-duplication module illustratively cooperates with protocol servers and a file system of a storage operating system executing on the storage system to implement the novel de-duplication technique. The de-duplication module illustratively generates a block store, an index file and a hash table on storage space provided by the storage system. The hash table is utilized for tracking fingerprints and locations of blocks within the block store. The index file is utilized for storing directory information identifying the contents of data containers stored on the storage system, while the block store is utilized to store raw data blocks that comprise the data containers.

Abstract: According to one or more embodiments of the present invention, a network cache intercepts data requested by a client from a remote server interconnected with the cache through one or more wide area network (WAN) links (e.g., for Wide Area File Services, or “WAFS”). The network cache stores the data and sends the data to the client. The cache may then intercept a first write request for the data from the client to the remote server, and determine one or more portions of the data in the write request that changed from the data stored at the cache (e.g., according to one or more hashes created based on the data). The network cache then sends a second write request for only the changed portions of the data to the remote server.

Abstract: A system and method for signature based data container recognition is provided. When a new data container, such as a lun, is created, a security appliance generates a signature of the data container, by, e.g., examining the contents of one or more data blocks of the data container. The generated signature is then associated with the appropriate encryption key for the data container and is stored either within a configuration database of the security appliance or on a key management system operating within a security appliance environment. To identify the encryption key associated with a data container, the security appliance generates a signature of the data container and compares the generated signature with the stored signatures. Should there be a matching signature, the security appliance utilizes the encryption key associated with the matching signature to process data access requests to/from the data container.

Abstract: A technique for eliminating duplicate data is provided. Upon receipt of a new data set, one or more anchor points are identified within the data set. A bit-by-bit data comparison is then performed of the region surrounding the anchor point in the received data set with the region surrounding an anchor point stored within a pattern database to identify forward/backward delta values. The duplicate data identified by the anchor point, forward and backward delta values is then replaced in the received data set with a storage indicator.