Abstract: An extent-based storage architecture is implemented by a storage server receiving a read request for an extent from a client, wherein the extent includes a group of contiguous blocks and the read request includes a file block number. The storage server retrieves an extent identifier from a first sorted data structure, wherein the storage server uses the received file block number to traverse the first sorted data structure to the extent identifier. The storage server retrieves a reference to the extent from a second sorted data structure, wherein the storage server uses the retrieved extent identifier to traverse the second sorted data structure to the reference, and wherein the second sorted data structure is global across a plurality of volumes. The storage server retrieves the extent from a storage device using the reference and returns the extent to the client.

Abstract: The data path in a network storage system is streamlined by sharing a memory among multiple functional modules (e.g., N-module and D-module) of a storage server that facilitates symmetric access to data from multiple clients. The shared memory stores data from clients or storage devices to facilitate communication of data between clients and storage devices and/or between functional modules, and reduces redundant copies necessary for data transport. It reduces latency and improves throughput efficiencies by minimizing data copies and using hardware assisted mechanisms such as DMA directly from host bus adapters over an interconnection, e.g. switched PCI-e “network”. This scheme is well suited for a “SAN array” architecture, but also can be applied to NAS protocols or in a unified protocol-agnostic storage system. The storage system can provide a range of configurations ranging from dual module to many modules with redundant switched fabrics for I/O, CPU, memory, and disk connectivity.

Abstract: A request is received to remove duplicate data. A log data container associated with a storage volume in a storage server is accessed. The log data container includes a plurality of entries. Each entry is identified by an extent identifier in a data structures stored in a volume associated with the storage server. For each entry in the log data container, a determination is made if the entry matches another entry in the log data container. If the entry matches another entry in the log data container, a determination is made of a donor extent and a recipient extent. If an external reference count associated with the recipient extent equals a first predetermined value, block sharing is performed for the donor extent and the recipient extent. A determination is made if the reference count of the donor extent equals a second predetermined value. If the reference count of the donor extent equals the second predetermined value, the donor extent is freed.

Abstract: An extent-based storage architecture is implemented by a storage server receiving a read request for an extent from a client, wherein the extent includes a group of contiguous blocks and the read request includes a file block number. The storage server retrieves an extent identifier from a first sorted data structure, wherein the storage server uses the received file block number to traverse the first sorted data structure to the extent identifier. The storage server retrieves a reference to the extent from a second sorted data structure, wherein the storage server uses the retrieved extent identifier to traverse the second sorted data structure to the reference, and wherein the second sorted data structure is global across a plurality of volumes. The storage server retrieves the extent from a storage device using the reference and returns the extent to the client.

Abstract: A network caching system has a multi-protocol caching filer coupled to an origin server to provide storage virtualization of data served by the filer in response to data access requests issued by multi-protocol clients over a computer network. The multi-protocol caching filer includes a file system configured to manage a sparse volume that “virtualizes” a storage space of the data to thereby provide a cache function that enables access to data by the multi-protocol clients. To that end, the caching filer further includes a multi-protocol engine configured to translate the multi-protocol client data access requests into generic file system primitive operations executable by both the caching filer and the origin server.

Abstract: A technique uniquely identifies block context signatures in a storage volume hierarchy of a storage system. In particular, the technique assigns unique volume identifiers (“IDs,” e.g., buffer tree or “bufftree” IDs) to volumes of the storage volume hierarchy, and allows clone volumes to determine that data blocks belong to an appropriate ancestor volume (e.g., and were written by the ancestor volume prior to creation of the clone). In this manner, the novel technique may uniquely identify and accurately determine whether an accessed data block is a correct data block (e.g., alleviating occurrences of data ID aliasing).

Abstract: A system and method for managing data during consistency points in a storage system is provided. A buffer data control structure is modified to include a flags array that tracks various status flags for both a current and a next consistency point (CP). By utilizing multiple pointers within a buffer control structure, the storage system may permit write operations to continue to a data container undergoing write allocation. Received writes during a write allocation procedure are stored in raw data buffers and the buffer control structure is marked as being dirty for a next CP.

Abstract: A method of operating a storage server includes executing a process of restoring a data set in an active file system of the storage server from a persistent point-in-time image of a data set, and during the process of restoring the data set, servicing input/output requests directed at the data set by a client of the storage server.

Abstract: A system and method for enabling write operations to files undergoing write allocation is provided. The system and method generate a shadow state entry of metadata associated with an inode of the file upon receipt of a write operation. During the write allocation process for the inode, the shadow state information is merged with the data stored in the inode to be written to disk.

Abstract: A system and method for managing file data during consistency points in a file system is provided. A buffer data control structure is modified to include a flags array that tracks various status flags for both a current and a next consistency point (CP). By utilizing multiple pointers within a buffer control structure, the file system may permit write operations to continue to a file undergoing write allocation. Received writes during a write allocation procedure are stored in raw data buffers and the buffer control structure is marked as being dirty for a next CP.

Abstract: A multi-stage technique invalidates and replaces loadable physical volume block numbers (pvbns) stored in indirect blocks of a dual vbn (“flexible”) virtual volume (vvol) of a storage system to enable efficient image transfers and/or fragmentation handling of the flexible vvol. Each loadable pvbn of a pvbn/virtual vbn (vvbn) block pointer pair is converted into a special block pointer having a predefined reserved value that provides a temporary “pvbn_unknown” placeholder until replaced by a real (actual) pvbn. The technique further allows the storage system to serve data from the flexible vvol using the placeholders while the actual pvbns are computed, thereby eliminating latencies associated with completion of actual pvbn replacement for the pvbn_unknown placeholders.

Abstract: An architecture, including a file-level protocol, for supporting sparse volumes on a storage system is provided. The file-level protocol provides coherency checking for use in retrieving data stored on a backing store remote from a storage system.

Abstract: A method of operating a storage system includes moving a block of data stored in the storage system to a new location in the storage system, such as for purposes of defragmentation. After the block has been moved to the new location, the storage system receives a request requiring access to the block. The storage system uses metadata appended to an identified block to detect that the block may have been relocated. The metadata can further be used to distinguish a block that has been relocated from a lost write.

Abstract: An architecture, including a file-level protocol, for supporting sparse volumes on a storage system is provided. The file-level protocol provides coherency checking for use in retrieving data stored on a backing store remote from a storage system.

Abstract: The data path in a network storage system is streamlined by sharing a memory among multiple functional modules (e.g., N-module and D-module) of a storage server that facilitates symmetric access to data from multiple clients. The shared memory stores data from clients or storage devices to facilitate communication of data between clients and storage devices and/or between functional modules, and reduces redundant copies necessary for data transport. It reduces latency and improves throughput efficiencies by minimizing data copies and using hardware assisted mechanisms such as DMA directly from host bus adapters over an interconnection, e.g. switched PCI-e “network”. This scheme is well suited for a “SAN array” architecture, but also can be applied to NAS protocols or in a unified protocol-agnostic storage system. The storage system can provide a range of configurations ranging from dual module to many modules with redundant switched fabrics for I/O, CPU, memory, and disk connectivity.

Abstract: In one embodiment, the invention provides an apparatus for caching data in a network, with the apparatus including a proxy cache configured to receive request for an object from a client and to fetch data blocks from a server. The proxy cache may be configured to cache the data blocks in a hierarchical relationship within the object. The object may be, for example, a data file or a directory. The data blocks that are cached in the proxy cache define an active data set which is based upon a request from a client.

Abstract: Custom logging allows users deploying a proxy cache to customize information logged by the proxy cache. Users may choose to log a certain set of fields in any suitable order within log files. Each log file contains headers that define the set of fields logged, thereby making the log file self-contained for analysis tools. Proxy cache customers use log files for several purposes including monitoring and billing. The invention provides a single structure that controls whether a field is selected for logging or not, and its position within the log file. Changes to the format can be made while the proxy cache is running, causing a new header to be written to the log file and enabling administrators to alter the log format on the fly.

Abstract: In an embodiment, a method to provide data consistency in a storage system, includes: providing, by a server to a proxy cache, a lock associated with a delegated file in the server; in response to a write request from a client, modifying data in a cached copy of the delegated file in the proxy cache; revoking, by the server, the lock associated with the delegated file, in response to a request from another device to access the delegated file so that the delegated file is now a formerly delegated file; and writing the modified data to the formerly delegated file in the server to update the formerly delegated file; and permitting access to the formerly delegated file by the another device.

Abstract: A system and method for managing file data during consistency points in a file system is provided. A buffer data control structure is modified to include a flags array that tracks various status flags for both a current and a next consistency point (CP). By utilizing multiple pointers within a buffer control structure, the file system may permit write operations to continue to a file undergoing write allocation. Received writes during a write allocation procedure are stored in raw data buffers and the buffer control structure is marked as being dirty for a next CP.

Abstract: A method for transferring data of a hybrid virtual volume of a computer data storage system from a source to a destination is disclosed. The method first translates intermingled virtual and physical volume block numbers of the hybrid virtual volume into a data stream having only virtual volume block numbers. The method then sends the data stream to a destination computer.