Abstract: A method and system for replicating file system entities in a manner that preserves block-level access and file system efficiency mechanisms such as deduplication are disclosed. In an embodiment, a replication engine receives a stream of file system entities that include a file system inodes and file system data blocks. The replication engine generates object-based storage (OBS) objects based on data and reference information specified by the file system entities. As part of generating the OBS objects, the replication engine generates at least one inode file object that associates file block numbers of a file system inode file and the inode numbers. The replication engine uses inode information to generate reference objects that logically associate file block numbers with data block numbers in per inode manner. The replication engine further generates data objects that contains the file system data blocks and that associates the data blocks with corresponding data block numbers.

Abstract: An enterprise may use file level and block level storage for operational access to reap the performance benefits, and use object storage for back-up storage and scale out. Data of objects and objects themselves can be organized to allow for efficient storage management operations, including storage snapshot operations and deduplication operations. A root file container can be represented in object storage with a metadata object (“file container object”) and child file container can be represented in object storage with a metadata object (“file group object”). Each file can be represented in object storage with a metadata object that indicates the data units of the file depending upon the storage technology hosting the file (e.g., a file data block in file level storage). The data units of the file can be represented in object storage as elements of a data object.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server, in response to receiving a request from a client for a page stored at a first virtual address, determining that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address or that the first and second virtual addresses map to a deduplicated page within a storage system, and transmitting metadata to the client mapping the first virtual address to a second virtual address that also maps to the deduplicated page. For one embodiment, the metadata is transmitted in anticipation of a request for the redundant/deduplicated page via the second virtual address. For an alternate embodiment, the metadata is sent in response to a determination that a page that maps to the second virtual address was previously sent to the client.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server, in response to receiving a request from a client for a page stored at a first virtual address, determining that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address or that the first and second virtual addresses map to a deduplicated page within a storage system, and transmitting metadata to the client mapping the first virtual address to a second virtual address that also maps to the deduplicated page. For one embodiment, the metadata is transmitted in anticipation of a request for the redundant/deduplicated page via the second virtual address. For an alternate embodiment, the metadata is sent in response to a determination that a page that maps to the second virtual address was previously sent to the client.

Abstract: Techniques and components for backing up data are disclosed. A first subset of data units is selected from a plurality of data units of a storage system to create a first partial baseline for backup of the first storage system. The number of data units in the first subset of data units is determined based on a window size parameter n. Data within the storage system that has changed since a previous backup operation are identified. A first backup including the first partial baseline and the first changed data is written to a second storage system. For a data loss event, the techniques include selecting, based on a time of the determined data loss event in combination with n, the first backup and n?1 additional backups, wherein each of the n?1 additional backups include a partial baseline and changed data. The first backup and the n?1 additional backups are written to a third storage system.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server, in response to receiving a request from a client for a page stored at a first virtual address, determining that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address or that the first and second virtual addresses map to a deduplicated page within a storage system, and transmitting metadata to the client mapping the first virtual address to a second virtual address that also maps to the deduplicated page. For one embodiment, the metadata is transmitted in anticipation of a request for the redundant/deduplicated page via the second virtual address. For an alternate embodiment, the metadata is sent in response to a determination that a page that maps to the second virtual address was previously sent to the client.

Abstract: Techniques and components for backing up data are disclosed. A first subset of data units is selected from a plurality of data units of a storage system to create a first partial baseline for backup of the first storage system. The number of data units in the first subset of data units is determined based on a window size parameter n. Data within the storage system that has changed since a previous backup operation are identified. A first backup including the first partial baseline and the first changed data is written to a second storage system. For a data loss event, the techniques include selecting, based on a time of the determined data loss event in combination with n, the first backup and n?1 additional backups, wherein each of the n?1 additional backups include a partial baseline and changed data. The first backup and the n?1 additional backups are written to a third storage system.

Abstract: First partial baseline data of a first storage system is identified. First changed data of the first storage system is identified. The first changed data comprises data that has changed since a previous point in time. First backup data is written to a second storage system. The first backup data comprises the first partial baseline data and the first changed data. After writing the first backup data to the second storage system, second partial baseline data of the first storage system is identified. The second partial baseline data does not include the first partial baseline data. Second changed data of the first storage system is identified. The second changed data comprises data that has changed since writing the first backup data. Second backup data is written to the second storage system. The second backup data comprises the second partial baseline data and the second changed data.

Abstract: An enterprise may use file level and block level storage for operational access to reap the performance benefits, and use object storage for back-up storage and scale out. Data of objects and objects themselves can be organized to allow for efficient storage management operations, including storage snapshot operations and deduplication operations. A root file container can be represented in object storage with a metadata object (“file container object”) and child file container can be represented in object storage with a metadata object (“file group object”). Each file can be represented in object storage with a metadata object that indicates the data units of the file depending upon the storage technology hosting the file (e.g., a file data block in file level storage). The data units of the file can be represented in object storage as elements of a data object.

Abstract: First partial baseline data of a first storage system is identified. First changed data of the first storage system is identified. The first changed data comprises data that has changed since a previous point in time. First backup data is written to a second storage system. The first backup data comprises the first partial baseline data and the first changed data. After writing the first backup data to the second storage system, second partial baseline data of the first storage system is identified. The second partial baseline data does not include the first partial baseline data. Second changed data of the first storage system is identified. The second changed data comprises data that has changed since writing the first backup data. Second backup data is written to the second storage system. The second backup data comprises the second partial baseline data and the second changed data.

Abstract: A method and system for replicating file system entities in a manner that preserves block-level access and file system efficiency mechanisms such as deduplication are disclosed. In an embodiment, a replication engine receives a stream of file system entities that include a file system inodes and file system data blocks. The replication engine generates object-based storage (OBS) objects based on data and reference information specified by the file system entities. As part of generating the OBS objects, the replication engine generates at least one inode file object that associates file block numbers of a file system inode file and the inode numbers. The replication engine uses inode information to generate reference objects that logically associate file block numbers with data block numbers in per inode manner. The replication engine further generates data objects that contains the file system data blocks and that associates the data blocks with corresponding data block numbers.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server, in response to receiving a request from a client for a page stored at a first virtual address, determining that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address or that the first and second virtual addresses map to a deduplicated page within a storage system, and transmitting metadata to the client mapping the first virtual address to a second virtual address that also maps to the deduplicated page. For one embodiment, the metadata is transmitted in anticipation of a request for the redundant/deduplicated page via the second virtual address. For an alternate embodiment, the metadata is sent in response to a determination that a page that maps to the second virtual address was previously sent to the client.

Abstract: Described herein is a novel technique for implementing a policy-based caching engine in a storage system cluster (cluster) to automatically implement volume caching at select nodes of the cluster in optimizing cluster performance. The novel caching engine may be implemented in a management console of the cluster storing policy information related to volume caching operations requested by the caching engine. Policy information may include node or cluster attributes, operational events, and a pre-defined cached volume configuration of import to the user. Based on the stored policy information, the caching engine may determine a set of nodes on which to initiate a volume caching operation and generate a request for such operation to be performed on a particular node. Only select nodes in the cluster may thus implement volume caching to thereby conserve processing resources for more critical operations at other select nodes in the cluster.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server. In response to receiving a request from a client for a page stored at a first virtual address, the storage server determines that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address. Or the storage server determines that the first and second virtual addresses map to a deduplicated page within a storage system. The storage server then transmits metadata to the client. The metadata maps the first virtual address to a second virtual address that also maps to the deduplicated page.

Abstract: I/O bandwidth reduction using storage-level common page information is implemented by a storage server, in response to receiving a request from a client for a page stored at a first virtual address, determining that the first virtual address maps to a page that is a duplicate of a page stored at a second virtual address or that the first and second virtual addresses map to a deduplicated page within a storage system, and transmitting metadata to the client mapping the first virtual address to a second virtual address that also maps to the deduplicated page. For one embodiment, the metadata is transmitted in anticipation of a request for the redundant/deduplicated page via the second virtual address. For an alternate embodiment, the metadata is sent in response to a determination that a page that maps to the second virtual address was previously sent to the client.

Abstract: A cache is used in a network storage system that includes a plurality of data storage nodes in a storage cluster, to automatically spread read and write access load, by a plurality of storage clients, for file system data and metadata, across the plurality of data storage nodes.

Abstract: A cache is used in a network storage system that includes a plurality of data storage nodes in a storage cluster, to automatically spread read and write access load, by a plurality of storage clients, for file system data and metadata, across the plurality of data storage nodes.

Abstract: A variety of methods and apparatus are taught for providing dynamic distributed file system client authentication. One method for providing dynamic distributed file system client authentication within a distributed file system computing environment includes the steps of receiving an NFS request from an NFS client, determining whether the NFS client has an access status sufficient to perform the NFS request, and performing the NFS request when the NFS client has sufficient access status. In some embodiments, the NFS request includes a file handle representing a given file system available on the server computer system and a file operation to be performed upon the given file system. A server computer in accordance with one embodiment of the present invention is operable to provide dynamic NFS client authentication.