Abstract: A cloud-native global file system in which a local filer creates objects and forward them to a cloud-based object store is augmented to include constant-time rekeying (CTR). At volume creation time on the filer, a random Intermediate Key (IK) is generated. The IK is encrypted using one or more public key(s) for the volume in question, and then stored in encrypted form in a volume metadata file (e.g., cloudvolume.xml) alongside the other volume information. Once created, the IK is treated like any other volume metadata. During startup of a volume manager on the filer, the one or more per-volume IK blobs (present) are decrypted using an appropriate secret key, and then cached in memory. All objects sent to the cloud are then symmetrically encrypted to the current IK for that volume. All objects read from the cloud are decrypted using the locally-cached IK.

Abstract: A cloud-native global file system in which a local filer creates objects and forward them to a cloud-based object store is augmented to include a reshapable caching scheme for the local filer. Like striped caches, the approach uses a stripe, but the striping is implemented via a true RAID 0 (disk striping) rather than as a striped LV (logical volume) device. This approach allows for a “reshape” operation to convert from a n-way stripe set to a n+1-way stripe set. Preferably, a reshape involves redistributing each block on disk to its new calculated home. For example, going from a single disk to a two disk set would move every other block from disk 1 to disk 2, and rearrange the blocks on disk 1 to fill in the “holes”. Performance after the reshape matches that of a striped cache. In one embodiment, the cache is structured as a “degraded” RAID 4.

Abstract: A versioned file storage system (VFS) and method for operating and using the same is disclosed. In an aspect, a mechanism is provided wherein a first VFS interface unit coupling the VFS to a local data storage system is replaced with a second VFS interface unit in a coordinated procedure for taking the first unit offline, making a reliable snapshot of its data and cache structure in the VFS, and loading the same data and cache structure into the second VFS interface brought online. The first and second VFS interfaces transfer the necessary data to achieve the switch through respective side-loading daemons running in each interface. Clients in the local data storage system experience minimal interruption of services in the VFS.

Abstract: A versioned file system comprises a set of structured data representations. At a first time, an interface creates and exports to a cloud data store a first structured data representation corresponding to a first version of the local file system. The first structured data representation is an XML tree having a root element, one or more directory elements associated with the root element, and one or more file elements associated with a given directory element. Upon a change within the file system, the interface creates and exports a second structured data representation corresponding to a second version of the file system. The second structured data representation differs from the first structured data representation up to and including the root element of the second structured data representation. The interface continues to generate and export the structured data representations to the data store.

Abstract: An interface between an existing local file system and a data store generates and exports to the data store a versioned file system. The state of the local file system at a given point in time can be determined using the versioned file system. In operation, the interface creates and exports to the data store structured data representations, which correspond to versions of the local file system, as well as actual file system data that these structured data representations represent. A cache associated with the interface stores recently used data and metadata, as well as all write data and metadata waiting to be transferred to the data store. Prior to transfer, one or more data transformations may be applied to the data and metadata. In one embodiment, the data store comprises one or more cloud storage service providers. A content control system is used to configure and manage the interface.

Abstract: A cloud-based write-once object store is configured to store inode-based data exported to the store from an enterprise file system. For each version of data (e.g., a file) exported to the store, there is a version of the inode corresponding to that data. As versions of the data are exported to the cloud, the system creates multiple versions of the inode. The set of inode versions corresponding to the versions of the file have a pointer associated therewith that specifies the latest version of the data associated with the inode. The inode versions in the set share the same pointer. The inode versions represent a revision history for the inode. For each inode version corresponding to a version of the data, information is received and stored in a new portion of the object store. The inode version for a file version comprises a list of data chunks for the file.

Abstract: A cloud-native global file system used to provide primary file storage for enterprise data is augmented to provide “direct-to-cloud” (D2C) migration of an initial data set. In addition to providing a filer, the service provider configures a direct-to-cloud migration tool, typically as a downloadable virtual machine (VM), that is executed in association with the hardware of the enterprise filesystem whose data set is being migrated. The migration tool reads the existing filesystem, transforms the data into a format consistent with that done by a filer, and pushes the result (a single version) directly to an object store in the cloud. In a first pass (run), typically a large bulk of the data is moved and, depending on how the migration tool is configured, one or more additional passes may then be carried out. When migration is complete (or substantially complete), the filer then is activated to complete the migration (if need be) and take over the go-forward operations.

Abstract: A technique for data sharing among multiple filers that share a volume in a private or public cloud object store is implemented. In this approach, a mechanism is provided to enable a local filer to determine whether other filers that are sharing the volume have a consistent view of new data being written to the cloud object store by the local filer. The begins by associating together a collection of one or more files in a “push class.” On demand, a push operation for the push class is initiated on the local filer. Preferably, the push is managed according to one or more push criteria associated with the push class. Typically, the push operation pushes file data and metadata associated with the one or more files of the push class in respective phases, with the file data being pushed to the cloud during a first phase and the metadata associated with that file data being pushed during a second phase that follows the first phase.

Abstract: A versioned file system comprising network accessible storage is provided. Aspects of the system include globally locking files or groups of files so as to better control the stored files in the file system and to avoid problems associated with simultaneous remote access or conflicting multiple access requests for the same files. A method for operating, creating and using the global locks is also disclosed. A multiprotocol global lock can be provided for filing nodes that have multiple network protocols for generating local lock requests.

Abstract: A versioned file system comprises a set of structured data representations. At a first time, an interface creates and exports to a cloud data store a first structured data representation corresponding to a first version of the local file system. The first structured data representation is an XML tree having a root element, one or more directory elements associated with the root element, and one or more file elements associated with a given directory element. Upon a change within the file system, the interface creates and exports a second structured data representation corresponding to a second version of the file system. The second structured data representation differs from the first structured data representation up to and including the root element of the second structured data representation. The interface continues to generate and export the structured data representations to the data store.

Abstract: An interface between an existing local file system and a data store generates and exports to the data store a versioned file system. The state of the local file system at a given point in time can be determined using the versioned file system. In operation, the interface creates and exports to the data store structured data representations, which correspond to versions of the local file system, as well as actual file system data that these structured data representations represent. A cache associated with the interface stores recently used data and metadata, as well as all write data and metadata waiting to be transferred to the data store. Prior to transfer, one or more data transformations may be applied to the data and metadata. In one embodiment, the data store comprises one or more cloud storage service providers. A content control system is used to configure and manage the interface.

Abstract: A versioned file system comprises a set of structured data representations. Each structured data representation corresponds to a version, and each version comprises a tree of write-once objects. Each version in the versioned file system has associated therewith a borrow window. When it is desired to reconstruct the file system to a point in time, i.e., to perform a “restore,” it is only required to walk a single structured data representation. During a restore, metadata is pulled back from the cloud first, so users can see the existence of needed files immediately. The remainder of the data is pulled back from the cloud when the user goes to open the file. As a result, the entire file system is restored to a previous time nearly instantaneously. A fast restore is performed if an object being restored exists within a borrow window of the version from which the system is restoring.

Abstract: A method of data sharing among multiple entities is provided. Each entity exports to a data store a structured data representation comprising a versioned file system local to that entity. The method begins by forming a sharing group that includes two or more entities. Sharing of the structured data representations by members of the sharing group is enabled. The filers use a single distributed lock to protect each version of the file system. This lock is managed to allow each filer access to the shared file system volume to create its new version. To share a fully-versioned file system, asynchronous updates at each of the filers is permitted, and each node is then allowed to “push” its individual changes to the store to form the next version of the file system. A mechanism also may be used to reduce the period during which filers in the group operate under lock.

Abstract: A versioned file system comprises a set of structured data representations, such as XML. Each structured data representation corresponds to a “version,” and each version comprises a tree of write-once objects rooted at a root directory manifest. Each version in the versioned file system has associated therewith a “borrow window.” When it is desired to reconstruct the file system to a point in time (or, more generally, a given state), i.e., to perform a “restore,” it is only required to walk (use) a single structured data representation (a tree). During a restore, metadata is pulled back from the cloud first, so users can see the existence of needed files immediately. The remainder of the data is then pulled back from the cloud if/when the user goes to open the file. As a result, the entire file system (or any portion thereof) can be restored to a previous time nearly instantaneously.

Abstract: An interface between a local file system and a data store (e.g., a “write-once”store) generates and exports to the data store a “versioned” file system. The state of the local file system at a given point in time can be determined using the versioned file system. The versioned file system comprises a set of structured data representations that comprise metadata associated with actual file system data. The interface creates and exports to the data store structured data representations, which correspond to versions of the local file system, as well as the actual file system data that these structured data representations represent. A cache associated with the interface stores recently used data and metadata, as well as write data and metadata waiting to be transferred to the data store. Prior to transfer, data transformations, such as compression, encryption, and de-duplication, may be applied to the data and metadata.

Abstract: A method of data sharing among multiple entities is provided. Each entity exports to a data store a structured data representation comprising a versioned file system local to that entity. The method begins by forming a sharing group that includes two or more entities. Sharing of the structured data representations by members of the sharing group is enabled. The filers use a single distributed lock to protect each version of the file system. This lock is managed to allow each filer access to the shared file system volume to create its new version. To share a fully-versioned file system, asynchronous updates at each of the filers is permitted, and each node is then allowed to “push” its individual changes to the store to form the next version of the file system. A mechanism also may be used to reduce the period during which filers in the group operate under lock.

Abstract: A versioned file system comprises a set of structured data representations. At a first time, an interface creates and exports to a cloud data store a first structured data representation corresponding to a first version of the local file system. The first structured data representation is an XML tree having a root element, one or more directory elements associated with the root element, and one or more file elements associated with a given directory element. Upon a change within the file system, the interface creates and exports a second structured data representation corresponding to a second version of the file system. The second structured data representation differs from the first structured data representation up to and including the root element of the second structured data representation. The interface continues to generate and export the structured data representations to the data store.

Abstract: A versioned file system comprises a set of structured data representations, such as XML. Each structured data representation corresponds to a “version,” and each version comprises a tree of write-once objects rooted at a root directory manifest. Each version in the versioned file system has associated therewith a “borrow window.” When it is desired to reconstruct the file system to a point in time (or, more generally, a given state), i.e., to perform a “restore,” it is only required to walk (use) a single structured data representation (a tree). During a restore, metadata is pulled back from the cloud first, so users can see the existence of needed files immediately. The remainder of the data is then pulled back from the cloud if/when the user goes to open the file. As a result, the entire file system (or any portion thereof) can be restored to a previous time nearly instantaneously.

Abstract: A method of data sharing among multiple entities is provided. Each entity exports to a data store a structured data representation comprising a versioned file system local to that entity. The method begins by forming a sharing group that includes two or more entities. Sharing of the structured data representations by members of the sharing group is enabled. The filers use a single distributed lock to protect each version of the file system. This lock is managed to allow each filer access to the shared file system volume to create its new version. To share a fully-versioned file system, asynchronous updates at each of the filers is permitted, and each node is then allowed to “push” its individual changes to the store to form the next version of the file system. A mechanism also may be used to reduce the period during which filers in the group operate under lock.

Abstract: A versioned file system comprises a set of structured data representations. At a first time, an interface creates and exports to a data store a first structured data representation corresponding to a first version of a local file system. The first structured data representation is an XML tree having a root element, one or more directory elements associated with the root element, and one or more file elements associated with a given directory element. Upon a change within the file system (e.g., file creation, file deletion, file modification, directory creation, directory deletion and directory modification), the interface creates and exports a second structured data representation corresponding to a second version of the file system. The second structured data representation differs from the first structured data representation up to and including the root element of the second structured data representation. The data store may comprise a cloud storage service provider.