Abstract: Example implementations are directed to restoration of data not only from the servers, but also from the clients connected to the server. Algorithms are incorporated to identify content that was created or modified after the last backup. This algorithm also identifies and resolves changes in the mount points for shared folders, preventing information leakage. When the server recovers from a failure, it notifies the clients about the recovery on their next connection. Each client then determines the current state of its mount points and file paths and compares them with the server's mount points and file paths. After the comparison, the client comprehends and indicates the mount point differences by renaming them and sends entire local data (all files, folders, mount points) to the server. The server reconciles the differences.

Abstract: A method, of managing storage of content of a system in storage units including run units which do not allow spin-down functionality and spin-down units which allow spin-down functionality, comprises: establishing a storage tiering rule which sets a policy indicating what content is eligible to be stored on a run unit and what content is eligible to be stored on a spin-down unit, the storage tiering rule being applicable to at least a group of the contents in the content system to determine stored content eligibility thereof for storage in the storage units and migration between the storage units; and identifying candidates of contents to migrate between the storage units, based on the storage tiering rule, state of the content system, and the stored content eligibility of the at least a group of the contents.

Abstract: Example implementations described herein are directed to a transaction query engine involving a token that is generated by the server and sent to the corresponding the client. The client uses the token to query transactions made on the server. The token can include mountpoint information detailing shared and private information of the filesystems, transaction identifiers identifying transactions of a filesystem, and a filesystem identifier to identify the corresponding filesystem. The server processes this token and updates the token based on the transaction information of the filesystem, and sends the token back to the corresponding client.

Abstract: An “operate with missing region” feature allows a cluster to continue servicing reads for available regions even when some regions are missing. Upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. Typically, “read-only” means no client write or update requests succeed while the cluster is in this state. Those requests should fail with an error message that indicates failure because of the read-only status. Preferably, such requests are re-tried. In this mode, regions are only allowed to perform read operations but the cluster continues to operate with missing regions, and missing regions are entered on a region map. The cluster then automatically recovers returning missing regions, after which it leaves the read-only state.

Abstract: A method, of managing storage of content of a system in storage units including run units which do not allow spin-down functionality and spin-down units which allow spin-down functionality, comprises: establishing a storage tiering rule which sets a policy indicating what content is eligible to be stored on a run unit and what content is eligible to be stored on a spin-down unit, the storage tiering rule being applicable to at least a group of the contents in the content system to determine stored content eligibility thereof for storage in the storage units and migration between the storage units; and identifying candidates of contents to migrate between the storage units, based on the storage tiering rule, state of the content system, and the stored content eligibility of the at least a group of the contents.

Abstract: A method, of managing storage of content of a system (400) in storage units (430) including run units which do not allow spin-down functionality and spin-down units which allow spin-down functionality, comprises: establishing a storage tiering rule (702) which sets a policy indicating what content is eligible to be stored on a run unit and what content is eligible to be stored on a spin-down unit, the storage tiering rule being applicable to at least a group of the contents in the content system to determine stored content eligibility thereof for storage in the storage units and migration between the storage units; and identifying candidates of contents to migrate between the storage units (706), based on the storage tiering rule, state of the content system, and the stored content eligibility of the at least a group of the contents.

Abstract: A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into systems each having multiple nodes. Objects in the systems have access control lists (ACLs) associated therewith. A method of providing a search key to each ACL in any one system comprises: selecting a region for each metadata object pertaining to the objects in the system; and generating, for each ACL associated with an object in the system, an ACL ID (identifier) as a search key which includes the region number of the region selected for the metadata object pertaining to the object, a region map level corresponding to the region selected, and a monotonically increasing counter value that is automatically incremented each time an ACL ID is generated. The ACL ID remains unique during reconfiguration via splitting/merging existing distributed portions of the database.

Abstract: Example implementations are directed to restoration of data not only from the servers, but also from the clients connected to the server. Algorithms are incorporated to identify content that was created or modified after the last backup. This algorithm also identifies and resolves changes in the mount points for shared folders, preventing information leakage. When the server recovers from a failure, it notifies the clients about the recovery on their next connection. Each client then determines the current state of its mount points and file paths and compares them with the server's mount points and file paths. After the comparison, the client comprehends and indicates the mount point differences by renaming them and sends entire local data (all files, folders, mount points) to the server. The server reconciles the differences.

Abstract: Example implementations described herein are directed to a transaction query engine involving a token that is generated by the server and sent to the corresponding the client. The client uses the token to query transactions made on the server. The token can include mountpoint information detailing shared and private information of the filesystems, transaction identifiers identifying transactions of a filesystem, and a filesystem identifier to identify the corresponding filesystem. The server processes this token and updates the token based on the transaction information of the filesystem, and sends the token back to the corresponding client.

Abstract: An “operate with missing region” feature allows a cluster to continue servicing reads for available regions even when some regions are missing. Upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. Typically, “read-only” means no client write or update requests succeed while the cluster is in this state. Those requests should fail with an error message that indicates failure because of the read-only status. Preferably, such requests are re-tried. In this mode, regions are only allowed to perform read operations but the cluster continues to operate with missing regions, and missing regions are entered on a region map. The cluster then automatically recovers returning missing regions, after which it leaves the read-only state.

Abstract: An “operate with missing region” feature allows a cluster to continue servicing reads for available regions even when some regions are missing. Upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. Typically, “read-only” means no client write or update requests succeed while the cluster is in this state. Those requests should fail with an error message that indicates failure because of the read-only status. Preferably, such requests are re-tried. In this mode, regions are only allowed to perform read operations but the cluster continues to operate with missing regions, and missing regions are entered on a region map. The cluster then automatically recovers returning missing regions, after which it leaves the read-only state.

Abstract: A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into systems each having multiple nodes. Objects in the systems have access control lists (ACLs) associated therewith. A method of providing a search key to each ACL in any one system comprises: selecting a region for each metadata object pertaining to the objects in the system; and generating, for each ACL associated with an object in the system, an ACL ID (identifier) as a search key which includes the region number of the region selected for the metadata object pertaining to the object, a region map level corresponding to the region selected, and a monotonically increasing counter value that is automatically incremented each time an ACL ID is generated. The ACL ID remains unique during reconfiguration via splitting/merging existing distributed portions of the database.

Abstract: A method, of managing storage of content of a system (400) in storage units (430) including run units which do not allow spin-down functionality and spin-down units which allow spin-down functionality, comprises: establishing a storage tiering rule (702) which sets a policy indicating what content is eligible to be stored on a run unit and what content is eligible to be stored on a spin-down unit, the storage tiering rule being applicable to at least a group of the contents in the content system to determine stored content eligibility thereof for storage in the storage units and migration between the storage units; and identifying candidates of contents to migrate between the storage units (706), based on the storage tiering rule, state of the content system, and the stored content eligibility of the at least a group of the contents.

Abstract: This disclosure enables a user to incrementally increase or decrease the size of the cluster and, in particular, to incrementally increase or decrease the number of map regions implemented in a metadata manager associated with the cluster while maintaining availability of the cluster. In a representative embodiment, a configurable target map size is identified, either manually (e.g., by the user) or programmatically. If the target map size is found to be within an acceptable value (given the available node configuration), a procedure to create a new map is triggered. The new map preferably is generated as follows. Initially, one or more new temporary region(s) are created either to split or join the existing regions (depending on whether the map is being increased or decreased). For each temporary region, the process also generates MDPL backup regions. Metadata from an original region (or regions) is then migrated to the temporary region(s).

Abstract: An “operate with missing region” feature allows a cluster to continue servicing reads for available regions even when some regions are missing. Upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. Typically, “read-only” means no client write or update requests succeed while the cluster is in this state. Those requests should fail with an error message that indicates failure because of the read-only status. Preferably, such requests are re-tried. In this mode, regions are only allowed to perform read operations but the cluster continues to operate with missing regions, and missing regions are entered on a region map. The cluster then automatically recovers returning missing regions, after which it leaves the read-only state.

Abstract: Cluster recovery time is reduced by an incremental refresh technique that is described herein. The goal of the technique is to salvage the portion of the redundant distributed database lost (e.g., during a fault) by performing an incremental refresh of only the updates occurring to that portion of the database since the time of loss.

Abstract: An “operate with missing region” feature of this disclosure allows the cluster to continue servicing reads for available regions even when some regions are missing. In particular, upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. As used herein, “read-only” means that no client write or update requests will succeed while the cluster is in this state. In particular, those requests should fail with an error message (or the like) that indicates failure because of the read-only status. Preferably, such requests are then re-tried. In this mode, all regions are only allowed to perform read operations. During the read-only state, the cluster continues to operate with missing regions, and missing regions are entered on the region map.

Abstract: This disclosure enables a user to incrementally increase or decrease the size of the cluster and, in particular, to incrementally increase or decrease the number of map regions implemented in a metadata manager associated with the cluster while maintaining availability of the cluster. In a representative embodiment, a configurable target map size is identified, either manually (e.g., by the user) or programmatically. If the target map size is found to be within an acceptable value (given the available node configuration), a procedure to create a new map is triggered. The new map preferably is generated as follows. Initially, one or more new temporary region(s) are created either to split or join the existing regions (depending on whether the map is being increased or decreased). For each temporary region, the process also generates MDPL backup regions. Metadata from an original region (or regions) is then migrated to the temporary region(s).

Abstract: An “operate with missing region” feature of this disclosure allows the cluster to continue servicing reads for available regions even when some regions are missing. In particular, upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. As used herein, “read-only” means that no client write or update requests will succeed while the cluster is in this state. In particular, those requests should fail with an error message (or the like) that indicates failure because of the read-only status. Preferably, such requests are then re-tried. In this mode, all regions are only allowed to perform read operations. During the read-only state, the cluster continues to operate with missing regions, and missing regions are entered on the region map.

Abstract: Cluster recovery time is reduced by an incremental refresh technique that is described herein. The goal of the technique is to salvage the portion of the redundant distributed database lost (e.g., during a fault) by performing an incremental refresh of only the updates occurring to that portion of the database since the time of loss.