Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: Systems, methods, and non-transitory computer readable media for storing and processing metadata corresponding to files are presented. The system comprises first, second, and third computing nodes. The computing nodes comprise a plurality of slices of a metadata database that is separate and independent from file storage, and each slice is configured to store metadata. The computing nodes further comprise a slice route table that indicates primary and secondary locations of each unique slice in the plurality of slices. The slice route table comprises a plurality of entries, each of which comprise a slice number corresponding to a slice in the plurality of slices of the metadata database, a primary computing node number corresponding to a primary computing node that comprises the slice and corresponds to the primary location, and a secondary computing node number corresponding to a secondary computing node that comprises the slice and corresponds to the secondary location.

Abstract: Systems, methods, and non-transitory computer readable media for profiling messages between multiple computing cores are presented. A first computing core generates a first query message comprising a message header and a message payload. The message header comprises a profiling bit based on a profiling periodicity parameter. The first computing core generates a first set of shadow events corresponding to the first query message. A second computing core receives the first set of shadow events, generates a timestamp for each of the shadow events based on a time source that is local to the second computing core, and determines if each of the shadow events corresponds to a receive event. The second computing core correlates, based on the determining, each of the shadow events with the first query message, and calculates a first latency of the first query message based on the timestamps of the correlated shadow events.

Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: Each node and volume in a storage cluster makes a decision whether to reduce power consumption based on lack of requests from client applications and nodes over a time period. Node configuration parameters determine how long to wait until idling a node or volume, and how long to wait while idle before performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to ensure that idle nodes are kept idle. In an archive mode, writes bids are reversed, nodes with less capacity submit lower bids, fuller nodes fill up faster and are then idled, while empty or near empty nodes may remain idle before winning a write bid.

Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: Each node and volume in a storage cluster makes a decision whether to reduce power consumption based on lack of requests from client applications and nodes over a time period. Node configuration parameters determine how long to wait until idling a node or volume, and how long to wait while idle before performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to ensure that idle nodes are kept idle. In an archive mode, writes bids are reversed, nodes with less capacity submit lower bids, fuller nodes fill up faster and are then idled, while empty or near empty nodes may remain idle before winning a write bid.

Abstract: Each node and volume in a storage cluster makes a decision whether to reduce power consumption based on lack of requests from client applications and nodes over a time period. Node configuration parameters determine how long to wait until idling a node or volume, and how long to wait while idle before performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to ensure that idle nodes are kept idle. In an archive mode, writes bids are reversed, nodes with less capacity submit lower bids, fuller nodes fill up faster and are then idled, while empty or near empty nodes may remain idle before winning a write bid.

Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: Each node and volume in a fixed-content storage cluster makes an independent decision whether to reduce power consumption based on lack of requests from client applications and nodes over a configurable time period. Node configuration parameters sleepAfter and wakeAfter respectively determine how long to wait until idling a node or volume, and how long to wait while idle before again performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy of the file. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to help ensure that idle nodes are kept idle. Normally, nodes with more capacity will submit a lower bid to write a file.

Abstract: Each node and volume in a storage cluster makes a decision whether to reduce power consumption based on lack of requests from client applications and nodes over a time period. Node configuration parameters determine how long to wait until idling a node or volume, and how long to wait while idle before performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to ensure that idle nodes are kept idle. In an archive mode, writes bids are reversed, nodes with less capacity submit lower bids, fuller nodes fill up faster and are then idled, while empty or near empty nodes may remain idle before winning a write bid.

Abstract: A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

Abstract: Each node and volume in a storage cluster makes a decision whether to reduce power consumption based on lack of requests from client applications and nodes over a time period. Node configuration parameters determine how long to wait until idling a node or volume, and how long to wait while idle before performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to ensure that idle nodes are kept idle. In an archive mode, writes bids are reversed, nodes with less capacity submit lower bids, fuller nodes fill up faster and are then idled, while empty or near empty nodes may remain idle before winning a write bid.

Abstract: Each node and volume in a storage cluster makes a decision whether to reduce power consumption based on lack of requests from client applications and nodes over a time period. Node configuration parameters determine how long to wait until idling a node or volume, and how long to wait while idle before performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to ensure that idle nodes are kept idle. In an archive mode, writes bids are reversed, nodes with less capacity submit lower bids, fuller nodes fill up faster and are then idled, while empty or near empty nodes may remain idle before winning a write bid.

Abstract: Each node and volume in a fixed-content storage cluster makes an independent decision whether to reduce power consumption based on lack of requests from client applications and nodes over a configurable time period. Node configuration parameters sleepAfter and wakeAfter respectively determine how long to wait until idling a node or volume, and how long to wait while idle before again performing integrity checks. A bid value is calculated by each node and reflects how much it will cost for that node to write a file, read a file, or keep a copy of the file. A node with the lowest bid wins, and nodes that are idle have a premium added to each bid to help ensure that idle nodes are kept idle. Normally, nodes with more capacity will submit a lower bid to write a file.