Abstract: Example distributed storage systems, controller nodes, and methods provide hierarchical blacklisting of storage system components in response to failed storage requests. Storage elements are accessible through hierarchical storage paths traversing multiple system components. Blacklisted components are aggregated and evaluated against a hierarchy threshold at each level of the hierarchy and all components below the component are blacklisted if the hierarchy threshold is met. Blacklisted components are avoided during subsequent storage requests.

Abstract: Example distributed storage systems, controller nodes, and methods provide distributed and redundant data blocks accessed based on storage path cost values. Storage elements are accessible through hierarchical storage paths traversing multiple system components. Data blocks are distributed among the storage elements. System costs are calculated based on the storage path for reaching each storage element and a storage path is selected based on a comparison of the system costs for each storage element. Data blocks are accessed through the selected storage path.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of data functions, such as map-reduce functions, using overlapping symbols. Storage nodes are configured to partition data units into symbols that include an overlap data portion of an adjacent symbol and erasure encode the symbols. The storage nodes may then decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, and process the subunits using map-functions to generate intermediate contexts. A reduce-function may be used to determine a function result using the intermediate contexts.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of data functions, such as serial functions. Storage nodes are configured to partition decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, process the subunits using a serial function to generate intermediate contexts, and send the intermediate context to a next storage node for continued processing using the serial function.

Abstract: Example distributed storage systems, controller nodes, and methods provide selective allocation of redundant data blocks to background operations. Background operations may be identified targeting a data unit stored in redundant data blocks in a storage pool with a plurality of storage elements. A subset of data units may be selected for the background operation and the system components including those data units may be isolated. Data requests to the isolated system components may be selectively prevented while the background operation executes on the subset of data units in the isolated system components.

Abstract: Systems and methods for distributed storage systems using dynamic spreading policies are described. Distributed storage elements may be accessed using various storage paths through hierarchically organized storage system components to store data blocks corresponding to data elements. A higher priority hierarchical spreading policy is selected for determining the storage elements to receive the data blocks. If the first hierarchical spreading policy is determined not to have been met, a lower priority hierarchical spreading policy is selected and the data blocks are stored according to the lower priority hierarchical spreading policy. Data block stored at the lower priority hierarchical spreading policy may automatically be migrated to the higher priority hierarchical spreading policy.

Abstract: Example storage systems, storage nodes, and methods provide parallel storage node processing of data functions, such as map-reduce functions. Storage nodes are configured to decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, and process the subunits in parallel using map-functions to generate intermediate contexts. A reduce-function may be used to determine a function result using the intermediate contexts.

Abstract: Example distributed storage systems, controller nodes, and methods provide hierarchical blacklisting of storage system components in response to failed storage requests. Storage elements are accessible through hierarchical storage paths traversing multiple system components. Blacklisted components are aggregated and evaluated against a hierarchy threshold at each level of the hierarchy and all components below the component are blacklisted if the hierarchy threshold is met. Blacklisted components are avoided during subsequent storage requests.

Abstract: Example distributed storage systems, controller nodes, and methods provide selective allocation of redundant data blocks to background operations. Background operations may be identified targeting a data unit stored in redundant data blocks in a storage pool with a plurality of storage elements. A subset of data units may be selected for the background operation and the system components including those data units may be isolated. Data requests to the isolated system components may be selectively prevented while the background operation executes on the subset of data units in the isolated system components.

Abstract: Example distributed storage systems, controller nodes, and methods provide distributed and redundant data blocks accessed based on storage path cost values. Storage elements are accessible through hierarchical storage paths traversing multiple system components. Data blocks are distributed among the storage elements. System costs are calculated based on the storage path for reaching each storage element and a storage path is selected based on a comparison of the system costs for each storage element. Data blocks are accessed through the selected storage path.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of predefined data functions, such as map-reduce functions. Storage nodes storing a plurality of symbols for a data unit are configured to select a predefined function using a data type of the data unit. Each storage node identifies subunits of the data unit from the symbols and processes the subunits using the predefined function to generate function results. A final result is returned based on the function results from each storage node.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of data functions, such as serial functions. Storage nodes are configured to partition decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, process the subunits using a serial function to generate intermediate contexts, and send the intermediate context to a next storage node for continued processing using the serial function.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of predefined data functions, such as map-reduce functions. Storage nodes storing a plurality of symbols for a data unit are configured to select a predefined function using a data type of the data unit. Each storage node identifies subunits of the data unit from the symbols and processes the subunits using the predefined function to generate function results. A final result is returned based on the function results from each storage node.

Abstract: Example storage systems, storage nodes, and methods provide storage node processing of data functions, such as map-reduce functions, using overlapping symbols. Storage nodes are configured to partition data units into symbols that include an overlap data portion of an adjacent symbol and erasure encode the symbols. The storage nodes may then decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, and process the subunits using map-functions to generate intermediate contexts. A reduce-function may be used to determine a function result using the intermediate contexts.

Abstract: Example storage systems, storage nodes, and methods provide parallel storage node processing of data functions, such as map-reduce functions. Storage nodes are configured to decode erasure encoded symbols, identify subunits of a data unit from the decoded symbols, and process the subunits in parallel using map-functions to generate intermediate contexts. A reduce-function may be used to determine a function result using the intermediate contexts.

Abstract: Systems and methods for distributed storage systems using dynamic spreading policies are described. Distributed storage elements may be accessed using various storage paths through hierarchically organized storage system components to store data blocks corresponding to data elements. A higher priority hierarchical spreading policy is selected for determining the storage elements to receive the data blocks. If the first hierarchical spreading policy is determined not to have been met, a lower priority hierarchical spreading policy is selected and the data blocks are stored according to the lower priority hierarchical spreading policy. Data block stored at the lower priority hierarchical spreading policy may automatically be migrated to the higher priority hierarchical spreading policy.

Abstract: Example distributed storage systems, file system interfaces, and methods provide cache coherence management. A system receives a file data request including a file data reference and identifies a data cache location with a coherence value for the file data reference. The system queries a reference data store for a coherence reference corresponding to the file data reference and compares the coherence value to the coherence reference. In response to the coherence value matching the coherence reference, the system executes the file data request using the data cache location.

Abstract: Example distributed storage systems, file system interfaces, and methods provide cache coherence management. A system receives a file data request including a file data reference and identifies a data cache location with a coherence value for the file data reference. The system queries a reference data store for a coherence reference corresponding to the file data reference and compares the coherence value to the coherence reference. In response to the coherence value matching the coherence reference, the system executes the file data request using the data cache location.