Abstract: A method of distributing data in a distributed storage system includes receiving a file, dividing the received file into chunks, and determining a distribution of the chunks among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance levels, and each maintenance level includes one or more maintenance units. Each maintenance unit has an active state and an inactive state. Moreover, each storage device is associated with a maintenance unit. The determining of the distribution of the chunks includes identifying a random selection of the storage devices matching a number of chunks of the file and being capable of maintaining accessibility of the file when one or more maintenance units are in an inactive state. The method also includes distributing the chunks to storage devices of the distributed storage system according to the determined distribution.

Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.

Abstract: A number of load units are connected to receive power from a number of power supply units. A potential load bus is connected to have a voltage level representative of both a total potential power requirement of the number of load units and a total potential power supply capability of the number of power supply units. A first control circuit enables operation of the number of load units when the voltage level on the potential load bus indicates that a sufficient supply of power is available. An actual load bus is connected to have a voltage level representative of both an actual total power consumption of the number of load units and an actual total power supply available from of the number of power supply units. A second control circuit signals an impending loss of sufficient power supply based on the monitored voltage level on the actual load bus.

Abstract: A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.

Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks using a computer processor in communication with the non-transitory memory. The method also includes distributing chunks to storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance units each having active and inactive states. Moreover, each storage device is associated with a maintenance unit. The chunks are distributed across multiple maintenance units to maintain accessibility of the file when a maintenance unit is in an inactive state.

Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.

Abstract: A method for efficiently using resources (e.g., memory devices) in data centers of a distributed storage system includes identifying high-availability jobs and low-availability jobs that demand usage of resources of the distributed system. The method further includes allocating resource usage to the jobs, determining a first load of the jobs on resources available during a failure event, and determining a second load of the jobs on the resources lost during the failure event. The method includes determining a scaled third load of the jobs on the resources available during the failure event based on the first and second loads and reallocating resource usage assigned to the low-availability jobs to the high-availability jobs during the failure event. The reallocation is associated with the scaled third load of the jobs.

Abstract: A number of load units are connected to receive power from a number of power supply units. A potential load bus is connected to have a voltage level representative of both a total potential power requirement of the number of load units and a total potential power supply capability of the number of power supply units. A first control circuit enables operation of the number of load units when the voltage level on the potential load bus indicates that a sufficient supply of power is available. An actual load bus is connected to have a voltage level representative of both an actual total power consumption of the number of load units and an actual total power supply available from of the number of power supply units. A second control circuit signals an impending loss of sufficient power supply based on the monitored voltage level on the actual load bus.

Abstract: A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.

Abstract: A method includes identifying high-availability jobs and low-availability jobs that demand usage of resources of a distributed system. The method includes determining a first quota of the resources available to low-availability jobs as a quantity of the resources available during normal operations, and determining a second quota of the resources available to high-availability jobs as a quantity of the resources available during normal operations minus a quantity of the resources lost due to a tolerated event. The method includes executing the jobs on the distributed system and constraining a total usage of the resources by both the high-availability jobs and the low-availability jobs to the quantity of the resources available during normal operations.

Abstract: A method that prevents unnecessary data recovery includes receiving, at a data processing device, a status of a resource of a distributed system. When the status of the resource indicates a resource failure, the method includes executing instructions on the data processing device to determine whether the resource failure is correlated to any other resource failures within the distributed system. When the resource failure is correlated to other resource failures within the distributed system, the method includes delaying execution on the data processing device of a remedial action associated with the resource. However, when the resource failure is uncorrelated to other resource failures within the distributed system, the method includes initiating execution on the data processing device of the remedial action associated with the resource.

Abstract: A method includes identifying high-availability jobs and low-availability jobs that demand usage of resources of a distributed system. The method includes determining a first quota of the resources available to low-availability jobs as a quantity of the resources available during normal operations, and determining a second quota of the resources available to high-availability jobs as a quantity of the resources available during normal operations minus a quantity of the resources lost due to a tolerated event. The method includes executing the jobs on the distributed system and constraining a total usage of the resources by both the high-availability jobs and the low-availability jobs to the quantity of the resources available during normal operations.

Abstract: Methods for configuring networks and systems including nodes and switches. In one aspect, a method includes selecting a (v, b, r, k, ?) balanced incomplete block design (BIBD), wherein the BIBD includes v elements and b blocks of k elements, each element of the v elements is in r blocks of the b blocks, and each distinct pair of elements of the v elements is in ? blocks of the b blocks. Nodes are connected to switches. Each node is assigned to a different element of the BIBD. Each switch is assigned to a different block of the BIBD. For each switch, the switch is connected to the nodes assigned to the elements included in the block assigned to the switch.

Abstract: A method includes identifying high-availability jobs and low-availability jobs that demand usage of resources of a distributed system. The method includes determining a first quota of the resources available to low-availability jobs as a quantity of the resources available during normal operations, and determining a second quota of the resources available to high-availability jobs as a quantity of the resources available during normal operations minus a quantity of the resources lost due to a tolerated event. The method includes executing the jobs on the distributed system and constraining a total usage of the resources by both the high-availability jobs and the low-availability jobs to the quantity of the resources available during normal operations.

Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method also includes grouping one or more of the data chunks and one or more of the non-data chunks in a group. One or more chunks of the group is capable of being reconstructed from other chunks of the group. The method also includes distributing the chunks of the group to storage devices of the distributed storage system based on a hierarchy of the distributed storage system. The hierarchy includes maintenance domains having active and inactive states, each storage device associated with a maintenance domain, the chunks of a group are distributed across multiple maintenance domains to maintain the ability to reconstruct chunks of the group when a maintenance domain is in an inactive state.

Abstract: A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.

Abstract: A method of distributing data in a distributed storage system includes receiving a file into non-transitory memory and dividing the received file into chunks using a computer processor in communication with the non-transitory memory. The method also includes distributing chunks to storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes maintenance units each having active and inactive states. Moreover, each storage device is associated with a maintenance unit. The chunks are distributed across multiple maintenance units to maintain accessibility of the file when a maintenance unit is in an inactive state.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for storing data reliably across groups of storage nodes. In one aspect, a method includes receiving (n?f) data chunks for storage across n groups of storage nodes and generating (f?1) error-correcting code chunks using an error-correcting code and the (n?f) data chunks. The (n?f) data chunks are stored at a first group of storage nodes. Each data chunk of the (n?f) data chunks is stored at a respective second group of storage nodes. Each code chunk of the (f?1) code chunks is stored at a respective third group of storage nodes. Each second group of storage nodes and each third group of storage nodes is distinct from each other and from the first group of storage nodes.

Abstract: Methods, systems, and apparatus, including computer program products, for orthogonal coding for data storage. In one aspect, a method includes receiving a block of data comprising m rows and n columns of data chunks. For each row in the block of data, (c-n) columns of error-correcting row code chunks are generated using a first linear error-correcting code in systematic form and the particular row's data chunks. For each column in the block of data and for each generated column, particular column and (r-m) error-correcting column code chunks for the particular column are allocated to a distinct group of storage nodes, wherein: the column code chunks are generated using a second linear error-correcting code in systematic form and the particular column's data chunks or row code chunks; m and n are greater than one; and c is greater than n and r is greater than m.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating the stack distance of an access to a cache. In one aspect, a method includes determining a current estimate value of the stack distance. For each of K buckets, a minimum value and a maximum value are initialized, wherein each bucket corresponds to a range of possible fingerprint values for accesses. A first access is processed, including: determining a first fingerprint value for the first access; identifying a first bucket for the first access based on the first fingerprint value; and determining that the first fingerprint value is between the minimum value and the maximum value for the first bucket, and in response, increasing the current estimate value and adjusting the minimum value or the maximum value for the first bucket.