Abstract: Techniques are provided for offloading the management of sensor data and generating custom views of sensor data. Sensor data received from a data network through a message is stored within storage managed by a computing device. A handle is generated to identify the sensor data. The sensor data within the message is replaced with the handle, and the message is transmitted to a device within the data network. The device may use handles of sensor data to request custom views of sensor data.

Abstract: Techniques are provided for offloading the management of sensor data and generating custom views of sensor data. Sensor data received from a data network through a message is stored within storage managed by a computing device. A handle is generated to identify the sensor data. The sensor data within the message is replaced with the handle, and the message is transmitted to a device within the data network. The device may use handles of sensor data to request custom views of sensor data.

Abstract: Techniques are provided for offloading the management of sensor data and generating custom views of sensor data. Sensor data received from a data network through a message is stored within storage managed by a computing device. A handle is generated to identify the sensor data. The sensor data within the message is replaced with the handle, and the message is transmitted to a device within the data network. The device may use handles of sensor data to request custom views of sensor data.

Abstract: Techniques are provided for offloading the management of sensor data and generating custom views of sensor data. Sensor data received from a data network through a message is stored within storage managed by a computing device. A handle is generated to identify the sensor data. The sensor data within the message is replaced with the handle, and the message is transmitted to a device within the data network. The device may use handles of sensor data to request custom views of sensor data.

Abstract: A rebuild node of a storage system can assess risk of the storage system not being able to provide a data object. The rebuild node(s) uses information about data object fragments to determine health of a data object, which relates to the risk assessment. The rebuild node obtains object fragment information from nodes throughout the storage system. With the object fragment information, the rebuild node(s) can assess object risk based, at least in part, on the object fragments indicated as existing by the nodes. To assess object risk, the rebuild node(s) treats absent object fragments (i.e., those for which an indication was not received) as lost. When too many object fragments are lost, an object cannot be rebuilt. The erasure coding technique dictates the threshold number of fragments for rebuilding an object. The risk assessment per object influences rebuild of the objects.

Abstract: A method includes receiving a write request to write a current data block to a Shingled Magnetic Recording (SMR) storage device. In response, the current data block is written to a current physical block in an open zone of the SMR storage device. A corresponding copy of the current data block is written to a nonvolatile memory. A determination is made of whether a wandering write error occurred during the writing of the data to the open zone. In response to the wandering write error occurring, for each of the number of written physical blocks in the open zone that have the corresponding copy in the nonvolatile memory, the data in the physical block is validated using the corresponding copy. In response to validation, the data in the corresponding copy is written as corrected data for the physical block to a new zone in the SMR storage device.

Abstract: A storage system writes an object across zones of a set of zones (“zone set”). Each zone of a zone set is contributed from an independently accessible storage medium. To create a zone set, the storage system arbitrarily selects disks to contribute a zone for membership in the zone set. This results in a fairly even distribution of zone sets throughout the storage system, which increases fault tolerance of the storage system. Although disk selection for zone set membership is arbitrary, the arbitrary selection can be from a pool of disks that satisfy one or more criteria (e.g., health or activity based criteria). In addition, weights can be assigned to disks to influence the arbitrary selection. Although manipulating the arbitrary selection with weights or by reducing the pool of disks reduces the arbitrariness, this evenly distributes zone sets while accounting for client demand and/or disk health.

Abstract: A write request is received to write a data block having a logical block address to a nonvolatile storage device. The method includes writing a value of the data block to the nonvolatile storage device. The writing includes locating a position in a tree-based data structure that includes first and second nodes. The first node is configured to store a first set of data blocks having logical block addresses in a first numerical range, and the second node is configured to store a second set of data blocks having logical block addresses in a second numerical range. The position is located in the first node or the second node depending on the value of the logical block address. The writing includes storing the value of the data block in the position in the tree-based data structure.

Abstract: A rebuild node of a storage system can assess risk of the storage system not being able to provide a data object. The rebuild node(s) uses information about data object fragments to determine health of a data object, which relates to the risk assessment. The rebuild node obtains object fragment information from nodes throughout the storage system. With the object fragment information, the rebuild node(s) can assess object risk based, at least in part, on the object fragments indicated as existing by the nodes. To assess object risk, the rebuild node(s) treats absent object fragments (i.e., those for which an indication was not received) as lost. When too many object fragments are lost, an object cannot be rebuilt. The erasure coding technique dictates the threshold number of fragments for rebuilding an object. The risk assessment per object influences rebuild of the objects.

Abstract: A storage system writes an object across zones of a set of zones (“zone set”). Each zone of a zone set is contributed from an independently accessible storage medium. To create a zone set, the storage system arbitrarily selects disks to contribute a zone for membership in the zone set. This results in a fairly even distribution of zone sets throughout the storage system, which increases fault tolerance of the storage system. Although disk selection for zone set membership is arbitrary, the arbitrary selection can be from a pool of disks that satisfy one or more criteria (e.g., health or activity based criteria). In addition, weights can be assigned to disks to influence the arbitrary selection. Although manipulating the arbitrary selection with weights or by reducing the pool of disks reduces the arbitrariness, this evenly distributes zone sets while accounting for client demand and/or disk health.

Abstract: A method includes receiving a write request to write a current data block to a Shingled Magnetic Recording (SMR) storage device. In response, the current data block is written to a current physical block in an open zone of the SMR storage device. A corresponding copy of the current data block is written to a nonvolatile memory. A determination is made of whether a wandering write error occurred during the writing of the data to the open zone. In response to the wandering write error occurring, for each of the number of written physical blocks in the open zone that have the corresponding copy in the nonvolatile memory, the data in the physical block is validated using the corresponding copy. In response to validation, the data in the corresponding copy is written as corrected data for the physical block to a new zone in the SMR storage device.

Abstract: A durable file system has been designed for storage devices that do not support write in place and/or that are susceptible to errors or failures. The durable file system also facilitates organization and access of large objects (e.g., gigabytes to terabytes in size). Since the write of a large object often involves multiple write operations, the writing is also referred to as “ingesting.” When ingesting an object, the durable file system writes the object with indexing information for the object to persistent storage across multiple zones that each map to an independently accessible storage medium (e.g., disks on different spindles). After persisting the indexing information with the object, the durable file system updates a file system index in working memory (e.g., non-volatile system memory) with the indexing information for the object.

Abstract: A durable file system has been designed for storage devices that do not support write in place and/or that are susceptible to errors or failures. The durable file system also facilitates organization and access of large objects (e.g., gigabytes to terabytes in size). Regardless of whether target storage devices are configured with sequential write constraints, the durable file system writes object fragments across a set of sequences or ranges of storage units, such as logical blocks. The durable file system sequentially writes an object fragment into each storage unit sequence along with indexing information for the object fragments. In addition to writing the indexing information for the object fragments into the set of storage unit sequences, the durable file system updates the file system index with the object indexing information.

Abstract: A write request is received to write a data block having a logical block address to a nonvolatile storage device. The method includes writing a value of the data block to the nonvolatile storage device. The writing includes locating a position in a tree-based data structure that includes first and second nodes. The first node is configured to store a first set of data blocks having logical block addresses in a first numerical range, and the second node is configured to store a second set of data blocks having logical block addresses in a second numerical range. The position is located in the first node or the second node depending on the value of the logical block address. The writing includes storing the value of the data block in the position in the tree-based data structure.

Abstract: A durable file system has been designed for storage devices that do not support write in place and/or that are susceptible to errors or failures. The durable file system also facilitates organization and access of large objects (e.g., gigabytes to terabytes in size). The durable file system can efficiently reclaim storage space at zone set granularity since each constituent zone can be reclaimed concurrently when the zone set is chosen for space reclamation. Furthermore, space reclamation for the durable file system does not interfere with object availability because the object data is available throughout reclamation. The durable file system copies data of a live object to a different zone set and updates the file system index before reclaiming the target zone set (e.g., before resetting write pointers to the constituent zones).