Abstract: A system and method for specifying the placement of and managing the placement of objects on a distributed networked grid. Some embodiments of the invention comprise a user interface for specifying rules for the placement of objects on the grid and an execution module that places objects on the grid based on the rules specified. Rules may comprise a matching criteria specification, which determines whether a rule applies to a particular object, and a placement specification, which directs the locations at which an object will be placed under the rule.

Abstract: A service level objective (SLO) subsystem that includes an SLO engine in a network storage system that allows new degrees of freedom to be added to the SLO subsystem, without requiring any modification of an associated SLO engine or the system's existing SLOs, and without requiring the SLO engine to have any a priori knowledge of the new degrees of freedom. In certain embodiments, the system has a modular design which includes the SLO engine and one or more SLO modules that interface with the SLO engine. Each SLO module corresponds to a different degree of freedom. Each SLO module evaluates SLO compliance and determines what specific action(s) within its corresponding degree of freedom should be taken to comply with an SLO.

Abstract: Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes and at least one latent storage. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.

Abstract: A method of controlling power within a multiple-data-storage-devices enclosure is disclosed. In at least one embodiment, the method comprises receiving a network connection and power from a data connection port to which a network cable is removably coupled; identifying one or more target data storage devices within the multiple-data-storage-devices enclosure to activate, wherein the one or more target data storage devices are a subset of all data storage devices within the multiple-data-storage-devices enclosure; powering off at least one of the data storage devices that draws power from the data connection port to make available additional power to supply from the data connection port; and powering the target data storage devices with the power received through the data connection port to activate the target data storage devices after all other data storage devices are powered off.

Abstract: A system can apply file placement rules to dynamically place files and directories within file system views backed by objects in an object storage system. After detection of an update to a first file system view that causes an update of an object in a storage grid, an object manager begins evaluation of file placement rules against metadata of the object. For each file placement rule that is triggered, the object manager determines identifies gateways that export the first file system view. The object manager then instructs the gateways to update their representations of the first file system view. The disclosed embodiments may be able to scale to managing hundreds of billions of files spanning thousands of file system views, especially in the presence of disconnected operation.

Abstract: In some embodiments, a multiple-data-storage-devices cartridge can implement a method of writing data via a data range application programming interface (“API”). The method can include: receiving a write request from a requester device, wherein the write request is a direct, broadcast or multicast, or fanout message and includes a size indication for a contiguous range of data; responsive to receiving the write request, sending a response message to the requester device indicating an intent to store the contiguous range of data; receiving the contiguous range of data from the requester device; powering on a target data storage device from amongst data storage devices within the cartridge while keeping at least another data storage device in the cartridge powered off; and writing the contiguous range of data to the target data storage device.

Abstract: In at least one embodiment, a method of metering lifespan consumption of data storage devices in a multiple-data-storage-devices enclosure is disclosed. The method includes: receiving a storage access request; identifying a first data storage device within the enclosure as being targeted by the storage access request, wherein the enclosure includes two or more data storage devices; accessing the first data storage device in response to the storage access request; and in response to the identifying or the accessing, updating a record of a lifespan consumption meter assigned to the first data storage device.

Abstract: Techniques for operating a storage front-end system are disclosed. The techniques include identifying a synchronous group of data storage devices across two or more enclosures, each of which comprise a plurality of data storage devices. Data across the data storage devices is accessible by a storage front-end system as an aggregate memory space. The techniques further include sending an activation request to the enclosures to synchronously activate the data storage devices in the synchronous group and performing a data maintenance task in the aggregate memory space of the data storage devices.

Abstract: Techniques for operating a storage system are disclosed. A read request with an object identifier for a data object is received. A synchronous group of data storage devices across a plurality of enclosures is identified. The synchronous group is associated with the object identifier. A request is sent to the plurality of enclosures to synchronously activate the data storage devices in the synchronous group. After sending the request, data fragments associated with the object identifier are retrieved from the synchronous group of data storage devices. The data fragments are erasure decoded into a contiguous data range to reconstruct the data object.

Abstract: A write request including payload data is received. The payload data of the write request is stored in a staging area of a storage manager. A transformation pipeline is determined based, at least in part, on an attribute of the write request. The transformation pipeline is queued for execution. Data fragments are generated based, at least in part, on the payload data and the transformation pipeline. The data fragments are transmitted to a plurality of enclosures.

Abstract: A storage access request is received. A data storage device is identified as being targeted by the storage access request. The data storage device is located in an enclosure that includes two or more data storage devices, including the data storage device. An access operation is performed in accordance with the storage access request on the data storage device. A record of a fault sensor assigned to the data storage device is updated based at least partly on monitoring the performing of the access operation.

Abstract: A method of power management of a multiple-data-storage-devices enclosure is disclosed. In some embodiments, the method includes: receiving a network connection and power from a data connection port detachably coupled to a network cable; identifying a subset of data storage devices within the enclosure to activate; powering off components within the enclosure other than the subset of the data storage devices; for each data storage device in the subset that is not yet powered, activating the data storage device by: monitoring power consumption drawn from the data connection port; identifying a sequence of components associated with the data storage device, wherein the components within the sequence, when powered, together provide access to the data storage device; and powering on each component in the sequence when a previous component in the sequence has reached a steady state power consumption level.

Abstract: A data distribution system is provided which supplies customers with an executable for requested secured data files to provide the customer with fulfillment software, obviating the need for the customer to download fulfillment software prior to requesting secure data. The data distribution system is characterized by server technology which can dynamically encrypt secured data files just prior to a customer request to download the data file. A framework for building a universal data distribution infrastructure is provided which employs Requesters.

Abstract: Methods and systems that permit file protocols and object protocols to co-exist using a file namespace and an object namespace in a unified storage system are disclosed. Data stored in the unified storage system are file-objects that preserve the behaviors expected by both file clients and object clients.

Abstract: A method of power management of a multiple-data-storage-devices enclosure is disclosed.

Abstract: Systems and methods that allow operators to configure how files and directories are placed within file system views into a storage system, and how these configured file placements are performed by the distributed system. Possible features include a mechanism by which the desired placement of files and directories can be specified, a mechanism by which the placement goals can be realized in a given topology of a distributed system, and a mechanism by which changes to the placement of objects can applied retroactively. The disclosed embodiments may be able to scale to managing hundreds of billions of files spanning thousands of file system views, especially in the presence of disconnected operation.

Abstract: In the area of storage management, service automation can be realized through the use of “MAPE” loop(s). A Planner (P) interacts with the Monitoring (M), Analysis (A) and Execution (E) components in a closed loop. For each new option or potential planning action the Planner (P) invokes the Analysis (A) component. The correctness, as well as effectiveness, of the planning decision is dependent on the Analysis (A) component. Embodiments can utilize an adaptive Analysis (A) component (i.e., an analysis component that can be retrained) that also associates a value of confidence and a corresponding error in the evaluation along with a predicted impact. The Planner (P) component uses this additional information for quoting the final impact of a particular planning action as part of an adaptive MAPE loop to provide improved resource utilization and resource management.

Abstract: Example embodiments provide various techniques for securing communications within a group of entities. In one example method, a request from an entity to join the group is received and a signed, digital certificate associated with the entity is accessed. Here, the signed, digital certificate is signed with a group private key that is associated with a certification authority for the group. The signed, digital certificate is added to a group roster, and this addition is to admit the entity into the group. The group roster with the signed, digital certificate is itself signed with the group private key and distributed to the group, which includes the entity that transmitted the request. Communication to the entity is then encrypted using the signed, digital certificate included in the group roster.

Abstract: Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.

Abstract: Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.