Abstract: The described technology is generally directed an extension to the IFS domains architecture, referred to herein as filter domains. IFS domains allows tagging of files in a tree-like dataset. Thus, a domain can be defined at the root of the dataset such as the topmost directory under which all files reside. These domains are inherently hierarchichal, path-based entities. Filter domains extends this organization to allow domains to be applied beyond hierarchical tree structures in order to also provide arbitrary grouping of file objects based on any suitable membership condition.

Abstract: Direct read in clustered file systems is described herein. A method as described herein can include determining, for a write operation on a resource stored by a data storage system, as initiated by an initiator node, a reference count for the resource, the reference count comprising a number of target storage regions of the data storage system to be modified by write data during the write operation; facilitating conveying, from the initiator node to a lock coordinator node, the reference count for the resource; facilitating conveying, from the initiator node to respective participant nodes that are respectively assigned to the target storage regions, the write data and a key value for the write operation; and facilitating causing the respective participant nodes to convey respective notifications that comprise the key value in response to the respective participant nodes writing the write data to the target storage regions.

Abstract: Incremental replication between foreign system dataset stores is described herein. A method as described herein can include connecting to a target storage system to facilitate an initial transfer of a dataset from the system to the target storage system, wherein the system is distinct from the target storage system; in response to determining that a first version of the dataset is present on the target storage system, comparing the first version of the dataset to a second version of the dataset stored on the system; and, in response to determining that the second version of the dataset is more recent than the first version of the dataset, transferring a differential between the first version of the dataset and the second version of the dataset to the target storage system, the differential comprising less than all of the second version of the dataset.

Abstract: Incremental replication between foreign system dataset stores is described herein. A method as described herein can include connecting to a target storage system to facilitate an initial transfer of a dataset from the system to the target storage system, wherein the system is distinct from the target storage system; in response to determining that a first version of the dataset is present on the target storage system, comparing the first version of the dataset to a second version of the dataset stored on the system; and, in response to determining that the second version of the dataset is more recent than the first version of the dataset, transferring a differential between the first version of the dataset and the second version of the dataset to the target storage system, the differential comprising less than all of the second version of the dataset.

Abstract: Implementations are provided herein for offering partitioned performance within a distributed file system and more specifically, for offering adaptive predicted impact of resource consumption by pending work items. Core resource consumption per work item can be estimated prior to processing the work item. When processing the work item, the actual amount of resources used to process the work item can be measured and recorded. The file system can then update future estimates for performing work items based on past results. Resources made available to process future requests can be throttled based on dynamically updated estimates of resource consumption by pending work items.

Abstract: Implementations are provided herein for offering partitioned performance within a distributed file system and providing throttling at the granular level. A set of hardware and network resources available to process work items can be determined. A set of resource accounting tokens based on resource records generated when processing work items can be dynamically updated. A granular resource accounting aggregate for a customizable field of data can be selected for throttling, such as a unique user identifier, a unique group identifier, a unique client internet protocol address, a unique file, etc. A granular throttling level can then be established based on a granular throttling policy. In response to the resource accounting aggregate meeting the throttling level, the user, group, internet protocol address, etc. can be throttled at, at least one of, the cluster layer, the node layer or the protocol layer.

Abstract: Implementations are provided herein for offering partitioned performance within a distributed file system. Core resource consumption per work item can be tracked independently. Discriminative data already known by the file system surrounding the context of the work item can be used to determine a reference resource accounting specification applicable to the work item. When processing the work item, a detailed resource record can be generated that inventories the resources used in processing the work item. The resource record associated with the work item can be recorded into a set of resource accounting tokens that track activity resource consumption at a granular level. A universal table of resource accounting tokens can be dynamically updated upon the processing of work items and generation of associated resource records throughout the distributed file system.

Abstract: Implementations are provided herein for offering partitioned performance within a distributed file system; specifically, for providing adaptive policies and leases within the partitions. An amount of resources available to a cluster of nodes operating as a distributed file system can be determined and those resources can be apportioned to individual nodes based on hardware profiles of the nodes. A set of resource accounting tokens can be dynamically updated and used as a basis to generate a cluster resource accounting aggregate, a set of node resource accounting aggregates, and a set of protocol resource accounting aggregates. The dynamically updated resource accounting aggregates can then be used to dynamically throttle resource available to process work requests at the cluster, node, and protocol head layers based on policy.

Abstract: Systems and methods are provided herein for establishing data constructs necessary for efficient and transparent tiering of data from local storage to cloud storage, and there use in transferring data to and from cloud storage. Files data can split into chunks and stored as individual objects in cloud storage. A stub file can replace the local file in local storage and describe a mapping of cloud objects that contain data to specific regions of the file. Upon creation of the stub and transfer of data to cloud storage, local storage capacity previously dedicated to file data can be freed for other uses.

Abstract: In general, embodiments of the invention relate to reading data from and writing data to a storage system. Specifically, embodiments of the invention relate to a read only mode for a portion of a storage system. In one embodiment, a selective read-only mode for a portion of a storage system is implemented by monitoring a condition that affects a subset of persistent storage in a storage system, by detecting the condition, by entering a read-only mode for the subset, and by enforcing a policy of processing write requests and read requests to the storage system, which includes processing the write requests without modifying user data stored on the subset and processing the read requests, including requests for user data stored on the subset.

Abstract: In general, embodiments of the invention relate to reading data from and writing data to a storage system. Specifically, embodiments of the invention relate to a read only mode for a portion of a storage system. In one embodiment, a selective read-only mode for a portion of a storage system is implemented by monitoring a condition that affects a subset of persistent storage in a storage system, by detecting the condition, by entering a read-only mode for the subset, and by enforcing a policy of processing write requests and read requests to the storage system, which includes processing the write requests without modifying user data stored on the subset and processing the read requests, including requests for user data stored on the subset.

Abstract: A software development kit prioritizes certain aspects of an acquired image and introduces the prioritized aspects to a main processor. Alternatively, a coprocessor, or the coprocessor and the development kit, manipulate(s) the acquired image and introduce(s) the manipulated image to the processor. The reprogramming of either one of the development kit and the coprocessor may be initiated by either one of them or by the processor and the programming may be provided by the main processor. A central station and a gate array may also be individually reprogrammable by the main processor which sets up, programs and controls an intelligent imaging platform in accordance with the individual reprogramming. A reprogramming of an audio acquisition stage may also be initiated by that stage and any of the other stages and the processor and may be provided by the processor. The audio information may be related to the acquired image.