Abstract: Example apparatus and methods selectively generate and store erasure codes differently based on priorities associated with the erasure codes or based on conditions in a data storage system (DSS) that protects messages using erasure codes. Producing a systematic erasure code (EC) may be prioritized over producing a non-systematic EC. Producing an EC associated with correcting X erasures may be prioritized over producing an EC associated with correcting Y erasures, X and Y being numbers, X<Y. The priorities may depend on conditions in the DSS including an erasure code A/B policy, numbers of errors experienced by the DSS, types of errors experienced by the DSS, frequency of errors, an amount of power required to store or retrieve an EC in the DSS, or a network bandwidth required to store or retrieve an EC in the DSS. The priorities may be user configurable or self-adapting.

Abstract: Example apparatus and methods reserve space in a journal using an observation based approach instead of a fixed sized approach or a worst case scenario approach. One example method receives a request to allocate space in a journal to support a file system transaction. The example method reserves an amount of space in the journal based on a pre-existing reservation size estimate. Unlike conventional systems, the estimate is not based on a worst-case scenario. The example method observes the actual amount of storage used in the journal by the file system transaction and then selectively automatically adjusts the pre-existing reservation size estimate. The estimate may slowly shrink if no overflows are encountered but may quickly grow if an overflow is detected.

Abstract: Example apparatus and methods treat some erasure codes differently than other erasure codes. For example, erasure codes that are only involved in error-recovery may never be read and thus may be stored using a different approach than erasure codes that are involved in more regular data reading. If different types of data stores are available, then the erasure codes that are more likely to be read may be stored in data stores having a first (e.g., higher, faster) type of read performance while the erasure codes that are less likely to be read may be stored in data stores having a second (e.g., lower, slower, less expensive) type of read performance. Different data stores may be located on different data storage devices. Different data stores may even be located on a single data storage device.

Abstract: A detection that a client-server system, which is operating in a first state mode, has switched from a first use case to a second use case for accessing a property of an object associated with a server. The first state mode during the first use case reduces messaging in the client-server system as compared to using a second state mode. A determination is performed as to whether using the second state mode during the second use case would reduce messaging in the system as compared to using the first state mode. The client-server system is transitioned to the second state mode, if operating in the second state mode would reduce messaging in the client-server system as compared to operating in the first state mode for the second use case.

Abstract: Example apparatus and methods selectively replicate some erasure codes associated with a message and selectively distribute, without replicating, other erasure codes associated with the message. The message may have k symbols and n erasure codes may have been generated for the message, n>=k. In one embodiment, erasure codes that store plaintext information from the message (e.g., un-encoded symbols) may be replicated (e.g., sent to all devices using erasure codes associated with the message) while erasure codes that do not store plaintext information may be distributed (e.g., selectively moved to less than all devices) without being replicated. Some (e.g., less than k) erasure codes that do not store plaintext information may be stored unencrypted in the cloud. The generator matrix will not be stored in the cloud.

Abstract: Disclosed are systems and methods for fingerprint analysis for anti-virus scanning. In an embodiment, a method of scanning for infected data items is disclosed. The method provides identifying a plurality of changed data items on a server machine. The method further provides, from a data system in communication with the server machine, performing a virus scan on the plurality of changed data items. The method further provides that the data system maintains a list of data items that the virus scan found to be infected.

Abstract: Methods, apparatus, and other embodiments associated with doubly distributing erasure encoded data in a data storage system are described. One example apparatus includes a set of data storage devices and a set of logics that includes an encoding logic that generates an erasure encoded object that includes code-words, and chunks the code-words into code-word chunks, and a distribution logic that interleaves members of the set of code-word chunks into a plurality of records, and distributes the records across the data storage devices and within individual data storage devices. Example apparatus may include a read logic that reads the plurality of stored records from the data storage devices, and ignores read errors, and a repair logic that monitors the set of data storage devices, replaces or repairs failing data storage devices, generates replacement records, and stores the replacement records on a replacement data storage device.

Abstract: Responsive to an instruction to collapse a derivative version of an ancestor data volume into the ancestor data volume, it is determined if a characteristic of the derivative version of the ancestor data volume satisfies a criteria relative to a characteristic of the ancestor data volume. If the characteristic of the derivative version satisfies the criteria, the ancestor data volume is merged into the derivative version of the underlying data to form an updated derivative version. The updated derivative version is established as the ancestor data volume.

Abstract: Example apparatus and methods create and maintain a functional mirror of file system metadata for an active file system. The functional mirror may be implemented as a set of database tables that allow a journal functionality for information received from the active file system. Example apparatus and methods also restore the file system metadata from the functional mirror. Mirror creation and maintenance can operate in parallel. Mirror maintenance and metadata restore can also operate in parallel. The functional mirror may be created primarily using a systematic, system-wide, multi-threaded crawler approach whose results are updated using an event-driven approach that accounts for changes that may make obsolete information acquired during the crawler approach. The metadata may be restored using a systematic, system-wide, multi-threaded crawler approach acting together with an event-driven approach that restores certain files out of order as they are needed.

Abstract: Example apparatus and methods support reconstructing an item from a set of erasure codes (e.g., fountain codes). Information about computer resources available to support reconstructing the item may be accessed and analyzed to control the spawning of multiple computer processes to support reconstructing the item. The information may include, for example, utilization and capacity data. The information may be provided by sensor agents that monitor the resources. The multiple computer processes may operate at least partially in parallel. Resources may be identified and computer processes may be spawned until all the resources that can contribute to the reconstruction are used. In one embodiment, computer processes may be spawned until the marginal utility of spawning another process falls below a threshold.

Abstract: Example apparatus and methods control the number of rateless erasure codes (e.g., fountain codes) stored in an object store for an item (e.g., file stored as object). The codes for the item may be generated according to an M/N policy. A first safety factor that controls how many codes are stored initially in the object store is identified. A first number of codes are then stored in the object store, where the first number is selected as a function of the first safety factor. A second safety factor for the item and a condition under which the second safety factor is to be used to control the number of codes to be stored in the object store is also identified. When the condition is detected, a second number of codes are stored in the object store, where the second number is selected as a function of the second safety factor.

Abstract: Example apparatus and methods improve deduplication efficiency for a deduplication application or process. A first blocklet repository may have been created according to a first deduplication approach that was optimized for a first set of conditions. Example apparatus and methods create a second blocklet repository from the first blocklet repository by deduplicating the first blocklet repository using a second deduplication approach that is optimized for a second set of conditions. While the first blocklet repository may have been appropriate for the first set of conditions, the second blocklet repository may be appropriate for the second set of conditions. For example, conditions that exist for an immature repository or during ingest may be different than conditions that exist for a mature repository or for applications that use a repository rather than build a repository. The first and second repositories may reside on separate deduplication apparatus.

Abstract: Apparatus, methods, and other embodiments associated with reducing metadata controller communication volume are described. One example apparatus produces tracking data by tracking metadata controller communications and then controls which of two different types of metadata controller communication message types are used for metadata controller communications based, at least in part, on the tracking data. One message type provides actual state and/or location information about a metadata controller. A second message type provides only information about the state and/or location information available at a metadata controller. Selectively using the second message type facilitates reducing metadata controller communication volume.

Abstract: A hardware and/or software facility for durably and securely storing data within a shared community storage network. A user may have a storage device that they intend to share with others in the network. All or a portion of the storage device is registered with the community storage network as a storage node. Once registered with the network, third party data may be stored on the storage node and remotely accessed by third parties. In addition, data stored on the storage device by the user may be stored in the shared community storage network by encrypting the data, adding redundancy, and distributing it to other storage nodes within the storage network. Data that is stored in the storage network is accessible to the user even if their storage device is inaccessible or fails.

Abstract: A method for monitoring changes in an inventory of media cartridges in a media library includes the steps of determining with a controller that the inventory of media cartridges in the media library has changed, and asynchronously reporting with the controller the specific change in inventory of media cartridges that has occurred in the media library. The method can also include the step of determining whether the change in inventory included an increase or a decrease in the quantity of media cartridges within the media library, or whether the change in inventory did not include an increase or decrease in the quantity of media cartridges within the media library. Further, the method can also include the step of determining whether the change in inventory of media cartridges is required to be reported. A media library system includes a media library having a plurality of media cartridges, and a controller that determines a change in an inventory of media cartridges in the media library.

Abstract: An open systems based media storage library system (10) for use by one or more hosts (5) includes a first hosted partition (12A) and a pool partition (14). The first hosted partition (12A) can be accessed by one of the hosts (5). Additionally, the first hosted partition (12A) includes a plurality of first hosted resources (16, 18). The plurality of first hosted resources (16, 18) can include a plurality of first hosted storage slots (16) and a plurality of first hosted storage media (18). The plurality of first hosted storage slots (16) includes one or more first physical storage slots (16P) and one or more first logical storage slots (16L). The plurality of first hosted storage media (18) are positioned within the first physical storage slots (16P). The pool partition (14) is inaccessible by any of the hosts (5). The pool partition (14) includes a plurality of pool resources (20, 22). One of the plurality of pool resources (20, 22) is selectively assigning to the first hosted partition (12A).

Abstract: Example apparatus and methods produce a set of rateless erasure codes (e.g., fountain codes) for a file stored in a primary data store (e.g., hard drive) or in an archive system. The archive system may store the file in a redundant array of independent disks (RAID). A first subset of the rateless erasure codes are stored in an object storage using a synchronous protocol. A second subset of rateless erasure codes are stored in the object storage using an asynchronous protocol. The object storage system may inform the archive system when desired redundancy has been achieved or when desired redundancy has been lost. The archive system may buffer rateless erasure codes before providing the codes to the object storage to improve performance. A failure in the archive system or object storage system may be mitigated by retaining the file in the primary data store until the desired redundancy is achieved.

Abstract: A magnetic recording device includes a preprocessor, an interpolator and a slicer. The preprocessor receives at least n saturated input signals including an nth saturated input signal The preprocessor is configured to process each of the n saturated input signals to produce a corresponding nth output signal. The n output signals include an nm output signal from the nth saturated input signal. The interpolator processes the nth output signal to determine an nth interpolator output. The slicer determines an nth slicer output for the nh output signal. The nth slicer output is at one of three different levels. The preprocessor can receive and process an n +1th saturated input signal to produce an n +1th output signal that is based on a difference between the nth interpolator output and the level of the nth slicer output.

Abstract: A data control system facilitates transfer of a virtual disk from a primary storage system to a secondary storage system. The data control system, responsive to an instruction to transfer the virtual disk, wherein the virtual disk comprises a plurality of data blocks, determines whether each of the plurality of data blocks is allocated or unallocated; for each data block of the plurality of data blocks determined to be allocated, the data control system reads the data block from memory in the primary storage system and transfers the data block for storage in the secondary storage system; and for each data block of the plurality of data blocks determined to be unallocated, the data control system refrains from reading the data block from memory in the primary storage system.

Abstract: Disclosed is a method of operating a data storage system. The method comprises identifying changed segments of a primary storage volume, receiving a data request for a plurality of data items in a secondary storage volume, identifying changed data items of the plurality of data items in the secondary storage volume based on a correspondence between the plurality of data items in the secondary storage volume and the changed segments of the primary storage volume, and transferring the changed data items in response to the data request.