Abstract: Example apparatus and methods concern selective adaptive predictive data placement to improve the operating and/or electrical efficiency of a data storage apparatus. A future input/output operation is predicted from a current input/output operation, the state of the data storage apparatus, relationships between data currently being processed and data previously processed, environmental factors, or other factors. The apparatus and methods may improve data storage efficiency by selectively pre-fetching data, relocating data on the data storage apparatus or within a plurality of data storage apparatus, speculatively producing erasure codes or other error correction codes, speculatively deduplicating data, or other adaptive functions. Relocation and pre-fetching may be configured to achieve different policies focused on electrical efficiency, operating efficiency, use spreading, or other considerations.

Abstract: Example methods and apparatus release storage locations in a tiered storage system by deleting expired tier-copies of a file stored in the tiered storage system. One embodiment includes a policy-class circuit that defines a tier-copy policy associated with a tier, where the tier-copy policy includes an expiration interval (EI) associated with the tier, and a schedule associated with the tier. A copy expiration circuit checks, according to the schedule, for expiration-eligible tier-copies of a file stored in the tiered storage system based, at least in part, on an EI associated with a tier, and on an access time (AT) associated with the file, where the AT denotes the last time the file was accessed. The copy expiration circuit, upon determining that a tier-copy is expiration eligible, declares the tier-copy as expiration eligible. A delete/restore circuit, upon the copy expiration circuit declaring that the tier-copy is expiration eligible, deletes the tier-copy.

Abstract: Embodiments disclosed herein provide systems, methods, and computer-readable media to implement an object store with removable storage media. In a particular embodiment, a method provides identifying first data for storage on a first removable storage medium and designating at least a portion of the first data to a first data object. The method further provides determining a first location where to store the first data object in a first value store partition of the first removable storage medium and writing the first data object to the first location. Also, the method provides writing a first key that identifies the first data object and indicates the first location to a first key store partition of the first removable storage medium.

Abstract: Embodiments redirect data from a source to a plurality of deduplication blockpools. In one embodiment, operations include accessing a binary large object (BLOB) that includes a plurality of blocklets, a blocklet having a hash value; upon determining that the plurality of blocklets includes less than a threshold number of blocklets: selecting, according to a first rule set, a target blockpool from among the plurality of deduplication blockpools; upon determining that the plurality of blocklets includes at least the threshold number of blocklets: selecting, according to a second, different rule set, a target blockpool. The second rule set includes a BalanceQuery rule that computes a fitness value for a member of the plurality of deduplication blockpools, and selects a target blockpool based, at least in part, on the fitness value. Embodiments provide the BLOB to the target blockpool.

Abstract: A media drive system (310) configured for use with a media drive (314) that performs read/write operations relative to a media cartridge (316) includes a system housing (312). The system housing (312) includes a housing body (312A) and a controller (350) that is secured to the housing body (312A). The controller (350) is configured to control functionality of the tape drive (314). More specifically, the inclusion of the controller (350) as part of the system housing (312) enables the media drive system (310) to achieve greatly enhanced functionality. For example, the media drive system (310) can greatly enhance the speed of various desired read/write operations performed within the media drive (314), especially when the requested files or file segments are not necessarily initially provided in sequential order on the media cartridge (316). The controller (350) can include one or both of a processor (352) and a memory system (354).

Abstract: Example apparatus and methods control whether and when hybrid rateless Reed Solomon (RS) error correcting codes (ECC) for a message are produced, stored, and distributed. The control may be based on a property (e.g., reliability, error state, speed) of a message recipient. Example apparatus and methods may also control whether and when fountain codes for the message are produced, stored, and distributed. Once again, the control may be based on a property of a message or ECC recipient. Both the hybrid rateless RS ECC and the fountain codes may be produced from data stored in a modified RS matrix. The modified RS matrix may store row-centric error detection codes (EDC) instead of conventional cyclic redundancy check (CRC) characters. The modified RS matrix may store column-centric ECC that may be produced serially. Different types or numbers of ECC may be produced, stored, and provided for different messages stored at different recipients.

Abstract: Embodiments include an apparatus for replicating a file system (FS) that stores a file in a first location in a first data storage device, the apparatus comprising a query circuit configured to generate a spatial query that identifies a primary directory tree in the FS, and to receive a response to the query from the FS, where the response identifies a range of the primary directory tree in which a changed file may be found; a scan filter circuit configured to generate a scan filter based on the response, where the scan filter defines a portion of the primary directory tree that is smaller than the primary directory tree to search for a changed file; and a replication circuit configured to scan the portion of the primary directory tree, and upon detecting a changed file, replicate the changed file from the first location to a different location.

Abstract: Adaptive pre-fetching devices can predict data placement to improve the operating and/or electrical efficiency of a data storage system. A future input/output operation can be predicted from a current input/output operation, the state of the data storage apparatus, relationships between data currently being processed and data previously processed, or other factors. The apparatus and methods can improve data storage efficiency by selectively pre-fetching data, relocating data on the data storage apparatus, the backing storage, or within a plurality of data storage apparatus based on working set predictors to reduce cache misses or outperform fetch processes from the backing storage.

Abstract: Methods and apparatus deduplicate and erasure code a message in a data storage system. One example apparatus includes a first chunking circuit that generates a set of data chunks from a message, an outer precoding circuit that generates a set of precoded data chunks and a set of parity symbols from the set of data chunks, a second chunking circuit that generates a set of chunked parity symbols from the set of parity symbols, a deduplication circuit that generates a set of deduplicated data chunks by deduplicating the set of precoded chunks or the set of chunked parity symbols, an unequal error protection (UEP) circuit that generates an encoded message from the set of deduplicated data chunks, and a storage circuit that controls the data storage system to store the set of deduplicated data chunks, the set of parity symbols, or the encoded message.

Abstract: Example apparatus and methods control a data storage system to store data in a self-describing logical data storage capsule using a logical cylindrical recording format. Example apparatus and methods assign a searchable, globally unique identifier to the capsule and associate the globally unique identifier with a user. The logical data storage capsule is migrated from a first data storage medium to a second data storage medium without translating or reformatting the data storage capsule. The data storage capsule contains information describing to a data storage device how to migrate the capsule without translating or reformatting the data storage capsule. Example apparatus and methods dynamically select an error correction approach for storing data in the data storage capsule, de-duplicate, and encrypt the data storage capsule. The data storage capsule may be local, or may be part of a cloud-based storage system.

Abstract: Embodiments include a data aware deduplicating object store. The data aware deduplicating data store includes a consistent hashing logic that manages a consistent hashing architecture for the object store. The consistent hashing architecture includes a metadata ring and a bulk ring. The consistent hashing architecture may be a multiple ring architecture comprising a metadata ring and two or more bulk rings. A bulk ring may include a key/value (k/v) data store, where a k/v data store stores a shard of an index and a reference count that facilitates the individual approach to garbage collection or data reclamation. The data aware deduplicating data store also includes a deduplication logic that provides data deduplication for data to be stored in the object store. The deduplication logic performs variable length deduplication and provides a shared nothing approach.

Abstract: Embodiments disclosed herein provide systems, methods, and computer readable media for storing data to a plurality of physical storage volumes. In a particular embodiment, a method provides identifying first data for storage on the plurality of physical storage volumes. Each of the plurality of storage volumes corresponds to respective ones of a plurality of data channels. The method further provides segmenting the first data into a plurality of data segments corresponding to respective ones of the plurality of data channels and transferring the plurality of data segments as respective bit streams over the respective ones of the plurality of data channels to the respective ones of the plurality of physical storage volumes. The plurality of storage volumes stores the respective bit streams in the exact condition in which the bit streams are received.

Abstract: Embodiments provide adaptive storage management for optimizing multi-tier data storage. A storage manager may interact with storage decision advisors. The manager may adaptively make storage management decisions (e.g., flush, evict, recall, delete) after considering recommendations from and the credibility of the storage decision advisors. The manager may update the credibility of storage decision advisors based on how their recommendations affected optimization. The manager may adaptively choose when to rebalance or reconfigure the credibility of the storage decision advisors. Storage decision advisors may themselves be adaptive. Storage decision advisors may examine credibility feedback from the storage manager to determine which recommendations were useful and which were not. Storage decision advisors may then change when they will make a recommendation, when they will abstain from making a recommendation, the type of recommendation provided, or other behavior.

Abstract: A diagnostic tape (318) for use with a tape drive (310) having a tape head (322) includes a first tape section (430A) and a second tape section (430B). The first tape section (430A) and the second tape section (430B) are configured to move across the tape head (322) during use of the tape drive (310). The first tape section (430A) includes a first patterned data code (432A) that is indicative of a first spacing between the tape head (322) and the first tape section (430A). The second tape section (430B) includes a second patterned data code (432B) that is indicative of a second spacing between the tape head (322) and the second tape section (430B). The second patterned data code (432B) is different than the first patterned data code (432A). The diagnostic tape (318) can further include a tape head cleaning section (630F) including abrasive material (650) that is configured to move across the tape head (322) to clean the tape head (322).

Abstract: Example methods and apparatus asynchronously verify data stored in a cloud data storage system. One embodiment comprises a monitoring circuit that determines if a data auditing condition associated with a cloud storage system or archived data stored in the cloud storage system has been met, a metadata mirror circuit that controls a metadata mirror to provide metadata, including a first checksum, associated with the archived data to the apparatus, a checksum circuit that computes a second checksum based on the archived data, a verification circuit that generates an audit of the first checksum and the second checksum by comparing the second checksum with the first checksum, and a reporting circuit that generates a log of the audit, that provides the log to the data storage system, and that provides a notification of a data integrity failure to a user associated with the archived data.

Abstract: Methods, apparatus, and other embodiments facilitate discovering and serializing metadata for a file system. After initializing or accessing a serialization data structure, recursive search threads that begin at a selected location in the file system are spawned. The threads analyzes the selected location in the file system, acquire metadata associated with the selected location in the file system, serialize the metadata associated with the selected location in the file system into the serialization data structure, and then selectively spawn additional recursive search threads. Additional recursive search threads may be spawned upon determining that the current recursive search thread has reached a branch point in a hierarchy of the file system. Different threads may perform different types of searches (e.g., depth-first breadth-first) and may operate in parallel. The serialization data structure may be persisted and surfaced in an object (e.g., JSON object) that can be queried or searched.

Abstract: Example apparatus and methods identify files that are so small or so large that they compromise the efficient operation of a file system that uses re-assignable one-to-one inodes and inode numbers. Small files are aggregated into collections of files and large files are subdivided into collections of smaller files. Information for locating multiple related files with fewer lookups is generated and stored in a folder. An inode having a new type of inode number is then created. The new type of inode number encodes information for finding the folder. The encoded information may include a folder identifier that acts as a primary key into a database that is configured to locate a member of the aggregated or subdivided files with a single lookup. A filter file system may be updated with the new inode. The new inode number is unique within the filter file system and may not be re-assigned.

Abstract: A method for converting metadata in a hierarchical configuration within a filesystem from a first format to a second format includes reading metadata that is in the first format within the hierarchical configuration; writing all of the metadata that is in the first format into a flat file; scanning the metadata to compile a list of inode chunks; sorting the list of inode chunks based on the on disk location of the inode chunks; and writing all of the metadata from the flat file back into the hierarchical configuration, the metadata being in the second format. The method can also include increasing the size of each of a first inode and a second inode within a first inode chunk in the filesystem, assigning the first inode to the first inode chunk, and assigning the second inode to a second inode chunk.

Abstract: An example method includes determining a configuration of two or more partitions for a sequential access medium. At least one partition stores data de-duplication data structures while at least one other partition stores a repository of unique data blocks associated with the data structures. The method also includes controlling a data de-duplication computer to configure the sequential access medium according to the configuration. The method includes producing an output sequence for writing the data structures and a set of unique data blocks associated with the set of data structures to the sequential access medium as configured with the two or more partitions. One embodiment includes controlling a data de-duplication computer to write the data de-duplication data structures and the set of unique data blocks to the sequential access medium according to the output sequence.

Abstract: Example apparatus and methods reserve space in a journal using an observation based approach instead of a fixed sized approach or a worst case scenario approach. One example method receives a request to allocate space in a journal to support a file system transaction. The example method reserves an amount of space in the journal based on a pre-existing reservation size estimate. Unlike conventional systems, the estimate is not based on a worst-case scenario. The example method observes the actual amount of storage used in the journal by the file system transaction and then selectively automatically adjusts the pre-existing reservation size estimate. The estimate may slowly shrink if no overflows are encountered but may quickly grow if an overflow is detected.