Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: A method implemented by a memory controller may comprise receiving segment pointers that identify memory devices and slices for associated segments. The method may comprise identifying a most-full first one of the memory devices. The method may comprise identifying a most-empty second one of the memory devices. The method may comprise identifying one of the segments in the most-full one of the memory devices. The method may comprise moving a slice from the identified one of the segments in the most-full one of the memory devices to the most-empty one of the memory devices. The method may comprise updating one of the segment pointers for the identified segment.

Abstract: A memory system uses a dynamic RAID scheme to dynamically encode RAID address space geometries. The dynamic RAID scheme solves issues with the algorithmic layout approach and flat virtual address space used in conventional RAID systems. The dynamic RAID scheme can be used for any RAID algorithm and does not require static mapping. In other words, there is no requirement that each strip be located in the same relative location in memory devices and there is no requirement that stripes use the same combination of memory devices.

Abstract: Apparatus, methods, and other embodiments associated with object synthesis are described. One example apparatus includes logic for identifying a block in a data de-duplication repository and for identifying a reference to the block. The apparatus also includes logic for representing a source object using a first named, organized collection of references to blocks in the data de-duplication repository and logic for representing a target object using a second named, organized collection of references. The apparatus is configured to synthesize the target object from the source object. Since synthesis may be complicated by edge cases, the apparatus is configured to account for conditions including a block in the target object needing less than all the data in a source object block, data to be used to synthesize the target object residing in a sparse hole in a data stream, and the target object needing data not present in the source object.

Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: A memory system uses a dynamic RAID scheme to dynamically encode RAID address space geometries. The dynamic RAID scheme solves issues with the algorithmic layout approach and flat virtual address space used in conventional RAID systems. The dynamic RAID scheme can be used for any RAID algorithm and does not require static mapping. In other words, there is no requirement that each strip be located in the same relative location in memory devices and there is no requirement that stripes use the same combination of memory devices.

Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: A memory system uses a dynamic RAID scheme to dynamically encode RAID address space geometries. The dynamic RAID scheme solves issues with the algorithmic layout approach and flat virtual address space used in conventional RAID systems. The dynamic RAID scheme can be used for any RAID algorithm and does not require static mapping. In other words, there is no requirement that each strip be located in the same relative location in memory devices and there is no requirement that stripes use the same combination of memory devices.

Abstract: A memory system uses a dynamic RAID scheme to dynamically encode RAID address space geometries. The dynamic RAID scheme solves issues with the algorithmic layout approach and flat virtual address space used in conventional RAID systems. The dynamic RAID scheme can be used for any RAID algorithm and does not require static mapping. In other words, there is no requirement that each strip be located in the same relative location in memory devices and there is no requirement that stripes use the same combination of memory devices.

Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: A method of managing a memory system is described, the memory system having a processor with a client interface, a buffer memory, and a storage memory. A LUN (logical unit) is exposed to the client, the LUN having a memory size, and QoS specification. When the processor receives a command from the client to write data to the LUN, determining if the QoS specification includes deduplication and: if the QoS does not include deduplication, processing the command by storing the data in the storage memory and creating metadata uniquely referencing a storage location of the data that is written; or if the QoS includes deduplication, determine if deduplication is possible while meeting a latency value as specified in the QoS specification and performing deduplication.

Abstract: Apparatus, methods, and other embodiments associated with object synthesis are described. One example apparatus includes logic for identifying a block in a data de-duplication repository and for identifying a reference to the block. The apparatus also includes logic for representing a source object using a first named, organized collection of references to blocks in the data de-duplication repository and logic for representing a target object using a second named, organized collection of references. The apparatus is configured to synthesize the target object from the source object. Since synthesis may be complicated by edge cases, the apparatus is configured to account for conditions including a block in the target object needing less than all the data in a source object block, data to be used to synthesize the target object residing in a sparse hole in a data stream, and the target object needing data not present in the source object.

Abstract: Apparatus, methods, and other embodiments associated with object synthesis are described. One example apparatus includes logic for identifying a block in a data de-duplication repository and for identifying a reference to the block. The apparatus also includes logic for representing a source object using a first named, organized collection of references to blocks in the data de-duplication repository and logic for representing a target object using a second named, organized collection of references. The apparatus is configured to synthesize the target object from the source object. Since synthesis may be complicated by edge cases, the apparatus is configured to account for conditions including a block in the target object needing less than all the data in a source object block, data to be used to synthesize the target object residing in a sparse hole in a data stream, and the target object needing data not present in the source object.

Abstract: Example apparatus and methods concern multi-tier bandwidth-centric deduplication. One example apparatus supports inline bandwidth-centric deduplication with post-processing space-centric deduplication to improve inline bandwidth-centric deduplication and thereby reduce bandwidth requirements. One example method may include determining whether a bandwidth-centric deduplication device can satisfy a deduplication request associated with a data communication and then deciding whether to engage a space-centric deduplication device to co-operate in attempting to satisfy the request. More generally, the method includes controlling a first deduplication device to participate in bandwidth reduction and selectively controlling a second deduplication device to also participate in the bandwidth reduction.

Abstract: Embodiments associated with configurable, repeatable, data generation are described. One example method includes manipulating a redundancy parameter that controls data redundancy in binary large objects (BLOBs) to be included in a generated data set. The redundancy parameters may control variations in repeatable variable length sequences included in BLOBs. The example method also includes manipulating a parameter(s) that controls custom designed sequences included in BLOBs. With the redundancy and custom designed sequences described, the example method then generates BLOBs based, at least in part, on the redundancy parameters and the custom-designed sequences. BLOBs may include byte sequences repeated at different frequencies and configurable user-designed sequences. Manipulating the redundancy parameter, manipulating the custom-designed sequences, generating the BLOBs, and providing the BLOBS may be performed by separate processes acting in parallel.

Abstract: Apparatus, methods, and other embodiments associated with de-duplication seeding are described. One example method includes re-configuring a data de-duplication repository with a blocklet from a data de-duplication seed corpus. Reconfiguring the repository may include adding a blocklet from the seed corpus to the repository, activating a blocklet identified with the seed corpus in the repository, removing a blocklet from the repository, and de-activating a blocklet in the repository. The example method may also include re-configuring a data de-duplication index associated with the data de-duplication repository with information about the blocklet. Reconfiguring the repository and the index increases the likelihood that a blocklet ingested by a data de-duplication apparatus that relies on the repository and the index will be treated as a duplicate blocklet by the data de-duplication apparatus.

Abstract: Apparatus, methods, and other embodiments associated with blocklet pattern identification are described. One example method includes accessing a blocklet produced by a computerized data de-duplication parsing process before providing the blocklet to a duplicate blocklet determiner. The example method also includes hashing a portion of the blocklet to produce a pattern indicating hash and then identifying the blocklet as a pattern blocklet if the pattern indicating hash matches a pre-determined pattern indicating hash. To improve efficiency in a data de-duplication process, the blocklet pattern identifying may be performed independently from a data structure and process used by the duplicate blocklet determiner. If the blocklet is a pattern blocklet, then the method includes selectively controlling the duplicate blocklet determiner to not process the pattern blocklet. The duplicate determination is not needed because a pattern determination has already been made.