Abstract: Methods and systems are disclosed to enable users to analyze failure modes of computer vision machine learning models using error featurization. In one embodiment, an image classification model is expressed in a scatter plot of prediction errors over a labeled dataset. A user interface allows practitioners to identify patterns in data that cause the model to fail and supports high precision analysis of critical failure modes of trained machine learning (ML) models. The embodiment helps ML practitioners improve their curation, labeling, and training processes. The embodiments allow ML practitioners to choose the most relevant data for subsequent improvement of an ML model. The highly targeted data curation leads to a multifold reduction in costs and time for labeling and training data.

Abstract: A system comprising a processor and a memory storing instructions that, when executed, cause the system to receive a request for garbage collection, identify a range of physical blocks in a storage device, query a bitmap, the bitmap having a bit for each physical block in the range of physical blocks, determine a status associated with a first bit from the bitmap, in response to determining the status associated with the first bit is a first state, add a first physical block associated with the first bit to a list of physical blocks for relocation, and relocate the list of physical blocks.

Abstract: A system comprising a processor and a memory storing instructions that, when executed, cause the system to receive a request for garbage collection, identify a range of physical blocks in a storage device, query a bitmap, the bitmap having a bit for each physical block in the range of physical blocks, determine a status associated with a first bit from the bitmap, in response to determining the status associated with the first bit is a first state, add a first physical block associated with the first bit to a list of physical blocks for relocation, and relocate the list of physical blocks.

Abstract: A system comprising a processor and a memory storing instructions that, when executed, cause the system to receive a request for garbage collection, identify a range of physical blocks in a storage device, query a bitmap, the bitmap having a bit for each physical block in the range of physical blocks, determine a status associated with a first bit from the bitmap, in response to determining the status associated with the first bit is a first state, add a first physical block associated with the first bit to a list of physical blocks for relocation, and relocate the list of physical blocks.

Abstract: A system and method for efficiently managing data through compression interfaces may include receiving, by a controller, data, generating, by the controller, a compressed payload based on the data, generating, by the controller, metadata describing the compressed payload, the metadata including fixed size metadata and variable size metadata, generating, by the controller, a data container comprising the uncompressed payload and the metadata, and transmitting, by the controller, the data container to an application. Some implementations of the system may include a storage media, and a storage controller executable by a processor that may include an interface processor, a controller logic, and a media processor configured to communicate with an application and the storage media to perform aspects of the method.

Abstract: Various aspects for data deduplication in a storage system are provided. For instance, a storage controller may perform operations including receiving a data chunk including a set of data blocks, determining a signature for the data chunk, and comparing the signature and a set of reference signatures to determine a match. Responsive to a match, the operations may further include identifying a reference data chunk including a set of comparison blocks associated with the matched reference signature, performing a deduplication technique on the set of data blocks based on the set of comparison blocks, and identifying a subsequent reference data chunk for an estimated next data chunk based on identification of the reference data chunk and prior to receipt of the next data chunk.

Abstract: A data storage apparatus that includes a storage device and a processor coupled to the storage device. The processor is configured to receive in a memory, a first logical block entry for a first dump group and a second logical block entry for a second dump group; store in a reverse translation table, the first logical block entry for the first dump group and the second logical block entry for the second dump group; determine a first sequence number associated with the stored first logical block entry and the stored second logical block entry in the reverse translation table, wherein the first sequence number is a snapshot marker that determines a timestamp associated with the first logical block and the second logical block; and persist the first logical block entry for the first dump group in the storage device.

Abstract: A data storage apparatus that includes a storage device and a processor coupled to the storage device. The processor is configured to receive a read request for a first translation table entry associated with a logical block, identify a dump unit associated with the logical block using a hash function, determine a dump group associated with the dump unit, and identify a second translation table entry associated with the dump unit.

Abstract: A system and method for efficiently managing data through compression interfaces may include receiving, by a controller, data, generating, by the controller, a compressed payload based on the data, generating, by the controller, metadata describing the compressed payload, the metadata including fixed size metadata and variable size metadata, generating, by the controller, a data container comprising the uncompressed payload and the metadata, and transmitting, by the controller, the data container to an application. Some implementations of the system may include a storage media, and a storage controller executable by a processor that may include an interface processor, a controller logic, and a media processor configured to communicate with an application and the storage media to perform aspects of the method.

Abstract: A system and method for efficiently managing data through compression interfaces may include receiving, by a controller, data, generating, by the controller, a compressed payload based on the data, generating, by the controller, metadata describing the compressed payload, the metadata including fixed size metadata and variable size metadata, generating, by the controller, a data container comprising the uncompressed payload and the metadata, and transmitting, by the controller, the data container to an application. Some implementations of the system may include a storage media, and a storage controller executable by a processor that may include an interface processor, a controller logic, and a media processor configured to communicate with an application and the storage media to perform aspects of the method.

Abstract: Various aspects for managing data blocks in a storage system are provided. For instance, a method may include storing, in a buffer memory, a plurality of comparison blocks, initiating a data deduplication process utilizing the plurality of comparison blocks, and performing garbage collection in conjunction with the data deduplication process. Garbage collection may include maintaining a hit count for comparison blocks of a passive set of comparison blocks in the buffer memory and deleting the passive set from the buffer memory when the hit count is decremented to a predetermined value. The hit count may be incremented and decremented based on utilization of a comparison block in the data deduplication process.

Abstract: A sequence of contiguous pages in an erase block in a non-volatile memory device is programmed and erased. Next, all of the pages in the erase block are programmed with data. Then, the data is read back and verified to determine whether there is an error in the data. When there is an error in the data, then the last page in the sequence is identified as being unstable. If there is no error in the data, then the last page in that sequence is identified as being stable. Thus, the recorded information identifies a point of instability in the erase block. Instabilities can be stabilized by performing additional writes to fill the partially filled word line.

Abstract: A system comprising a processor and a memory storing instructions that, when executed, cause the system to determine a first value of a first checkpoint associated with a first snapshot, receive a second value of a second checkpoint associated with a translation table entry from an additional source, determine whether the second value of the second checkpoint is after the first value of the first checkpoint, in response to determining that the second value of the second checkpoint is after the first value of the first checkpoint, retrieve the translation table entry associated with the second checkpoint from the additional source, and reconstruct the translation table using the translation table entry associated with the second checkpoint.

Abstract: Various aspects for managing data blocks in a storage system are provided. For instance, a method may include storing, in a buffer memory device, a comparison block library, selecting a first set of comparison blocks in the comparison block library to create an active set of comparison blocks, and utilizing the active set in conjunction with a data deduplication technique. A method may further include determining an occurrence of a predetermined event in the data deduplication technique, selecting a second set of comparison blocks in the comparison block library to create a new active set of comparison blocks in response to the predetermined event, replacing the active set with the new active set, and utilizing the new active set in conjunction with the data deduplication technique.

Abstract: A system comprising a processor and a memory storing instructions that, when executed, cause the system to receive a request to select translation table entries to store in a storage device, determine a plurality of translation table entries associated with a dump unit, allocate the plurality of translation table entries into a first group of translation table entries associated with a first node and a second group of translation table entries associated with a second node, the first group of translation table entries being frequently accessed and the second group of translation table entries being rarely accessed. determine a first status associated with a first recent access bit for a first translation table entry, the first translation table entry being included in the first group of translation table entries, and add the first translation table entry to the second group of translation table entries.

Abstract: A system comprising a processor and a memory storing instructions that, when executed, cause the system to receive a first translation table entry for a logical block, map the first translation table entry to a first dump unit, the first dump unit included in an array of dump units, identify a second translation table entry for the logical block in the first dump unit, the second translation table entry also being stored in a storage device, and generate a linked list in the storage device from the second translation table entry associated with the first dump unit, the linked list identifying previous translation table entries associated with the logical block.

Abstract: A sequence of contiguous pages in an erase block in a non-volatile memory device is programmed and erased. Next, all of the pages in the erase block are programmed with data. Then, the data is read back and verified to determine whether there is an error in the data. When there is an error in the data, then the last page in the sequence is identified as being unstable. If there is no error in the data, then the last page in that sequence is identified as being stable. Thus, the recorded information identifies a point of instability in the erase block. Instabilities can be stabilized by performing additional writes to fill the partially filled word line.

Abstract: Various aspects for managing data blocks in a storage system are provided. For instance, a method may include storing, in a buffer memory, a plurality of comparison blocks, initiating a data deduplication process utilizing the plurality of comparison blocks, and performing garbage collection in conjunction with the data deduplication process. Garbage collection may include maintaining a hit count for comparison blocks of a passive set of comparison blocks in the buffer memory and deleting the passive set from the buffer memory when the hit count is decremented to a predetermined value. The hit count may be incremented and decremented based on utilization of a comparison block in the data deduplication process.

Abstract: A system comprising a processor and a memory storing instructions that, when executed, cause the system to identify a plurality of dump units associated with a translation table in a storage device, determine a plurality of snapshot markers associated with the plurality of dump units, calculate a first value of a first snapshot marker from the plurality of snapshot markers in the storage device, identify a second snapshot marker from an additional source, the second snapshot marker having a second value satisfying the first value, retrieve a dump unit associated with the second snapshot marker from the additional source, and reconstruct the translation table using the dump unit.

Abstract: Various aspects for data deduplication in a storage system are provided. For instance, a storage controller may perform operations including receiving a data chunk including a set of data blocks, determining a signature for the data chunk, and comparing the signature and a set of reference signatures to determine a match. Responsive to a match, the operations may further include identifying a reference data chunk including a set of comparison blocks associated with the matched reference signature, performing a deduplication technique on the set of data blocks based on the set of comparison blocks, and identifying a subsequent reference data chunk for an estimated next data chunk based on identification of the reference data chunk and prior to receipt of the next data chunk.