Abstract: A data deduplication method includes: receiving a write request with first data block; writing the first data block into a storage device; writing metadata of the first data block into a first partition that is in a plurality of partitions of a metadata management structure and that is determined based on a feature of the first data block; and deleting the metadata of the first data block in the first partition and deleting the first data block from the storage device based on address information of the first data block when a fingerprint that is the same as the fingerprint of the first data block exists in the first partition. The method can prevent infrequently updated data from being evicted because resources are occupied by frequently updated data, thereby improving a deduplication ratio.

Abstract: A data prefetch method includes before data is prefetched, a first data access request that is first obtained, and a data prefetch policy that is determined based on the first data access request and a data lifecycle such that a first data set stored in a second storage medium is stored into a first storage medium according to the data prefetch policy. The first data set includes at least one piece of data, the data prefetch policy includes at least a prefetch length, the data lifecycle indicates duration for storing data in the first storage medium, and a read/write access rate of the second storage medium is less than that of the first storage medium.

Abstract: The present disclosure relates to methods and apparatuses for managing a shared cache. One example method includes determining an access characteristic of accessing the shared cache by IO requests of each of K types that access the shared cache, determining a partition size and an eviction algorithm of the IO requests of each type in the shared cache based on the determined access characteristic and a hit rate of the shared cache, and configuring a cache size of the IO requests of each type in the shared cache as the determined partition size, and configuring an eviction algorithm of the IO requests of each type in the shared cache as the determined eviction algorithm.

Abstract: A method describes that a lost first block in a first stripe is directly reconstructed based on a first result obtained from a target quantity of storage devices, so that there is no need to read an unlost block in the first stripe. The first block may be reconstructed provided that the target quantity of first results are obtained. An amount of data of the first result is less than that of the unlost block in the first stripe. Therefore, a data transmission process occupies less network bandwidth, and this improves block reconstruction performance.

Abstract: In a data storage method, a storage system comprises first and second medium layers for data storage, wherein the performance of the first medium layer is different from the performance of the second medium layer. Based on the performance difference between the two layers, the storage system stores data in the first medium layer and the second medium layer based on different erasure code ratios. The different erasure code ratios correspond to different write amplification, and result in different storage space utilization.

Abstract: A method and apparatus for determining stripe consistency, including an operation node that obtains n groups of CRC data blocks corresponding to a target stripe, where the target stripe includes n first data blocks, the n first data blocks include k original data blocks and m EC data blocks of the k original data blocks, the n groups of CRC data blocks are respective CRC data blocks of n groups of second data blocks, one group of second data blocks includes m transformation results obtained by performing m times of EC transformation on one first data block, both k and m are positive integers, and n=m+k.

Abstract: A data prefetching method includes a computing node obtaining information about accessing a storage node by a first application in a preset time period. The computing node determines information about prefetch data based on the access information. The computing node determines, based on the information about the prefetch data, a cache node prefetching the prefetch data, and generates a prefetch request for prefetching the prefetch data. The computing node sends the prefetch request to the cache node. The cache node performs a prefetching operation on the prefetch data in response to the prefetch request.

Abstract: A data processing method is provided. The method includes: obtaining a parameter related to performance of a memory of a storage node, adjusting an aggregation policy based on the parameter, aggregating a first input output IO request in the memory into a second IO request based on an adjusted aggregation policy, and performing persistent storage on data corresponding to the second IO request.

Abstract: In a data compression method, a computing device determines a compression feature value of to-be-compressed data based on a first parameter that affects a compression result of the to-be-compressed data. The computing device determines, based on the compression feature value, a compression policy for compressing the to-be-compressed data. The computing device then compresses the to-be-compressed data according to the compression policy to obtain compressed data, and stores the compressed data.

Abstract: In a data compression method, a computing device determines a compression feature value of to-be-compressed data based on a first parameter that affects a compression result of the to-be-compressed data. The computing device determines, based on the compression feature value, a compression policy for compressing the to-be-compressed data. The computing device then compresses the to-be-compressed data according to the compression policy to obtain compressed data, and stores the compressed data.

Abstract: A data prefetch method is disclosed. Before data is prefetched, a first data access request is first obtained, and a data prefetch policy is determined based on the first data access request and a data lifecycle, so that a first data set stored in a second storage medium is stored into a first storage medium according to the data prefetch policy. The first data set includes at least one piece of data, the data prefetch policy includes at least a prefetch length, the data lifecycle indicates duration for storing data in the first storage medium, and a read/write access rate of the second storage medium is less than that of the first storage medium.

Abstract: In a data storage method, a storage system comprises first and second medium layers for data storage, wherein the performance of the first medium layer is different from the performance of the second medium layer. Based on the performance difference between the two layers, the storage system stores data in the first medium layer and the second medium layer based on different erasure code ratios. The different erasure code ratios correspond to different write amplification, and result in different storage space utilization.

Abstract: A method and apparatus for determining stripe consistency, including an operation node that obtains n groups of CRC data blocks corresponding to a target stripe, where the target stripe includes n first data blocks, the n first data blocks include k original data blocks and m EC data blocks of the k original data blocks, the n groups of CRC data blocks are respective CRC data blocks of n groups of second data blocks, one group of second data blocks includes m transformation results obtained by performing m times of EC transformation on one first data block, both k and m are positive integers, and n=m+k.

Abstract: A method for obtaining a storage system capacity is provided. An available capacity that is of a storage system and that is associated with each stripe length is obtained based on an obtained stripe length that can be effectively configured. Therefore, an available capacity of a system is optimally selected.

Abstract: The present disclosure discloses a data reconstruction method and apparatus, a computer device, and a storage medium and system, and pertains to the field of data storage technologies. According to the method, a lost first block in a first stripe is directly reconstructed based on a first result obtained from a target quantity of storage devices, so that there is no need to read an unlost block in the first stripe. The first block may be reconstructed provided that the target quantity of first results are obtained. An amount of data of the first result is less than that of the unlost block in the first stripe. Therefore, a data transmission process occupies less network bandwidth, and this improves block reconstruction performance.

Abstract: An erasure code (EC)-based data processing method implemented by a storage controller includes obtaining K data chunks, dividing each of the K data chunks into two data slices, encoding the 2*K data slices based on a parity matrix including 2*M rows and 2*(K+M) columns of elements, to obtain 2*M parity slices, and separately storing the K data chunks and the M parity chunks in different storage devices.

Abstract: An erasure code (EC)-based data processing method implemented by a storage controller includes obtaining K data chunks, dividing each of the K data chunks into two data slices, encoding the 2*K data slices based on a parity matrix including 2*M rows and 2*(K+M) columns of elements, to obtain 2*M parity slices, and separately storing the K data chunks and the M parity chunks in different storage devices.

Abstract: A method for obtaining a storage system capacity is provided. An available capacity that is of a storage system and that is associated with each stripe length is obtained based on an obtained stripe length that can be effectively configured. Therefore, an available capacity of a system is optimally selected.