Abstract: Provided is a method and system for preventing transfer errors of big data, including dividing entire data to be transmitted by a first device into a plurality of pieces of entity data and allocating a resource to first entity data among the plurality of pieces of entity data, applying, by the first device, the first entity data to a filter to query for the first entity data, transmitting, by the first device, the first entity data to a second device based on a result of the query for the first entity data, and upon receiving a sync request for the first entity data from the second device, releasing, by the first device, the resource allocated to the first entity data.

Abstract: The present disclosure relates to a method and system for verifying end-to-end data integrity during big data transfer, and includes transferring, with a source node, at least one data set to a sink node; verifying, with the sink node, integrity of at least one object comprised in the data set based on a data and layer-aware Bloom filter (DLBF); verifying, with the sink node, integrity of at least one file comprised in the data set based on a two-phase Bloom filter (TPBF); and verifying, by the sink node, integrity of the data set based on the integrity verification of the object and the file, and can be applied to other embodiments.

Abstract: Provided is a method and system for preventing transfer errors of big data, including dividing entire data to be transmitted by a first device into a plurality of pieces of entity data and allocating a resource to first entity data among the plurality of pieces of entity data, applying, by the first device, the first entity data to a filter to query for the first entity data, transmitting, by the first device, the first entity data to a second device based on a result of the query for the first entity data, and upon receiving a sync request for the first entity data from the second device, releasing, by the first device, the resource allocated to the first entity data.