Abstract: Techniques are provided for block-level fail atomicity on byte-level non-volatile media. In one technique, an offset table and application data that stores content of a file are stored for a file. The offset table includes multiple entries, each entry being associated with a different offset value and storing a logical block address (LBA) that references a location in the application data. In response to receiving a request, that includes an input buffer and an offset value, to update the file: (a) an entry, in the offset table, that corresponds to the offset value and comprises a first LBA is identified; (b) a second LBA that is considered free is identified; (c) the second LBA is replaced with the first LBA; (d) the input buffer is written to a location, in the application data, that the second LBA references; and (e) the second LBA is added in the entry.

Abstract: Techniques are provided for block-level fail atomicity on byte-level non-volatile media. In one technique, an offset table and application data that stores content of a file are stored for a file. The offset table includes multiple entries, each entry being associated with a different offset value and storing a logical block address (LBA) that references a location in the application data. In response to receiving a request, that includes an input buffer and an offset value, to update the file: (a) an entry, in the offset table, that corresponds to the offset value and comprises a first LBA is identified; (b) a second LBA that is considered free is identified; (c) the second LBA is replaced with the first LBA; (d) the input buffer is written to a location, in the application data, that the second LBA references; and (e) the second LBA is added in the entry.

Abstract: A mechanism is provided in a data processing system for performing a remote direct memory access operation. Responsive to receiving in a network interface controller a hash value of data to be copied from a source address in a source node to a destination address in a destination node in the remote direct memory access operation, the network interface controller performs a lookup operation in a translation protection table in the network interface controller to match the hash value to a hash value for data existing in memory of the destination node. Responsive to the network interface controller finding a match in the translation protection table, the network interface controller completes the remote direct memory access operation without transferring the data from the source node to the destination node.

Abstract: A mechanism is provided in a data processing system for on-demand caching in a wide area network (WAN) separated distributed file system or clustered file system. The mechanism monitors file access by a plurality of cache sites in the WAN separated distributed file system or clustered file system. The mechanism identifies access patterns by cache sites. The mechanism shares the access patterns with the plurality of cache sites. A given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

Abstract: A mechanism is provided in a data processing system for reliable asynchronous solid-state device based de-duplication. Responsive to receiving a write request to write data to the file system, the mechanism sends the write request to the file system, and in parallel, computes a hash key for the write data. The mechanism looks up the hash key in a de-duplication table. The de-duplication table is stored in a memory or a solid-state storage device. Responsive to the hash key not existing in the de-duplication table, the mechanism writes the write data to a storage device, writes a journal transaction comprising the hash key, and updates the de-duplication table to reference the write data in the storage device.

Abstract: A mechanism is provided in a data processing system for on-demand caching in a wide area network (WAN) separated distributed file system or clustered file system. The mechanism monitors file access by a plurality of cache sites in the WAN separated distributed file system or clustered file system. The mechanism identifies access patterns by cache sites. The mechanism shares the access patterns with the plurality of cache sites. A given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

Abstract: A mechanism is provided in a data processing system for performing a remote direct memory access operation. Responsive to receiving in a network interface controller a hash value of data to be copied from a source address in a source node to a destination address in a destination node in the remote direct memory access operation, the network interface controller performs a lookup operation in a translation protection table in the network interface controller to match the hash value to a hash value for data existing in memory of the destination node. Responsive to the network interface controller finding a match in the translation protection table, the network interface controller completes the remote direct memory access operation without transferring the data from the source node to the destination node.

Abstract: A mechanism is provided in a data processing system for on-demand caching in a wide area network (WAN) separated distributed file system or clustered file system. The mechanism monitors file access by a plurality of cache sites in the WAN separated distributed file system or clustered file system. The mechanism identifies access patterns by cache sites. The mechanism shares the access patterns with the plurality of cache sites. A given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

Abstract: A mechanism is provided in a data processing system for performing a remote direct memory access operation. Responsive to receiving in a network interface controller a hash value of data to be copied from a source address in a source node to a destination address in a destination node in the remote direct memory access operation, the network interface controller performs a lookup operation in a translation protection table in the network interface controller to match the hash value to a hash value for data existing in memory of the destination node. Responsive to the network interface controller finding a match in the translation protection table, the network interface controller completes the remote direct memory access operation without transferring the data from the source node to the destination node.

Abstract: A mechanism is provided in a data processing system for reliable asynchronous solid-state device based de-duplication. Responsive to receiving a write request to write data to the file system, the mechanism sends the write request to the file system, and in parallel, computes a hash key for the write data. The mechanism looks up the hash key in a de-duplication table. The de-duplication table is stored in a memory or a solid-state storage device. Responsive to the hash key not existing in the de-duplication table, the mechanism writes the write data to a storage device, writes a journal transaction comprising the hash key, and updates the de-duplication table to reference the write data in the storage device.

Abstract: A mechanism is provided in a data processing system for reliable asynchronous solid-state device based de-duplication. Responsive to receiving a write request to write data to the file system, the mechanism sends the write request to the file system, and in parallel, computes a hash key for the write data. The mechanism looks up the hash key in a de-duplication table. The de-duplication table is stored in a memory or a solid-state storage device. Responsive to the hash key not existing in the de-duplication table, the mechanism writes the write data to a storage device, writes a journal transaction comprising the hash key, and updates the de-duplication table to reference the write data in the storage device.

Abstract: A mechanism is provided in a data processing system for performing a remote direct memory access operation. Responsive to receiving in a network interface controller a hash value of data to be copied from a source address in a source node to a destination address in a destination node in the remote direct memory access operation, the network interface controller performs a lookup operation in a translation protection table in the network interface controller to match the hash value to a hash value for data existing in memory of the destination node. Responsive to the network interface controller finding a match in the translation protection table, the network interface controller completes the remote direct memory access operation without transferring the data from the source node to the destination node.

Abstract: A mechanism is provided in a data processing system for on-demand caching in a wide area network (WAN) separated distributed file system or clustered file system. The mechanism monitors file access by a plurality of cache sites in the WAN separated distributed file system or clustered file system. The mechanism identifies access patterns by cache sites. The mechanism shares the access patterns with the plurality of cache sites. A given cache site within the plurality of cache sites combines the access patterns with local access information and identifies files to pre-fetch based on the combined information.

Abstract: A mechanism is provided in a data processing system for reliable asynchronous solid-state device based de-duplication. Responsive to receiving a write request to write data to the file system, the mechanism sends the write request to the file system, and in parallel, computes a hash key for the write data. The mechanism looks up the hash key in a de-duplication table. The de-duplication table is stored in a memory or a solid-state storage device. Responsive to the hash key not existing in the de-duplication table, the mechanism writes the write data to a storage device, writes a journal transaction comprising the hash key, and updates the de-duplication table to reference the write data in the storage device.