Abstract: A method for direct addressing in virtual addressed systems includes obtaining, by a system including a processor and in response to receiving a request to access a data item stored by a storage system, address information for the data item from a data structure. The address information includes a first physical storage address of the storage system, a first generation number associated the first physical storage address in the data structure, and an address redirector. The method also includes accessing, by the system and in response to the first generation number being determined to be different from a second generation number associated with the first physical storage address, the data item at a second physical storage address of the storage system instead of the first physical storage address, the second physical storage address being determined based on the address redirector.

Abstract: A method for efficient ingest tier references via virtual addresses includes generating, by a system including a processor, an address redirector for a data block in response to the data block being written to a first storage location, the address redirector including a reference to the first storage location. The method also includes writing, by the system, data including the data block to a second storage location in response to an amount of the data to be written to the second storage location being determined to be at least a threshold amount. The method further includes altering, by the system in response to the writing of the data to the second storage location, the address redirector to include a reference to the second storage location.

Abstract: A method for direct addressing in virtual addressed systems includes obtaining, by a system including a processor and in response to receiving a request to access a data item stored by a storage system, address information for the data item from a data structure. The address information includes a first physical storage address of the storage system, a first generation number associated the first physical storage address in the data structure, and an address redirector. The method also includes accessing, by the system and in response to the first generation number being determined to be different from a second generation number associated with the first physical storage address, the data item at a second physical storage address of the storage system instead of the first physical storage address, the second physical storage address being determined based on the address redirector.

Abstract: Data lock transitioning associated with writing of files can be performed and managed. In connection with a write operation to write a portion of a file to a node of a node network, an access management component (AMC) can analyze file-related information indicative of a status associated with the portion of the file, wherein the status can relate to whether the file portion is associated with a streaming write, whether a protocol indication indicates a file handle associated with the file has been closed, or whether an advisory lock has been released by a client device. The AMC can determine whether to transition from an exclusive data lock associated with the portion of the file to a shared data lock with respect to the portion of the file, based on a result of analyzing the file-related information. The AMC can fine tune determinations or predictions of data lock transitions.

Abstract: Data lock transitioning associated with writing of files can be performed and managed. In connection with a write operation to write a portion of a file to a node of a node network, an access management component (AMC) can analyze file-related information indicative of a status associated with the portion of the file, wherein the status can relate to whether the file portion is associated with a streaming write, whether a protocol indication indicates a file handle associated with the file has been closed, or whether an advisory lock has been released by a client device. The AMC can determine whether to transition from an exclusive data lock associated with the portion of the file to a shared data lock with respect to the portion of the file, based on a result of analyzing the file-related information. The AMC can fine tune determinations or predictions of data lock transitions.

Abstract: The described technology is generally directed towards maintaining a heat score per file based on a file's access/activity level, and using the heat score to re-tier files from faster to slower storage and vice-versa. Files on a slower tier that have more activity are thereby moved (“up-tiered”) to a faster access storage tier, while the files that have less activity on the faster access storage tier are moved (“down-tiered”) to a slower access storage tier. The heat score for a file is increased based on more input-output operation activity, and decayed over time. The decaying can be based on a relative importance value (e.g., business value) assigned to the file or a domain that includes the file. Ordering the files based on their respective heat scores allows only a fraction of the files to be considered for re-tiering before the re-tiering moves are performed.

Abstract: Systems and methods facilitating fault tolerance for transaction mirroring are described herein. A method as described herein can include receiving a commit command for a data transaction from an initiator node of the system, wherein the data transaction is associated with a first failure domain, and wherein the commit command is directed to a primary participant node and a secondary participant node of the system; determining whether a response to the commit command has been received at the primary participant node from the secondary participant node in response to the receiving; and, in response to determining that the response to the commit command was not received at the primary participant node, indicating that the secondary participant node is invalid in a data store associated with a second failure domain that is distinct from the first failure domain.

Abstract: Techniques for address mirroring of a file system journal are presented. A transactional file system update can correspond to a write operation to be performed at a first participant node of a data storage system. A journal transfer component can transfer the transactional file system update from an initiator node of the data storage system to a first journal associated with the first participant node and recovery information relating to the update to a second journal associated with a second participant node of the data storage system. The recovery information can be determined based on the update. To facilitate the update, a journal descriptor block can be generated and sent to the second participant node, such block comprising the recovery information, which can include transaction state, block addresses involved in the transaction, error correction-related information, or a mirror block address set for the block addresses where desired.

Abstract: Systems and methods for mirroring a file system journal are described herein. A method as described herein can include receiving, by a system operatively coupled to a processor, transactional file system updates corresponding to a write operation to be performed at a first node of the system; transferring, by the system, the transactional file system updates from an initiator node of the system to a first journal at the first node and a second journal at a second node of the system that is logically distinct from the first node and the initiator node; and committing, by the system, the transactional file system updates to the first journal and the second journal.

Abstract: Direct read in clustered file systems is described herein. A method as described herein can include determining, for a write operation on a resource stored by a data storage system, as initiated by an initiator node, a reference count for the resource, the reference count comprising a number of target storage regions of the data storage system to be modified by write data during the write operation; facilitating conveying, from the initiator node to a lock coordinator node, the reference count for the resource; facilitating conveying, from the initiator node to respective participant nodes that are respectively assigned to the target storage regions, the write data and a key value for the write operation; and facilitating causing the respective participant nodes to convey respective notifications that comprise the key value in response to the respective participant nodes writing the write data to the target storage regions.

Abstract: Systems and methods facilitating fault tolerance for transaction mirroring are described herein. A method as described herein can include receiving a commit command for a data transaction from an initiator node of the system, wherein the data transaction is associated with a first failure domain, and wherein the commit command is directed to a primary participant node and a secondary participant node of the system; determining whether a response to the commit command has been received at the primary participant node from the secondary participant node in response to the receiving; and, in response to determining that the response to the commit command was not received at the primary participant node, indicating that the secondary participant node is invalid in a data store associated with a second failure domain that is distinct from the first failure domain.

Abstract: Techniques for address mirroring of a file system journal are presented. A transactional file system update can correspond to a write operation to be performed at a first participant node of a data storage system. A journal transfer component can transfer the transactional file system update from an initiator node of the data storage system to a first journal associated with the first participant node and recovery information relating to the update to a second journal associated with a second participant node of the data storage system. The recovery information can be determined based on the update. To facilitate the update, a journal descriptor block can be generated and sent to the second participant node, such block comprising the recovery information, which can include transaction state, block addresses involved in the transaction, error correction-related information, or a mirror block address set for the block addresses where desired.

Abstract: Systems and methods for mirroring a file system journal are described herein. A method as described herein can include receiving, by a system operatively coupled to a processor, transactional file system updates corresponding to a write operation to be performed at a first node of the system; transferring, by the system, the transactional file system updates from an initiator node of the system to a first journal at the first node and a second journal at a second node of the system that is logically distinct from the first node and the initiator node; and committing, by the system, the transactional file system updates to the first journal and the second journal.

Abstract: Pre-decompressing a compressed form of data that has been pre-fetched into a cache to facilitate subsequent retrieval of a decompressed form of the data from the cache is presented herein. A system retrieves, from a first portion of a cache, a compression chunk comprising compressed data blocks representing a compressed form of a group of data blocks in response to a first cache hit from the first portion of the cache being incurred, decompresses the compression chunk to obtain a decompressed chunk comprising uncompressed data blocks representing an uncompressed form of the group of data blocks, and inserts the uncompressed data blocks into a second portion of the cache. Further, the system retrieves, from the second portion of the cache, an uncompressed data block of the uncompressed data blocks in response to a second cache hit from the second portion of the cache being incurred.

Abstract: Compression of data for a file system utilizing protection groups can be implemented and managed. A compression management component (CMC) can control compression of data via inline or post-process compression for storage in protection groups in memory, including determining whether to compress data, determining a compression algorithm to utilize to compress data, and/or determining whether to perform inline and/or post-process compression of data. CMC can generate protection group (PG) metadata for a PG in which compressed data is stored. PG metadata can comprise a logical extent map that describes which logical blocks contain compressed data, a list of cyclic redundancy check values for logical blocks, and a list of compression chunks that store individual metadata regarding individual compressed streams, wherein, for an individual compressed stream, the individual metadata comprises a compression format, compressed size, uncompressed size, and/or starting offset in physical space within the PG.

Abstract: Implementations are provided herein for having at least two data streams associated with each file in a file system. The first, a cache overlay layer, can store additional state information on a per block basis that details whether each individual block of file data within the cache overlay layer is clean, dirty, or indicates that a write back to the storage layer is in progress. The second, a storage layer, can be a use case defined repository that can transform data using data augmentation methods as a part of tiering data to external repositories. File system operations directed to the cache overlay layer can be processed asynchronously from file system operations directed to the storage layer.

Abstract: Techniques are provided for computer memory mapping and allocation. In an example, a virtual memory address space is divided into an active half and a passive half. Processors make memory allocations to their respective portions of the active half until one processor has made a determined number of allocations. When that occurs, and when all memory in the passive half that has been allocated has been returned, then the active and passive halves are switched, and all processors are switched to making allocations in the newly-active half.

Abstract: Pre-decompressing a compressed form of data that has been pre-fetched into a cache to facilitate subsequent retrieval of a decompressed form of the data from the cache is presented herein. A decompression component retrieves, from a first portion of a cache, a compression chunk comprising compressed data blocks representing a compressed form of a group of data blocks in response to a first cache hit from the first portion of the cache being incurred, decompresses the compression chunk to obtain a decompressed chunk comprising uncompressed data blocks representing an uncompressed form of the group of data blocks, and inserts the uncompressed data blocks into a second portion of the cache. Further, a read component retrieves, from the second portion of the cache, an uncompressed data block of the uncompressed data blocks in response to a second cache hit from the second portion of the cache being incurred.

Abstract: Compression of data for a file system utilizing protection groups can be implemented and managed. A compression management component (CMC) can control compression of data via inline or post-process compression for storage in protection groups in memory, including determining whether to compress data, determining a compression algorithm to utilize to compress data, and/or determining whether to perform inline and/or post-process compression of data. CMC can generate protection group (PG) metadata for a PG in which compressed data is stored. PG metadata can comprise a logical extent map that describes which logical blocks contain compressed data, a list of cyclic redundancy check values for logical blocks, and a list of compression chunks that store individual metadata regarding individual compressed streams, wherein, for an individual compressed stream, the individual metadata comprises a compression format, compressed size, uncompressed size, and/or starting offset in physical space within the PG.

Abstract: Techniques are provided for computer memory mapping and allocation. In an example, a virtual memory address space is divided into an active half and a passive half. Processors make memory allocations to their respective portions of the active half until one processor has made a determined number of allocations. When that occurs, and when all memory in the passive half that has been allocated has been returned, then the active and passive halves are switched, and all processors are switched to making allocations in the newly-active half.