Abstract: Methods and systems are provided for improving user authentication and access control by a network file system service in a multi-tenant public cloud environment by receiving a request for a connection to a file system from a file system client (client), sending an identification request for identification authentication of the client to a control system, receiving a response from the control system, establishing the connection to the file system upon determining that the connection to the file system is allowed based on cloud tenant information associated with the client, receiving an attempt to access the file system from the client by a sub-user, authenticating the sub-user based on the cloud tenant information, issuing a security token including a globally unique sub-user identifier of the sub-user, and using the security token to determine access rights of the sub-user to the file system for a subsequent request.

Abstract: Methods and systems implement local file systems of computing nodes which translate random-access read and random-access write operation calls to append-based operation calls in accordance with a distributed file system (“DFS”) implemented across storage nodes of a cloud network. Computer-executable applications running on the computing nodes may generate kernel-level read and write system calls by application programming interfaces (“APIs”) such as the Portable Operating System Interface (“POSIX”) standard, to a local file system. The local file system may translate these read and write system calls to file operations at a DFS implementing an append-only file system, as well as perform storage reclamation upon the DFS periodically and/or upon storage thresholds being exceeded.

Abstract: A key-value engine may perform garbage collection for a tree or hierarchical data structure on an append-only storage device with page mappings. The key-value engine may separate hot and cold data to reduce write amplification, track extent usage using a restricted or limited amount of memory, efficiently answer queries of valid extent usage, and adaptively or selectively defragment pages in snapshots in rounds of garbage collection.

Abstract: Embodiments of the present invention relate to synchronously replicating data in a distributed computing environment. To achieve synchronous replication both an eventual consistency approach and a strong consistency approach are contemplated. Received data may be written to a log of a primary data store for eventual committal. The data may then be annotated with a record, such as a unique identifier, which facilitates the replay of the data at a secondary data store. Upon receiving an acknowledgment that the secondary data store has written the data to a log, the primary data store may commit the data and communicate an acknowledgment of success back to the client. In a strong consistency approach, the primary data store may wait to send an acknowledgement of success to the client until it receives an acknowledgment that the secondary has not only written, but also committed, the data.

Abstract: A key-value engine of a storage system may perform a restart recovery after a system failure. The key-value engine may read a metadata log to locate a latest system checkpoint, and load a page mapping table from the latest system checkpoint. The key-value engine may replay to apply changes to the page mapping table from a system transaction log starting from a system transaction replay starting point. The key-value engine may further form one or more read-only replicas using an underlying file stream opened in a read-only mode during the recovery after the system failure to further facilitate fast recovery and provide fast response to user transactions that conduct read only transactions after the system failure.

Abstract: Systems and methods discussed herein, based on a key-value data store including multiple-tiered sorted data structures in memory and storage, implement granularly timestamped concurrency control. The multiple-tiering of the key-value data store enables resolving the snapshot queries by returning data record(s) according to granularly timestamped snapshot lookup instead of singularly indexed snapshot lookup. Queries return a merged collection of records including updates from data structures in memory and in storage, such that a persistent storage transaction may refer to non-committed updates up to a timeframe defined by the snapshot read timestamp. This way, inconsistency is avoided that would result from merely reading data records committed in storage, without regard as to pending, non-committed updates thereto.

Abstract: Embodiments of the present invention relate to synchronously replicating data in a distributed computing environment. To achieve synchronous replication both an eventual consistency approach and a strong consistency approach are contemplated. Received data may be written to a log of a primary data store for eventual committal. The data may then be annotated with a record, such as a unique identifier, which facilitates the replay of the data at a secondary data store. Upon receiving an acknowledgment that the secondary data store has written the data to a log, the primary data store may commit the data and communicate an acknowledgment of success back to the client. In a strong consistency approach, the primary data store may wait to send an acknowledgement of success to the client until it receives an acknowledgment that the secondary has not only written, but also committed, the data.

Abstract: In various embodiments, methods and systems for providing access to file systems are provided. An SFA-based operation having an operation modifier for a file in a file system is received. The SFA-based operation is defined based on a file-representational state transfer (REST) interface. The SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol. The SFA-based operation is executed using the operation modifier. A REST-based operation for the file in the distributed file storage is received. The REST-based operation is defined based on the SFA-REST interface. The operation modifier is referenced in order to execute the REST-based operation. The REST-based operation based on the SFA-REST interface. A schema can be implemented for accessing the file system. The schema includes tables to store files, the table comprises fields corresponding to elements of a SFA-REST interface. A snapshot component can be implemented to backup file shares in the file system.

Abstract: A power management system may provide power management recommendations to a computer system including a plurality of computing nodes (which may include processors, etc.), to cause the computing nodes to individually or collectively adjust power states or modes of respective processors to achieve power management of the computer system. The power management system may be provided with a power management framework that continuously utilizes direct and indirect service-level feedbacks to guide power management decisions. The power management system may employ a reinforcement learning algorithm to make power management decisions at a user level, and provide a fast decision overriding mechanism for platform events or service-requested performance boosts.

Abstract: Presented herein are methods of replicating versioned and hierarchical data structures, as well as data structures representing complex transactions. Due to interdependencies between data entities and a lack of guaranteed message ordering, simple replication methods employed for simple data types cannot be used. Operations on data structures exhibit dependencies between the messages making up the operations. This strategy can be extended to various types of complex transactions by considering certain messages to depend on other messages or on the existence of other entries at the data store. Regardless of origin, these dependencies can be enforced by suspending the processing of messages with unsatisfied dependencies until all of its dependencies have been met. Alternately, transactions can be committed immediately, creating entities that include versioned identifiers for each of their dependencies. These entities can then be garbage collected of the parent objects are not subsequently created.

Abstract: A mapping correspondence between memory addresses and request counts and a cache line flusher are provided, enabling selective cache flushing for persistent memory in a computing system to optimize write performance thereof. Random writes from cache memory to persistent memory are prevented from magnifying inherent phenomena of write amplification, enabling computing systems to implement persistent memory as random-access memory, at least in part. Conventional cache replacement policies may remain implemented in a computing system, but may be effectively overridden by operations of a cache line flusher according to example embodiments of the present disclosure preventing conventional cache replacement policies from being triggered. Implementations of the present disclosure may avoid becoming part of the critical path of a set of computer-executable instructions being executed by a client of cache memory, minimizing additional computation overhead in the critical path.

Abstract: A key-value store is provided, implementing multiple-tiered sorted data structures in memory and storage, including concurrent write buffers in memory, and page-level consolidation of updates on storage, where pages are trivially translated in physical-to-virtual address mapping. The key-value store is built on an indexed sorted data structure on storage, occupying much less storage space and incurring much less disk activity in consolidating updates than a conventional log-structured merge tree organized into files. Concurrent write buffers operate concurrently and independently so that data is committed from memory to storage in an efficient manner, while maintaining chronological sequence of delta pages. Trivial mapping allows mappings of a number of physical pages to be omitted, enabling page mapping tables to occupy less storage space, and simplifying processing workload of read operation retrievals from storage.

Abstract: A key-value engine of a storage system may perform a restart recovery after a system failure. The key-value engine may read a metadata log to locate a latest system checkpoint, and load a page mapping table from the latest system checkpoint. The key-value engine may replay to apply changes to the page mapping table from a system transaction log starting from a system transaction replay starting point. The key-value engine may further form one or more read-only replicas using an underlying file stream opened in a read-only mode during the recovery after the system failure to further facilitate fast recovery and provide fast response to user transactions that conduct read only transactions after the system failure.

Abstract: Systems and methods discussed herein, based on a key-value data store including multiple-tiered sorted data structures in memory and storage, implement granularly timestamped concurrency control. The multiple-tiering of the key-value data store enables resolving the snapshot queries by returning data record(s) according to granularly timestamped snapshot lookup instead of singularly indexed snapshot lookup. Queries return a merged collection of records including updates from data structures in memory and in storage, such that a persistent storage transaction may refer to non-committed updates up to a timeframe defined by the snapshot read timestamp. This way, inconsistency is avoided that would result from merely reading data records committed in storage, without regard as to pending, non-committed updates thereto.

Abstract: A key-value engine may perform garbage collection for a tree or hierarchical data structure on an append-only storage device with page mappings. The key-value engine may separate hot and cold data to reduce write amplification, track extent usage using a restricted or limited amount of memory, efficiently answer queries of valid extent usage, and adaptively or selectively defragment pages in snapshots in rounds of garbage collection.

Abstract: Methods and systems implement local file systems of computing nodes which translate random-access read and random-access write operation calls to append-based operation calls in accordance with a distributed file system (“DFS”) implemented across storage nodes of a cloud network. Computer-executable applications running on the computing nodes may generate kernel-level read and write system calls by application programming interfaces (“APIs”) such as the Portable Operating System Interface (“POSIX”) standard, to a local file system. The local file system may translate these read and write system calls to file operations at a DFS implementing an append-only file system, as well as perform storage reclamation upon the DFS periodically and/or upon storage thresholds being exceeded.

Abstract: A power management system may provide power management recommendations to a computer system including a plurality of computing nodes (which may include processors, etc.), to cause the computing nodes to individually or collectively adjust power states or modes of respective processors to achieve power management of the computer system. The power management system may be provided with a power management framework that continuously utilizes direct and indirect service-level feedbacks to guide power management decisions. The power management system may employ a reinforcement learning algorithm to make power management decisions at a user level, and provide a fast decision overriding mechanism for platform events or service-requested performance boosts.

Abstract: A mapping correspondence between memory addresses and request counts and a cache line flusher are provided, enabling selective cache flushing for persistent memory in a computing system to optimize write performance thereof. Random writes from cache memory to persistent memory are prevented from magnifying inherent phenomena of write amplification, enabling computing systems to implement persistent memory as random-access memory, at least in part. Conventional cache replacement policies may remain implemented in a computing system, but may be effectively overridden by operations of a cache line flusher according to example embodiments of the present disclosure preventing conventional cache replacement policies from being triggered. Implementations of the present disclosure may avoid becoming part of the critical path of a set of computer-executable instructions being executed by a client of cache memory, minimizing additional computation overhead in the critical path.

Abstract: Methods and systems are provided for improving a journaling file system by reducing journaling overhead utilizing a remapping interface to log-structured cloud block devices such that blocks in a journaling transaction are atomically mapped to their final disk locations with minimum overhead. With log-structured cloud block devices, all blocks may be stored in logfiles and indexed for access. By modifying the index of the destination block to point to the corresponding journal block, the additional write may be avoided, and a journal block may effectively be remapped from its original LBA to its destination LBA without any actual data movement, thereby reducing the file system journaling overhead.

Abstract: Indexed data structures are provided which are optimized for read and write performance in persistent memory of computing systems. Stored data may be searched by traversing an indexed data structure while still being sequentially written to persistent memory, so that the stored data may be accessed more efficiently than on non-volatile storage, while maintaining persistence against system failures such as power cycling. Mapping correspondences between leaf nodes of an indexed data structure and sequential elements of a sequential data structure may be stored in RAM, facilitating fast random access. Data writes are recorded as appended delta encodings which may be periodically compacted, avoiding write amplification inherent in persistent memory.