Abstract: A system and method for transferring data between a user space buffer in the address space of a user space process running on a virtual machine and a storage system are described The user space buffer is represented as a file with a file descriptor in the method, a file system proxy receives a request for I/O read or write from the user space process without copying data to be transferred. The file system proxy then sends the request to a file system server without copying data to be transferred. The file system server then requests that the storage system perform the requested I/O directly between the storage system and the user space buffer, the only transfer of data being between the storage system and the user space buffer.

Abstract: Examples disclosed herein relate to propagating changes made on a file system volume of a primary cluster of nodes to the same file system volume also being managed by a secondary cluster of nodes. An application is executed on both clusters, and data changes on the primary cluster are mirrored to the secondary cluster using an exo-clone file. The exo-clone file includes the differences between two or more snapshots of the volume on the primary cluster, along with identifiers of the change blocks and (optionally) state information thereof. Just these changes, identifiers, and state information are packaged in the exo-clone file and then exported to the secondary cluster, which in turn makes the changes to its version of the volume. Exporting just the changes to the data blocks and the corresponding block identifiers drastically reduces the information needed to be exchanged and processed to keep the two volumes consistent.

Abstract: System and method for managing multiple data storages using a file system of a computer system utilize a primary data storage to cache objects of logical object containers stored in a secondary data storage in caching-tier volumes. When an access request for an object stored in the secondary data storage is received at the file system and the object is not currently cached in the primary data storage, a caching-tier volume in the primary data storage is created that corresponds to a logical object container in the secondary data storage that includes the requested object. The caching-tier volume is used to cache the object as an inflated file so that the inflated file is available at the primary data storage in the caching-tier volume for a subsequent access request for the object stored in the secondary data storage.

Abstract: The disclosure provides an approach for performing an operation by a first process on behalf of a second process, the method comprising: obtaining, by the first process, a memory handle from the second process, wherein the memory handle allows access, by the first process, to at least some of the address space of the second process; dividing the address space of the memory handle into a plurality of sections; receiving, by the first process, a request from the second process to perform an operation; determining, by the first process, a section of the plurality of sections that is to be mapped from the address space of the memory handle to the address space of the first process for the performance of the operation by the first process; mapping the section from the address space of the memory handle to the address space of the first process; and performing the operation by the first process on behalf of the second process.

Abstract: The disclosure provides an approach for performing an operation by a first process on behalf of a second process, the method comprising: obtaining, by the first process, a memory handle from the second process, wherein the memory handle allows access, by the first process, to at least some of the address space of the second process; dividing the address space of the memory handle into a plurality of sections; receiving, by the first process, a request from the second process to perform an operation; determining, by the first process, a section of the plurality of sections that is to be mapped from the address space of the memory handle to the address space of the first process for the performance of the operation by the first process; mapping the section from the address space of the memory handle to the address space of the first process; and performing the operation by the first process on behalf of the second process.

Abstract: The present disclosure provides techniques for managing a cache of a computer system using a cache management data structure. The cache management data structure includes a cold queue, a ghost queue, and a hot queue. The techniques herein improve the functioning of the computer because management of the cache management data structure can be performed in parallel with multiple cores or multiple processors, because a sequential scan will only pollute (i.e., add unimportant memory pages) cold queue, and to an extent, ghost queue, but not hot queue, and also because the cache management data structure has lower memory requirements and lower CPU overhead on cache hit than some prior art algorithms.

Abstract: The disclosure herein describes synchronizing a data cache and an LSM tree file system on an object storage platform. Instructions to send a cached data set from the data cache to the LSM tree file system are received. An updated metadata catalog is generated. If the LSM tree structure is out of shape, compaction is performed on the LSM tree file system which may be on a different system or server. When an unmerged compacted metadata catalog is identified, a merged metadata catalog is generated, based on the compacted metadata catalog and the cached data set, and associated with the cached data set. The cached data set and the associated metadata catalog are sent to the LSM tree file system, whereby the data cache and the LSM tree file system are synchronized. Synchronization is enabled without the data cache or file system being locked and/or waiting for the other entity.

Abstract: The disclosure herein describes synchronizing cached index copies at a first site with indexes of a log-structured merge (LSM) tree file system on an object storage platform at a second site. An indication that the LSM tree file system has been compacted based on a compaction process is received. A cached metadata catalog of the included parent catalog version at the first site is accessed. A set of cached index copies is identified at the first site based on the metadata of the cached metadata catalog. The compaction process is applied to the identified set of cached index copies and a compacted set of cached index copies is generated at the first site, whereby the compacted set of cached index copies is synchronized with a respective set of indexes of the plurality of sorted data tables of the LSM tree file system at the second site.

Abstract: Certain aspects provide systems and methods of compacting data within a log-structured merge tree (LSM tree) using sharding. In certain aspects, a method includes determining a size of the LSM tree, determining a compaction time for a compaction of the LSM tree based on the size, determining a number of compaction entities for performing the compaction in parallel based on the compaction time, determining a number of shards based on the number of compaction entities, and determining a key range associated with the LSM tree. The method further comprises dividing the key range by the number of shards into a number of sub key ranges, wherein each of the number of sub key ranges corresponds to a shard of the number of shards and assigning the number of shards to the number of compaction entities for compaction.

Abstract: The disclosure herein describes writing data to a log-structured merge (LSM) tree file system on an object storage platform. Write data instructions indicating data for writing to the LSM tree file system are received. Based on the received instructions, the data is written to the first data cache. Based on an instruction to transfer data in the live data cache to the LSM tree file system, the first data cache is converted to a stable cache. A second data cache configured as a live data cache is then generated based on cloning the first data cache. The data in the first data cache is then written to the LSM tree file system. Use of a stable cache and a cloned live data cache enables parallel writing data to the file system by the stable cache and handling write data instructions by the live data cache.

Abstract: The disclosure herein describes providing and accessing data on an object storage platform using a log-structured merge (LSM) tree file system. The LSM tree file system on the object storage platform includes sorted data tables, each sorted data table including a payload portion and an index portion. Data is written to the LSM tree file system in at least one new sorted data table. Data is ready by identifying a data location of the data based on index portions of the sorted data tables and reading the data from a sorted data table associated with the identified data location. The use of the LSM tree file system on the object storage platform provides an efficient means for interacting with the data stored thereon.

Abstract: Exemplary methods, apparatuses, and systems include a proxy intercepting, from a client, a plurality of requests addressed to a first server. The proxy adds an entry for each of the plurality of requests to a data structure. Each entry includes data from one of the plurality of requests. The proxy forwards each of the plurality of requests to the first server. In response to detecting the first server was unavailable or otherwise failed, the proxy reconnects to the first server when the first server recovers or connects to a second server serving as a backup for the first server. The proxy transmits to the first or second server one or more commands using the entries of the data structure to restore state information for the client.

Abstract: Exemplary methods, apparatuses, and systems include a proxy intercepting, from a client, a plurality of requests addressed to a first server. The proxy adds an entry for each of the plurality of requests to a data structure. Each entry includes data from one of the plurality of requests. The proxy forwards each of the plurality of requests to the first server. In response to detecting the first server was unavailable or otherwise failed, the proxy reconnects to the first server when the first server recovers or connects to a second server serving as a backup for the first server. The proxy transmits to the first or second server one or more commands using the entries of the data structure to restore state information for the client.

Abstract: System and method for managing multiple data storages using a file system of a computer system utilize a primary data storage to cache objects of logical object containers stored in a secondary data storage in caching-tier volumes. When an access request for an object stored in the secondary data storage is received at the file system and the object is not currently cached in the primary data storage, a caching-tier volume in the primary data storage is created that corresponds to a logical object container in the secondary data storage that includes the requested object. The caching-tier volume is used to cache the object as an inflated file so that the inflated file is available at the primary data storage in the caching-tier volume for a subsequent access request for the object stored in the secondary data storage.

Abstract: Examples disclosed herein relate to propagating changes made on a file system volume of a primary cluster of nodes to the same file system volume also being managed by a secondary cluster of nodes. An application is executed on both clusters, and data changes on the primary cluster are mirrored to the secondary cluster using an exo-clone file. The exo-clone file includes the differences between two or more snapshots of the volume on the primary cluster, along with identifiers of the change blocks and (optionally) state information thereof. Just these changes, identifiers, and state information are packaged in the exo-clone file and then exported to the secondary cluster, which in turn makes the changes to its version of the volume. Exporting just the changes to the data blocks and the corresponding block identifiers drastically reduces the information needed to be exchanged and processed to keep the two volumes consistent.

Abstract: In a computer system, a method of controls interrupts which correspond to input/output (I/O) processing. For each delivery of an I/O completion interrupt, the method provides a recordation of a delivery time; identifies I/O completions for which deliveries of corresponding I/O completion interrupts involve deliveries of inter-processor interrupts; and for each of the identified I/O completions, accesses the recordation of the most recent delivery time to determine whether a selected period of time has elapsed since a last delivery of an inter-processor interrupt. As a response to a determination that the selected period has elapsed, an inter-processor interrupt is delivers. As a response to a determination that less than the duration of the selected period has elapsed, the method refrains from delivering an inter-processor interrupt.

Abstract: In a virtual machine, swap activities of a hypervisor and a guest OS are reconciled so that redundant input-output operations (IOs) can be avoided and a synchronous response time of the virtual machine improved. This is achieved with a map of memory pages to blocks of storage. For a write IO to write contents of a memory page into a target block, the map is examined to see if it contains a valid entry for the memory page. If the map contains the valid entry, the write IO is prevented from being issued and a data structure is updated so that subsequent IOs to the target block is redirected from the target block to a block that is associated with the physical memory page in the valid entry. On the other hand, if the map does not contain the valid entry, the write IO is issued.

Abstract: Completion interrupts corresponding to I/O requests issued by a virtual machine guest, which runs on a host platform, are virtualized in such a way that I/O completion interrupts to the requesting guest are delivered no faster than it can stably handle them, but, when possible, faster than the nominal speed of a virtual device to which a virtual machine addresses the I/O request. In general, completion events received from the host platform in response to guest I/O requests are examined with respect to time. If enough time has passed that the virtual device would normally have completed the I/O request, then the completion interrupt is delivered to the guest. If the nominal time has not elapsed, however, the invention enqueues and time-stamps the event and delivers it at the earliest of a) the normal maturity time, or b) at a safepoint.

Abstract: In order to synchronize copies of a virtual disk, a virtualization layer maintains a first record of file system blocks of a first copy of the virtual disk that are modified during an access session by a virtual machine using the first copy of the virtual disk. The file system blocks correspond to a file system of the virtual disk. During an attempt to synchronize the first copy with a second copy of the virtual disk, (i) a second record of file system blocks that are currently used by the file system is obtained from the guest operating system, and (ii) file system blocks in the first copy of the virtual disk that are present in both the first record and the second record are copied into the second copy of the virtual disk.

Abstract: Completion interrupts corresponding to I/O requests issued by a virtual machine guest, which runs on a host platform, are virtualized in such a way that I/O completion interrupts to the requesting guest are delivered no faster than it can stably handle them, but, when possible, faster than the nominal speed of a virtual device to which a virtual machine addresses the I/O request. In general, completion events received from the host platform in response to guest I/O requests are examined with respect to time. If enough time has passed that the virtual device would normally have completed the I/O request, then the completion interrupt is delivered to the guest. If the nominal time has not elapsed, however, the invention enqueues and time-stamps the event and delivers it at the earliest of a) the normal maturity time, or b) at a safepoint.