Abstract: Various embodiments set forth techniques for erasure coding of replicated data blocks. The techniques include receiving, by a pre-designated node, data associated with an erasure coded strip from a first node; receiving, by the pre-designated node, a replica for a first data block; saving the replica in an erasure coded strip; and in response to a trigger condition, replacing, by the pre-designated node, the replica and at least one replica of a second data block with an error correction block.

Abstract: Various embodiments set forth techniques for erasure coding of replicated data blocks. The techniques include receiving, by a pre-designated node, data associated with an erasure coded strip from a first node; receiving, by the pre-designated node, a replica for a first data block; saving the replica in an erasure coded strip; and in response to a trigger condition, replacing, by the pre-designated node, the replica and at least one replica of a second data block with an error correction block.

Abstract: Systems and methods for iterative, high-performance, low-latency data replication. A method embodiment commences upon identifying one or more replica target nodes to receive replicas of working data. Steps of the method then compose at least one replication message. The replication message includes the location or contents of working data as well as a listing of downstream replica target nodes. The replication capacity is measured at the subject node. Based on the measured replication capacity, the subject node sends instructions in the replication message to one or more downstream replica target nodes. Any one or more of the downstream replica target nodes receives the instructions and iterates the steps of measuring its own capacity and determining the instructions, if any, to send to further downstream replica target nodes. Each replica target node replicates the working data. In some cases, the measured replication capacity is enough to perform all replications in parallel.

Abstract: Systems and methods for demand-based remote direct memory access buffer management. A method embodiment commences upon initially partitioning a memory pool at a computer that is to receive memory contents from a sender. The memory pool is partitioned into memory areas that comprise a plurality of different sized buffers that serve as target buffers for one or more direct memory access data transfer operations from the data sources. An initial first set of buffer apportionments are associated with each one of the one or more data sources and those initial sets are advertised to the corresponding data sources. Over time, based on messages that have been loaded into the receiver's memory, the payload sizes of the messages are observed. Based on the observed the demand for buffers that are used for the message payload, the constituency of the advertised buffers can grow or shrink elastically as compared to previous advertisements.

Abstract: A method and apparatus for using cache size estimations for guiding hot-tier insertion decisions. The method and apparatus include an adaptive management element that determines what accesses of a resource should be logged and the parameters for logging. The determinations are used to configure an adaptive logging element to log only accesses corresponding to the selected resource(s) and to log only those accesses that match the identified parameters. The adaptive management element operates in a feedback loop: first determining what will be logged and second implementing that determination by an adaptive logging element. Upon a triggering event, the process returns to the first determination based on any then current parameters. In some embodiments, the parameters include a size estimate to achieve a given target hit rate (target hit rate size estimate) that is used in generating a score or weighting to identify the highest/best scoring/weighted disk(s) for logging.

Abstract: Systems and methods for iterative, high-performance, low-latency data replication. A method embodiment commences upon identifying one or more replica target nodes to receive replicas of working data. Steps of the method then compose at least one replication message. The replication message includes the location or contents of working data as well as a listing of downstream replica target nodes. The replication capacity is measured at the subject node. Based on the measured replication capacity, the subject node sends instructions in the replication message to one or more downstream replica target nodes. Any one or more of the downstream replica target nodes receives the instructions and iterates the steps of measuring its own capacity and determining the instructions, if any, to send to further downstream replica target nodes. Each replica target node replicates the working data. In some cases, the measured replication capacity is enough to perform all replications in parallel.

Abstract: Systems and methods for demand-based remote direct memory access buffer management. A method embodiment commences upon initially partitioning a memory pool at a computer that is to receive memory contents from a sender. The memory pool is partitioned into memory areas that comprise a plurality of different sized buffers that serve as target buffers for one or more direct memory access data transfer operations from the data sources. An initial first set of buffer apportionments are associated with each one of the one or more data sources and those initial sets are advertised to the corresponding data sources. Over time, based on messages that have been loaded into the receiver's memory, the payload sizes of the messages are observed. Based on the observed the demand for buffers that are used for the message payload, the constituency of the advertised buffers can grow or shrink elastically as compared to previous advertisements.

Abstract: A method and apparatus for using cache size estimations for guiding hot-tier insertion decisions. The method and apparatus include an adaptive management element that determines what accesses of a resource should be logged and the parameters for logging. The determinations are used to configure an adaptive logging element to log only accesses corresponding to the selected resource(s) and to log only those accesses that match the identified parameters. The adaptive management element operates in a feedback loop: first determining what will be logged and second implementing that determination by an adaptive logging element. Upon a triggering event, the process returns to the first determination based on any then current parameters. In some embodiments, the parameters include a size estimate to achieve a given target hit rate (target hit rate size estimate) that is used in generating a score or weighting to identify the highest/best scoring/weighted disk(s) for logging.

Abstract: Disclosed is an improved approach to implement memory-efficient cache size estimations. A HyperLogLog is used to efficiently approximate an MRC with sufficient granularity to size caches.

Abstract: Embodiments serve to balance overall performance of a finite-sized caching system having a first cache of a first cache size and a second cache of a second cache size. A tail portion and a head portion of each of the caches are defined wherein incoming data elements are initially stored in a respective head portion and wherein evicted data elements are evicted from a respective tail portion. Performance metrics are defined wherein a performance metric includes a predicted miss cost that would be incurred when replacing an evicted data elements. A quantitative function is defined to include cache performance metrics and a cache reallocation amount. The cache performance metrics are evaluated periodically to determine a then-current cache reallocation amount. The caches can be balanced by increasing the first cache size by the cache reallocation amount and decreasing the second cache size by the cache reallocation amount.

Abstract: Embodiments serve to balance overall performance of a finite-sized caching system having a first cache of a first cache size and a second cache of a second cache size. A tail portion and a head portion of each of the caches are defined wherein incoming data elements are initially stored in a respective head portion and wherein evicted data elements are evicted from a respective tail portion. Performance metrics are defined wherein a performance metric includes a predicted miss cost that would be incurred when replacing an evicted data elements. A quantitative function is defined to include cache performance metrics and a cache reallocation amount. The cache performance metrics are evaluated periodically to determine a then-current cache reallocation amount. The caches can be balanced by increasing the first cache size by the cache reallocation amount and decreasing the second cache size by the cache reallocation amount.