Abstract: Examples herein include a computer system and methods. Some computer systems comprise two or more devices (each device comprises at least one processing circuit), where each computing device comprises or is communicatively coupled to one or more optical network interface controller (O-NIC) cards. Each O-NIC card comprises at least two bidirectional optical channels to transmit data and to receive additional data from each O-NIC card communicatively coupled to a device, over a channel. The system also includes one or more interfaces and a memory. Program instructions execute a method on one or more processors in communication with a memory, and the method includes modifying, during runtime of at least one application, a pairing over a given bidirectional optical channel of an interface of the interfaces to a given device.

Abstract: Techniques for balancing data compression and read performance of data chunks of a storage system are described herein. According to one embodiment, similar data chunks are identified based on sketches of a plurality of data chunks stored in the storage system. A first portion of the similar data chunks as a first group is associated with a first storage area. The first storage area is associated with one or more data chunks that are dissimilar to the first group but are likely accessed together. The first group of the similar data chunks and its associated dissimilar data chunks are compressed and stored in the first storage area.

Abstract: Use of a survival queue to manage a container-based flash cache is disclosed. In various embodiments, a corresponding survival time is associated with each of a plurality of containers stored in a flash cache, each container comprising a plurality of data blocks. The survival time may be determined based at least in part on a calculated proportion of relatively recently accessed data blocks associated with the container is associated with the container. A container to evict from the flash cache is selected based at least in part on a determination that the corresponding survival time of the selected container has expired.

Abstract: Techniques to manage usage of a flash-based storage are disclosed. In various embodiments, the execution time of the flash-based storage is divided into quanta. Within each quantum comprising at least a subset of quanta, flash erasures are allowed without restriction up to a prescribed erasure quota. Erasures are throttled within a slack range bound at a lower end by the erasure quota and at an upper end by an upper bound, including by dividing the slack range into two or more intervals and within each interval applying a corresponding erasure control policy, wherein the respective corresponding erasure control policies applied to successive intervals in the slack range become increasingly strict in a stepwise manner.

Abstract: Techniques for improving data compression of a storage system using coarse and fine grained similarity are described herein. According to one embodiment, region sketches for a plurality of regions of the set of data are generated, each region storing a plurality of data chunks. A region sketch index having a plurality of entries is maintained, each corresponding to one of the region sketches of the regions. The entries of the region sketch index are sorted based on the sketches of the regions, such that regions with an identical region sketch are positioned adjacent to each other within the region sketch index, representing similar regions. The data chunks of the similar regions that are identified based on the sorted entries of the region sketch index are reorganized to improve data compression of the data chunks of the similar regions.

Abstract: Cassandra SSTable data is transformed to provide data rows that are a consistent size such that data in each row has a length that is contained within a selected fixed sized kilobyte segment for deduplication. Tables of a Cassandra cluster node are translated in parallel to JSON format using Cassandra SSTableDump and the table rows are parsed to provide data rows corresponding to the data in each table row. Each row of data is padded with a predictable pattern of bits such that the data row has a length corresponding to the selected fixed segment size and has boundary locations that correspond to multiple of the selected segment size. Since each row of data starts on a segment boundary, duplicate rows of data will be identified wherever they move within a table.

Abstract: In response to a request for accessing a file stored in a storage system, data objects associated with the file are retrieved from a storage device of the storage system. The data objects of the file are cached in a cache memory. An access sequence of the cached data objects within the file is determined based on metadata of the file, where the access sequence represents a sequential order in time of accessing the cached data objects within the file. In response to a request for cache space reclamation, one or more cached data objects are identified whose next access is a farthest in time from a data object currently being accessed amongst the cached data objects based on the access sequence of the data objects. The identified data objects are evicted from the cache memory whose next access is a farthest amongst the cached data objects.

Abstract: Cassandra SSTable data is transformed to provide data rows that are a consistent size such that data in each row has a length that is contained within a selected fixed sized kilobyte segment for deduplication. Tables of a Cassandra cluster node are translated in parallel to JSON format using Cassandra SSTableDump and the table rows are parsed to provide data rows corresponding to the data in each table row. Each row of data is padded with a predictable pattern of bits such that the data row has a length corresponding to the selected fixed segment size and has boundary locations that correspond to multiple of the selected segment size. Since each row of data starts on a segment boundary, duplicate rows of data will be identified wherever they move within a table.

Abstract: A system for directing for storage comprises a processor and a memory. The processor is configured to determine a segment overlap for each of a plurality of nodes. The processor is further configured to determine a selected node of the plurality of nodes based at least in part on the segment overlap for each of the plurality of nodes and based at least in part on a selection criteria. The memory is coupled to the processor and configured to provide the processor with instructions.

Abstract: Techniques for balancing data compression and read performance of data chunks of a storage system are described herein. According to one embodiment, similar data chunks are identified based on sketches of a plurality of data chunks stored in the storage system. A first portion of the similar data chunks as a first group is associated with a first storage area. The first storage area is associated with one or more data chunks that are dissimilar to the first group but are likely accessed together. The first group of the similar data chunks and its associated dissimilar data chunks are compressed and stored in the first storage area.

Abstract: Techniques for balancing data compression and read performance of data chunks of a storage system are described herein. According to one embodiment, similar data chunks are identified based on sketches of a plurality of data chunks stored in the storage system. A first portion of the similar data chunks as a first group is associated with a first storage area. The first storage area is associated with one or more data chunks that are dissimilar to the first group but are likely accessed together. The first group of the similar data chunks and its associated dissimilar data chunks are compressed and stored in the first storage area.

Abstract: A domain name server request is received from a local domain name server. Operational performance parameters of a web host and a first content distribution network are received. Aggregate operational performance parameters of a second content distribution network are received. A first record is sent to the local domain name server that indicates the domain name server request is to be handled by the web host on a condition that a performance value is below a predetermined operational performance level threshold. A second record is sent to the local domain name server that indicates the domain name server request is to be handled by the first or the second content distribution network on a condition that the performance value is equal to or above the predetermined operational performance level threshold.

Abstract: One embodiment provides a document management system comprising a storage system to store one or more encrypted documents, at least a first portion of a first encrypted document encrypted using a first encryption key, and an encryption key manager to manage a set of encryption keys for the documents on the storage system, the encryption key manager further to discard the first encryption key to provide secure removal of the portion of the encrypted document.

Abstract: A system and method for protecting streams in a mixed infrastructure includes determining processing elements that are to access a data stream in a stream processing environment and determining a security level for each processing element. Keys are generated per stream per processing element in accordance with the security level. The keys are associated with processing elements in an access control list in a location accessible by producing and consuming processing elements. The stream is decrypted for processing using keys released upon authenticating processing elements in accordance with the access control list. At security boundaries, the stream is re-encrypted in accordance with a next processing element.

Abstract: A data processing system and methods for performing cache eviction are disclosed. An exemplary method includes maintaining a metadata set for each cache unit of a cache device at a sub-cache unit granularity, wherein the cache device comprises a plurality of cache units, each cache unit having a plurality of segments, wherein the cache device is accessible by a cache client at a segment granularity. The exemplary method further includes in response to determining that a cache eviction is to be performed, selecting a predetermined number of cache units from the plurality of cache units, determining a score for each of the selected cache units based on the respective metadata set maintained at the sub-cache unit granularity, and evicting one or more of the selected predetermined number of cache units based on their scores. The metadata may include, for example, last access time (LAT) metadata, an access count, and hotness metadata, and metadata may be maintained at a segment or a segment group granularity.

Abstract: A data processing system and methods for performing cache eviction are disclosed. An exemplary method includes maintaining a metadata set for each cache unit of a cache device comprising a plurality of cache units, each cache unit having a plurality of segments. In response to determining that a cache eviction is to be performed, a cache unit is evicted based on its metadata set. The exemplary method includes selecting one or more segments of the evicted cache unit to copy to a second cache unit based on the metadata set of the evicted cache unit, copying the selected one or more segments to the second cache unit, and writing the second cache unit to a storage device. The metadata set may include deletion hints (DH) to indicate valid segments, last access time (LAT) or age based metadata, an access count, or a score for each segment based on the metadata set.

Abstract: In response to a request from a client to store a data block in a storage system, the data block is segmented into a plurality of subblocks. Each of the plurality of subblocks is individually compressed into a compressed subblock. The compressed subblocks are packed into a compressed data block. The compressed data block having the individually compressed subblocks therein is stored in a persistent storage device. Metadata of the compressed data block is stored in an index entry in an index of the storage system, including storing subblock locators indicating locations of the compressed subblocks. Each of the subblocks can be individually accessed based on a corresponding subblock locator without having to access remaining subblocks.

Abstract: An operating state of each of a plurality of storage units of a storage system is periodically monitored, including a storage capacity, a throughput, and overlap of clients associated with the storage units. In response to a request to redistribute data from a first of the storage units to another storage unit, a cost factor for each of remaining storage units to relocate the data of the first storage unit to each of the remaining storage units is determined. A cost factor of each of the remaining storage units is determined based on at least one of the storage capacity, the throughput, or the overlap of clients of the storage unit. A second of the storage units having a lowest cost factor amongst the remaining storage units is selected. At least a portion of the data of the first storage unit is migrated to the second storage unit.

Abstract: A computer-implemented method for compressing a data set, the method comprising receiving a first data block of the data set, selecting automatically by a compression management module a compression module from a plurality of compression modules to apply to the first data block based on projected compression efficacy or resource utilization, and compressing the first data block with the selected compression module to generate a first compressed data block.

Abstract: A computer-implemented method for compressing a data set, the method comprising receiving a first data block of the data set, selecting automatically by a compression management module a compression module from a plurality of compression modules to apply to the first data block based on projected compression efficacy or resource utilization, and compressing the first data block with the selected compression module to generate a first compressed data block.