Abstract: Disclosed herein are systems and method for de-duplicating blocks of data. In one aspect, an exemplary method comprises for each previously de-duplicated block of data of a de-duplication engine, storing de-duplicated pages references by hashes and a block descriptor, creating a set of hash components of the previously de-duplicated blocks, and for each newly received block of data for de-duplication, calculating a translation tolerant hash vector including a predetermined number of hash components, determining a similarity of the received block to the previously de-duplicated blocks based on a comparison of the hash components of the received block with the hash components in the set, and when the received block is determined as being similar to the previously processed blocks based on the comparison, storing the block without duplication in the de-duplication engine, including pages of the block referenced by hashes and a block descriptor.

Abstract: Disclosed herein are systems and method for building content for a de-duplication engine. In one aspect, an exemplary method comprises periodically receiving instructions for cycling through a coalescing phase and a sedimentation phase of the de-duplication engine, during a first coalescing phase, selecting a set of seed blocks that are similar to each other, when an instruction for proceeding to a next sedimentation phase is received, entering the sedimentation phase during which newly received blocks are processed to be stored near similar seed blocks, and when an instruction to proceed to a next coalescing phase is received, entering the coalescing phase to update the set of seed blocks.

Abstract: Methods of optimizing transmission of data from a client to a remote data center are disclosed, as well as systems and computer program products related to the same. An exemplary method comprises: receiving data, at a first intervening data center, transmitted from a client and addressed to a terminal data center, wherein the first intervening data center and the terminal data center are selected from a plurality of connected data centers; and transferring the data, from the first intervening data center through one or more additional intervening data centers selected from the plurality of connected data centers, until the data reaches the terminal data center; wherein each data center in the plurality of data centers independently selects a connected data center to transfer the data to based upon the amount of latency and/or bandwidth available at the connected data center.

Abstract: Disclosed are systems and methods for proactive disaster recovery. The described technique monitors for events raised by a system of interconnected external sensors and other devices for obtaining data on the external environment of servers. The system uses these events as a chain of triggers according to which preventative or preparatory actions for disaster recovery are performed.

Abstract: Method for writing objects into an object storage. Performing, on a protocol end point: receiving a client request for inserting an object into the object storage, wherein the object has a name and object data; generating a unique ID (UID) for the object; sending, to a name server (NS), a request for creating a guard entry (GE). The GE has a lifetime that defines when the name-object pair is inserted into the object storage. A request to an object server (OS) atomically creates a Garbage Collection Entry and assigns space for the object data. The GCE has a lifetime that defines when the object data is inserted into the object storage; sending, to the OS, object data for writing to storage; sending, to the NS, a request for writing the name; and sending response to the client, to report success after requests to the NS and the OS are successful.

Abstract: Disclosed are methods and systems for performing data backup which implement data binning using log-structured merge (LSM) trees during deduplication. An exemplary method includes: calculating a reduced hash value (RHV) associated with each of a plurality of data blocks; partitioning the plurality of reduced hash values into groups; selecting a representative hash value for each group; determining whether the representative hash value occurs in a first LSM tree, the first LSM tree stored in a volatile memory; and when the representative hash value occurs in the first LSM tree: loading the RHVs in the representative hash value's group into volatile memory; comparing each of the RHVs to one or more hash values in a second LSM tree to identify a matching hash value; and writing a segment identifier (ID) corresponding to the matching hash value in an archive, which references a data block in a segment store.

Abstract: Disclosed are systems and methods for proactive disaster recovery. The described technique monitors for events raised by a system of interconnected external sensors and other devices for obtaining data on the external environment of servers. The system uses these events as a chain of triggers according to which preventative or preparatory actions for disaster recovery are performed.

Abstract: Disclosed are systems, methods and computer program products for data deduplication during a backup using at least two LSM trees. An example method includes calculating, for a first data block, a first hash value associated with the first data block and determining a reduced hash value based on the first hash value. The method includes determining whether the first data block contains data duplicative of an existing data block in a prior backup based on whether the reduced hash value occurs in a first log-structured merge (LSM) tree. If so, the method includes comparing the first hash value to one or more hash values in a second LSM tree to identify a matching hash value, and writing a first segment identifier (ID) corresponding to the matching hash value in an archive, the first segment ID referencing the existing data block in a segment store.

Abstract: Methods and systems are provided for modifying configuration of a storage system using artificial intelligence. An exemplary method comprises storing an initial configuration of the storage system as configuration parameters, collecting health information and parameter information related to the storage system over a period of time, analyzing the collected health information using machine learning by comparing the health and the parameter information to a set of goals of the storage system, and in response to determining that the storage system is not operating in accordance with the set of goals, identifying a problem with the storage system using artificial intelligence by analyzing the health information and generating parameter changes that correct the problem, updating the configuration parameters with the parameter changes and applying the parameter changes to the storage system to correct the problem and restore performance of the storage system.

Abstract: A method for optimization of a standard TCP connection. The optimization is based on load distributions by a system kernel. A proposed architecture includes an application running a proxy-process. The application is connected locally to an assigned port (e.g., port 9000). A number of connections N (for example 4) is established to a proxy server located on the cloud storage. Thus, the application is not directly connected to a front end server. Instead, it is connected to the proxy process via a local port. The proxy process transfers redistributed data streams to the proxy server on the cloud, which combines N data streams into one and passes it to the front end server residing on the cloud.

Abstract: A system and method is provided for data storage using log-structured merge (LSM) trees. An example method includes storing data blocks for a number of files backup data files in a data archive, storing data values and corresponding segment identifiers for each data block in a first LSM tree, and storing the segment identifiers and associated physical addresses for each data block in a second LSM tree. The method further includes determining that one or more data blocks is no longer referenced by the backup data file and storing, in a third LSM tree, the physical address of the unused space in the data archive. Finally, the method includes copying data blocks from the end of the data archive to unused memory space at the physical address in the data archive and then deleting these data block at the end position of the archive to truncate the archive.

Abstract: A system and method is disclosed for storing a block of data in a distributed data-storage system. An example method includes identifying a list of a plurality of disks in the distributed data-storage system, randomly selecting a disk from the list of the plurality of disks and adding the selected disk to a subset of disks allocated for a copyset, and continuously performing the step of randomly selecting the disk and adding the disk to the subset of disks until the copyset contains a predetermined amount of allocated disks from the distributed data-storage system. Furthermore, this process is repeated to perform a plurality of copysets. Once the copysets are defined, the method further includes dividing the block of data into a number of data chunks equal to the predetermined amount of allocated disks, and, distributing the data chunks onto disks of one of the plurality of the copysets.

Abstract: A system and method is provided for partially validating data archive for an incremental data backup scheme. According to one aspect, the method includes generating the backup data file of electronic data stored electronic memory by transferring the electronic data to a data storage, generating initial incremental backup data of the electronic data where the incremental backup data includes changes made to the electronic data after the backup data file is generated, and comparing a first portion of the backup data file with a corresponding first portion of the electronic data during the generating of the incremental backup data. Furthermore, the method includes determining whether the first portion of the backup data file is valid based on the comparison and, if the first portion of the backup data file is invalid, generating another backup of the first portion of the electronic data.

Abstract: Disclosed are methods and systems for performing data backup which implement data binning using log-structured merge (LSM) trees during deduplication. An exemplary method includes: calculating a reduced hash value (RHV) associated with each of a plurality of data blocks; partitioning the plurality of reduced hash values into groups; selecting a representative hash value for each group; determining whether the representative hash value occurs in a first LSM tree, the first LSM tree stored in a volatile memory; and when the representative hash value occurs in the first LSM tree: loading the RHVs in the representative hash value's group into volatile memory; comparing each of the RHVs to one or more hash values in a second LSM tree to identify a matching hash value; and writing a segment identifier (ID) corresponding to the matching hash value in an archive, which references a data block in a segment store.

Abstract: A system and method is provided for data storage using log-structured merge (LSM) trees. An example method includes storing data blocks for a number of files backup data files in a data archive, storing data values and corresponding segment identifiers for each data block in a first LSM tree, and storing the segment identifiers and associated physical addresses for each data block in a second LSM tree. The method further includes determining that one or more data blocks is no longer referenced by the backup data file and storing, in a third LSM tree, the physical address of the unused space in the data archive. Finally, the method includes copying data blocks from the end of the data archive to unused memory space at the physical address in the data archive and then deleting these data block at the end position of the archive to truncate the archive.

Abstract: Methods of optimizing transmission of data from a client to a remote data center are disclosed, as well as systems and computer program products related to the same. An exemplary method comprises: receiving data, at a first intervening data center, transmitted from a client and addressed to a terminal data center, wherein the first intervening data center and the terminal data center are selected from a plurality of connected data centers; and transferring the data, from the first intervening data center through one or more additional intervening data centers selected from the plurality of connected data centers, until the data reaches the terminal data center; wherein each data center in the plurality of data centers independently selects a connected data center to transfer the data to based upon the amount of latency and/or bandwidth available at the connected data center.

Abstract: Disclosed are systems, methods and computer program products for data deduplication during a backup using at least two LSM trees. An example method includes calculating, for a first data block, a first hash value associated with the first data block and determining a reduced hash value based on the first hash value. The method includes determining whether the first data block contains data duplicative of an existing data block in a prior backup based on whether the reduced hash value occurs in a first log-structured merge (LSM) tree. If so, the method includes comparing the first hash value to one or more hash values in a second LSM tree to identify a matching hash value, and writing a first segment identifier (ID) corresponding to the matching hash value in an archive, the first segment ID referencing the existing data block in a segment store.

Abstract: A system and method is provided for partially validating data archive for an incremental data backup scheme. According to one aspect, the method includes generating the backup data file of electronic data stored electronic memory by transferring the electronic data to a data storage, generating initial incremental backup data of the electronic data where the incremental backup data includes changes made to the electronic data after the backup data file is generated, and comparing a first portion of the backup data file with a corresponding first portion of the electronic data during the generating of the incremental backup data. Furthermore, the method includes determining whether the first portion of the backup data file is valid based on the comparison and, if the first portion of the backup data file is invalid, generating another backup of the first portion of the electronic data.

Abstract: A system and method is disclosed for storing a block of data in a distributed data-storage system. An example method includes identifying a list of a plurality of disks in the distributed data-storage system, randomly selecting a disk from the list of the plurality of disks and adding the selected disk to a subset of disks allocated for a copyset, and continuously performing the step of randomly selecting the disk and adding the disk to the subset of disks until the copyset contains a predetermined amount of allocated disks from the distributed data-storage system. Furthermore, this process is repeated to perform a plurality of copysets. Once the copysets are defined, the method further includes dividing the block of data into a number of data chunks equal to the predetermined amount of allocated disks, and, distributing the data chunks onto disks of one of the plurality of the copysets.

Abstract: Method for generation of a live update including compiling original source code into a first intermediate representation (IR) code; compiling modified source code into second IR code; analyzing and comparing the first and second IR codes to identify variables and functions that were changed generating a part of final IR code with all the original variables and functions; generating an additional part of final IR code with new code for modified portions of the changed original functions, added functions and variables, and marking it for compilation into special code/data sections; and compiling a new object code and a final executable binary based on the final IR. The final executable object code includes the original code and data from original application binary, and a live update code and data from additional part of final IR generated. The live update code and data refer to original code and data where needed via standard object code relocation information.