Abstract: Provided are a computer program product, system, and method for managing reads and writes to data entities experiencing a security breach from a suspicious process. A suspicious process is detected that is determined to have malicious code. A breach time for a data entity is indicated in response to detecting the suspicious process performing an operation with respect to the data entity. A determination is made whether there is a copy of the data entity created after the breach time for the data entity. A copy of the data entity is created when there is no copy of the data entity created after the breach time.

Abstract: Aspects of the present disclosure relate to systems and methods for deploying payloads in a cloud service. In one aspect, one or more payloads may be deployed to a plurality of sample servers. Each of the one or more payloads may include a plurality of files. A hash value may be generated for each file of the plurality of files. A master hash value may be generated for each payload from the generated hashes for each file of the plurality of files. It may be determined whether the one or more payloads have changed since a previous deployment of the one or more payloads. When it is determined that at least one payload of the one or more payloads has changed, the at least one changed payload may be deployed to a plurality of data servers.

Abstract: A deduplication service can be provided to a storage domain from a services framework that expands and contracts to both meet service demand and to conform to resource management of a compute domain. The deduplication service maintains a fingerprint database and reference count data in compute domain resources, but persists these into the storage domain for use in the case of a failure or interruption of the deduplication service in the compute domain. The deduplication service responds to service requests from the storage domain with indications of paths in a user namespace and whether or not a piece of data had a fingerprint match in the fingerprint database. The indication of a match guides the storage domain to either store the piece of data into the storage backend or to reference another piece of data. The deduplication service uses the fingerprints to define paths for corresponding pieces of data.

Abstract: Embodiments for optimizing sequential write operations in a data deduplication environment by one or more processors. Similar data deduplication signatures for stored data at all storage devices in one or more storage systems may be maintained. A deduplication signature associated with a write operation at a storage device may be compared to the similar data deduplication signatures. Results of the comparison with each of the other storage devices may be shared prior to performing the write operation.

Abstract: Managing an update of software modules of a layered filesystem installed for running a software application on a data processor. The management of an updated module includes: (i) contributing a back layer from a software module that likely has less stability than the updated module; and/or (ii) rebuilding a layer using a software module that likely has more stability than the updated layer.

Abstract: For similarity based deduplication of remote data repositories, a parse module generates a rolling hash value based on a portion of an incoming stream of backup data. A comparison module compares the rolling hash value with entries stored in a rolling hash index, and in response to matching the rolling hash value with an entry in the rolling hash index, generates a strong hash value and determines if a match of the strong hash value exists in a first strong hash index. The comparison module, in response to a determination that the match does not exist in the first strong hash index, compares the strong hash value with entries in a second strong hash index in the remote data repository. A migration module, in response to a determination that the strong hash value does not match any hash entries, stores the portion of backup data as new data.

Abstract: A naming service for a hierarchical namespace. The naming service includes a database system with a first table having first records. Each of the first records is for a hierarchical node and includes a node identifier and an ACL identifier to a set of access control rules applicable to the node. The database system includes a second table including second records, each record in the second records has one of the ACL identifiers from the first table correlated with a definition access control rules in the set of access control rules. The naming service includes an access control rule identifier generator configured to detect a set of access control rules for each node in the namespace and generate an a ACL identifier for each set of access control rules that can be stored in records in the first table and stored in records in the second table.

Abstract: A method for storing data in a cloud includes providing at least one data file to be stored together with a predefined number t of replicas of the at least one data file within the cloud, at least one authentication tag corresponding to the at least one data file and t functions that are configurable to take at least a predefined time to compute. The at least one data file, the at least one authentication tag and the t functions are transmitted to the cloud. The at least one data file is stored within the cloud and t solutions of the t functions are computed within the cloud. The t replicas of the at least one data file are generated based on the t solutions of the t functions and the at least one data file within the cloud. The t replicas are stored within the cloud.

Abstract: For similarity based deduplication of remote data repositories, a parse module generates a rolling hash value based on a portion of an incoming stream of backup data. A comparison module compares the rolling hash value with entries stored in a rolling hash index, and in response to matching the rolling hash value with an entry in the rolling hash index, generates a strong hash value and determines if a match of the strong hash value exists in a first strong hash index. The comparison module, in response to a determination that the match does not exist in the first strong hash index, compares the strong hash value with entries in a second strong hash index in the remote data repository. A migration module, in response to a determination that the strong hash value does not match any hash entries, stores the portion of backup data as new data.

Abstract: In connection with a data distribution architecture, client-side “deduplication” techniques may be utilized for data transfers occurring among various file system nodes. In some examples, these deduplication techniques involve fingerprinting file system elements that are being shared and transferred, and dividing each file into separate units referred to as “blocks” or “chunks.” These separate units may be used for independently rebuilding a file from local and remote collections, storage locations, or sources. The deduplication techniques may be applied to data transfers to prevent unnecessary data transfers, and to reduce the amount of bandwidth, processing power, and memory used to synchronize and transfer data among the file system nodes. The described deduplication concepts may also be applied for purposes of efficient file replication, data transfers, and file system events occurring within and among networks and file system nodes.

Abstract: Generating a data object identifier by dividing the data in the data object into a plurality of chunks; processing each chunk using a clustering algorithm to generate, for each chunk, a pair of values characterizing the data in the chunk, thereby giving rise to a plurality of pairs of values (PoV); generating a plurality of nodes in a two dimensional space each corresponding to a respective PoV, wherein, for any given PoV, the values in the given PoV are indicative of location coordinates of the corresponding node in the two dimensional space; generating a plurality of features related to the plurality of nodes, each feature characterizing a spatial relationship between three or more nodes; and generating the data object identifier by arranging the features in a feature vector in accordance with predetermined rules.

Abstract: A system comprises a processor that that receives, through a network, a first data structure from a first source system. Further, the processor receives, through the network, a second data structure from a second source system. In addition, the processor normalizes the first data structure and the second data structure according to a common interface such that at least one constituent of the common interface is immutable, the common interface is backward compatible, and the common interface is forward compatible. The first data structure is at least partially distinct from the second data structure.

Abstract: In one embodiment, a method for ransomware-aware file backup is implemented on a computing device and includes: backing up a target population of files from a target file location in a backup transaction, computing a backup delta score for the backup transaction, where the computing comprises comparing backup data from the backup transaction with backup data from a previous backup transaction, determining whether the computer backup delta score exceeds a pre-defined threshold, and upon the computed backup delta score exceeding the pre-defined threshold: determining that the backup transaction is indicative of a ransomware infection, and performing at least one counter-measure in response to the ransomware infection.

Abstract: A facility for addressing nodes in a graph database is described. The facility receives from a first application a first indication that it treats a first property as a unique identifier, and receives from a second application distinct from the first application a second indication that it treats a second property distinct from the first property as a unique identifier. The facility receives from the first application a first update request for a distinguished entity, the distinguished entity identified in the first update request by its value for the first property. The facility performs the first update request with respect to the distinguished entity. The facility receives from the second application a second update request for the distinguished entity, the distinguished entity identified in the second update request by its value for the second property. The facility performs the second update request with respect to the distinguished entity.

Abstract: The disclosure relates data protection management (e.g. data recovery) for distributed storage systems. Specifically, the systems (and methods) of the disclosure provide an advanced mechanism for data recovery based on a notion of a data fragment's peer group, which may be used for “peer” recovery. Peer recovery allows a data fragment to be recovered when all the data fragments from its peer group are available. Accordingly, the described mechanism leverages the power of erasure coding and XOR operations to support recovery of data in situations where such data would previously be considered unrecoverable.

Abstract: Embodiments include methods, systems and computer program products for performing a tournament tree sort on a hardware accelerator having an external memory. The method includes receiving a plurality of key values by the hardware accelerator, assigning each of the plurality of key values a sequential key number as the plurality of key values are received and performing pairwise comparisons of each of the plurality of key values to identify a winning key and a losing key. The method also includes storing the losing key of each pairwise comparison in a first section of the external memory, wherein a location in the first section is based on the key number of the losing key and storing the winning key of each pairwise comparison in a second section of the external memory, wherein a location in the second section is based on the key number of the winning key.

Abstract: A transaction request is received at a log-based transaction manager, indicating a logical constraint to be satisfied before the corresponding transaction is committed. The transaction manager identifies a subset of transaction records stored in a persistent change log that are to be examined to evaluate the logical constraint. Based at least in part on the result of a comparison of one or more constraint-related data signatures included in the transaction request with corresponding data signatures in the subset of transaction records, a decision is made to commit the requested transaction.

Abstract: An application executing on a user device can receive a request to access a remote computer system. The application can automatically obtain an authentication code that is generated based at least in part on a seed value, which can be stored in the user device. The application can automatically generate an authentication request based at least in part on the access information and the authentication code, and transmit the authentication request to remote computer system.

Abstract: For adaptive similarity search resolution in a data deduplication system using a processor device in a computing environment, multiple resolution levels are configured for a similarity search. Input similarity elements are calculated in one resolution level for a chunk of input data. The input similarity elements of the one resolution level are used to find similar data in a repository of data where similarity elements of the stored similar repository data are of the multiple resolution levels.

Abstract: A method and an apparatus for querying data in a database, and the method includes: receiving a query instruction; determining a status of each block index in a block index set prestored in a global memory of a GPU; when each block index in the global memory of the GPU is in an asynchronous state, starting one or more GPU processes to filter the block index in the global memory of the GPU according to a query condition, to obtain a first block index query result; and determining a final data query result according to the first block index query result. In this method, querying is performed by directly using a GPU process, thereby omitting a process of repeatedly copying partitioned data from a memory in the prior art, shortening query time, and improving query efficiency.

Abstract: Various systems and methods precalculate client-side deduplication signatures. For example, one method can identify a data object to be included in a backup and then calculate a signature of a data object prior to the start of a backup operation to create the backup. The signature identifies the data object within a deduplicated data store configured to store a backup of the data object. The signature can be calculated in response to detecting that a current usage level of system resources (e.g., such as a current processor utilization) of a client computing device is less than a threshold usage level.

Abstract: Decision apparatus includes a first memory bank, containing a first table of hash composition factors, and a second memory bank, containing second and third tables of associative entries. A logic pipeline receives a sequence of data items and extracts a search key from each data item. A pre-hash circuit computes a first index by applying a first hash function to the search key. A first lookup circuit reads a hash composition factor from a location in the first memory bank indicated by the first index, and a hash circuit compute second and third indices as different combinations, determined by the hash composition factor, of second and third hash functions applied by the hash circuit to the search key. A second lookup circuit reads the entries in the second and third tables that are indicated respectively by the second and third indices.

Abstract: A method for data reconstruction includes: obtaining a data migration list, where the data migration list indicates a partition with data to be migrated on a storage node with data to be migrated; generating a data migration task according to the data migration list and routing table information, where the routing table information indicates correspondence between a storage node in a cluster and a partition; and concurrently executing data migration tasks of a storage node in the cluster to reconstruct data. In addition, a device for data reconstruction and a storage system are further provided accordingly.

Abstract: A system includes a sender computer and a recipient computer. The sender computer starts the chunking process by splitting a target file into chunks and providing signatures of the chunks of the target file to the recipient computer. After and in response to receiving the signatures of the chunks of the target file, the recipient computer splits the reference file into chunks, generates signatures of the chunks of the reference file, and identifies chunks that are common between the target file and the reference file. The recipient computer combines adjacent common chunks into larger chunks and creates, for example, MD5 signatures accordingly. The sender computer receives from the recipient computer information on chunks common between the target file and the reference file and provides a sequence of edit operations to the recipient computer to allow the recipient computer to reconstruct the target file.

Abstract: Systems and methods for maintaining a datacenter are provided. A repair component includes modules that are configured to scan the data objects based on the identifiers of the data objects, identify potential issues or concerns with the data objects, and then repair the issues as they are discovered or over time. The repair component can be run in a non-centralized and highly parallelized manner.

Abstract: A system, computer-readable storage medium storing at least one program, and a computer-implemented method for identifying a storage group in a distributed storage system into which data is to be stored is presented. A data structure including information relating to storage groups in a distributed storage system is maintained, where a respective entry in the data structure for a respective storage group includes placement metrics for the respective storage group. A request to identify a storage group into which data is to be stored is received from a computer system. The data structure is used to determine an identifier for a storage group whose placement metrics satisfy a selection criterion. The identifier for the storage group whose placement metrics satisfy the selection criterion is returned to the computer system.

Abstract: End-to-end data integrity is provided in parallel computing systems, such as High Performance Computing (HPC) environments. An exemplary method is provided for processing data in a distributed data storage system by obtaining the data and one or more corresponding checksum values from a compute node; and providing the data and the one or more corresponding checksum values to the distributed data storage system for storage. One or more checksum values corresponding to the data can be generated if the one or more checksum values are not received from a compute node. Exemplary processes are provided for copy; slice; merge: and slice and merge functions. The distributed data storage system comprises, for example, one or more Parallel Log-Structured File System (PLFS) storage elements and/or key-value storage elements storing one or more key-value pairs.

Abstract: The present disclosure also provides systems and methods for sharding objects stored in a distributed storage system. In accordance with one embodiment disclosed herein, a key sharding technique is used. Key sharding is an advantageously efficient technique when dealing with an object containing a collection of key-value records. In accordance with an embodiment of the invention, referenced chunks identified by the key shards may each store a subset of the collection of the key-value records, and the key-value records in the subset have key hashes that have a range of matching bits in common. One embodiment disclosed herein provides a method of performing a delta edit of a named object stored in a distributed storage system in which a payload of the named object is stored in key shards. Other embodiments, aspects and features are also disclosed.

Abstract: Data structure that is associated with an hierarchical file system of a computational system. The hierarchical file system includes a set of files arranged in a directory structure including a set of directories. The data structure includes a data store with a set of reference data objects (topics) and a set of coupling data elements. The set of coupling data elements includes at least: one coupling data element for coupling a first item selected from either the set of reference data objects or the set of files or the set of directories with a second item from either the set of reference data objects or the set of files or the set of directories.

Abstract: A method includes receiving, at a computing device comprising a processor, a request to determine bicliques in a graph, where the graph includes a first set of nodes, a second set of nodes, and a set of edges, each edge in the set of edges connecting a node in the first set of nodes to a node in the second set of nodes. The method also includes determining at least one biclique based on querying a bitmap index representing the graph, where the bitmap index includes a plurality of bit strings corresponding to the first set of nodes, and where a value stored in a particular location in each bit string indicates whether an edge connects a first node corresponding to the bit string to a second node corresponding to the particular location.

Abstract: Systems and methods of determining a similarity between data units in a nonvolatile memory are disclosed. One method includes obtaining first and second data units and dividing the first and second data units into a first plurality of non-overlapping chunks of data and a second plurality of non-overlapping chunks of data. The method further includes determining a first plurality of values and a second plurality of values associated with the chunks, and determining a similarity between the first second data units based on the first plurality values and of the second plurality of values. In one example embodiment, a similarity between an incoming data unit and another data unit is determined based on the number of buckets storing an incoming index value and another index value associated with the another data unit. A plurality of buckets in a table is determined based on a selected plurality of hash values.

Abstract: A method for generating a key in a network. The network includes at least one first user and one second user having a secured communication link to one another, and a third user, to which a secured communication link is to be established. The first user and the third user each generate a first partial value sequence from properties of the transmission channel between the first user and the third user. The second user and the third user each generate a second partial value sequence from properties of the transmission channel between the second user and the third user. In a secured part of the network, which includes at least the first and the second user, but not the third user, the key is ascertained from at least the first partial value sequence of the first user and the second partial value sequence of the second user. The key is also generated in the third user from at least the first partial value sequence and the second partial value sequence.

Abstract: Embodiments of the present disclosure are directed to, among other things, validating the integrity of received and/or stored data payloads. In some examples, a storage service may perform a first partitioning of a data object into first partitions based at least in part on a first operation. The storage service may also verify the data object, by utilizing a verification algorithm, to generate a first verification value. In some cases, the storage service may additionally perform a second partitioning of the data object into second partitions based at least in part on a second operation. The second partitions may be different from the first partitions. Additionally, the archival data storage service may verify the data object using the verification algorithm to generate a second verification value. Further, the storage service may determine whether the second verification value equals the first verification value.

Abstract: A method of operating a database node of a distributed database is provided. The distributed database includes a plurality of database nodes each storing a set of data. A latency that occurs when the database node contacts a further database node in order to respond to one or more data requests that relate to a subset of data that is one of stored and intended to stored at the further database node is measured. An average latency value for communicating with each of the database nodes in the distributed database is calculated. Notifications reporting the average latency values to data distribution optimization functions in the distributed database are sent. A latency measure is reported to a further node of the distributed database.

Abstract: A network interface includes a host interface for communicating with a node, and circuitry which is configured to communicate with one or more other nodes over a communication network so as to carry out, jointly with one or more other nodes, a redundant storage operation that includes a redundancy calculation, including performing the redundancy calculation on behalf of the node.

Abstract: A mechanism is provided in a storage system for efficient inline data de-duplication. The mechanism receives a write command and a hash key for a portion of data to be written from an application host to a write address. The write command indicates whether the application host is tolerant or intolerant to data loss. Responsive to the write command indicating the application host is tolerant to data loss, the mechanism performs a hash key lookup in a hash index. The mechanism determines whether the portion of data has previously been written to the storage system. Responsive to determining the portion of data has previously been written to the storage system, the mechanism stores a pointer to the previously written data at the write address.

Abstract: Disclosed herein are a server duplication system and method and a storage medium storing a server duplication computer program. The server duplication system includes a primary server including a hypervisor including a hypervisor-based fault tolerance module and a first file system virtual machine (FS VM), and a first standby server including a hypervisor including a hypervisor-based fault tolerance module that exchanges data with a fault tolerance module provided on the hypervisor of the primary server and duplicates the primary server. The first FS VM provides a first file system that is shared by a user virtual machine (USER VM), and a buffer cache that is used in conjunction with the first file system is provided on virtual memory. The first FS VM is duplicated into the standby server using the hypervisor-based fault tolerance module of the primary server and the hypervisor-based fault tolerance module of the standby server.

Abstract: A technique and mechanism to effect cross-pollination synchronization of PIM (personal information manager) data to multiple sources. Additionally, a novel feature of the subject invention is the ability to effect cross-pollination synchronization with any number of devices and sources in a network. The invention provides for a user to employ a mobile device or set of devices as a central repository for PIM data (e.g., calendar, contacts and tasks) synchronized from multiple sources (e.g., multiple PC's (personal computers), an exchange server). The invention provides for synchronization of PIM data from multiple types of server sources to multiple devices regardless of the synchronization protocol versions and different schemas.

Abstract: An electronic apparatus including: a display unit; a communication unit which communicates with a server to which at least one emulating device is connected locally, through a network; a storage unit which stores therein a predetermined application; and a controller which is connected to the server and transmits the application stored in the storage unit to the emulating device, and receives from the emulating device an execution image of the application processed and executed by the emulating device and displays the execution image on the display unit.

Abstract: Large-scale advanced database models, systems, and methods provide a responsive, scalable data storage solution which is ripe for use in data warehousing and analytics environments. These advanced database models, systems, and methods provide for dramatically increased performance in accessing data as well as increased storage capabilities in the data set.

Abstract: A computer-implemented method for variable-length chunking may include 1) identifying a first data stream subject to deduplication, 2) identifying a predetermined chunk of the first data stream that starts at a first location and ends at a second location within the first data stream, 3) identifying a second data stream with a matching chunk for the predetermined chunk that starts at a third location and ends at a fourth location within the second data stream, 4) identifying a subsequent chunk within the second data stream that starts at the fourth location and ends at a fifth location within the second data stream, 5) calculating a candidate boundary offset within the first data stream based on exceeding the second location by a difference between the fifth location and the fourth location, and 6) performing a boundary test at the candidate boundary offset. Various other methods, systems, and computer-readable media are disclosed.

Abstract: A computing system may include multiple computing devices. A file at a computing device may be accessed based on a reference file stored at the device that comprises a hash value of the content of the file. The hash value may be independent of the file location or file name and include a representation of the content of the file such that a given hash value is unique for given content. For example, the hash value may be generated using a hash function on the content of the file. The hash function may be generated by the hash function that reproducibly identifies the file based on the content of the file. In this manner, the reference file may be used to uniquely identify a file as well as serve as a basis for checking the integrity of files transmitted over a network from one computing device to another computing device.

Abstract: A system and method for space and time efficient bound calculation is disclosed. The method comprises inserting a plurality of key/value pairs into a “Bloom bounder”, each key/value pair comprising a key and a value. For each pair, the inserting includes calculating a plurality of hash values, each calculated by applying a different one of a plurality of hash functions to the key, and selectively updating one or more data arrays based on the plurality of hash values and the value received key/value pair. A bound may then be determined for a given query key by analyzing information in the one or more data arrays to determine a bound value, such that for every received key/value pair with a key matching the query key, the corresponding value is less than or equal to the bound value.

Abstract: A fingerprint is computed from selected metadata of binary files, and used in determining whether to regenerate code-behind file(s). In one case, after ascertaining that a binary-file-set's current metadata-fingerprint differs from a previous metadata-fingerprint, the corresponding code-behind-file-set is regenerated. A code-behind file may also be regenerated, even though its current fingerprint matches its previous fingerprint, because its metadata has changed. If the current fingerprint matches the previous fingerprint and metadata is unchanged, regeneration can be avoided. The fingerprint may consist of, or merely contain, definitions of classes, properties, and other items, and may be limited to metadata that is used when generating the code-behind-file-set. System assemblies are not fingerprinted. Timestamps may be used. Metadata-fingerprint computation code and regeneration determination code are described.

Abstract: Surrogate heuristic identification is described, including receiving data associated with content, evaluating the content to identify one or more data structures associated with the content, processing the one or more data structures to generate a fingerprint, wherein the fingerprint is hashed to provide a reference identifier, and sending the reference identifier to a client file and a service file.

Abstract: A distributed, deduplicated storage system according to certain embodiments is arranged in a parallel configuration including multiple deduplication nodes. Deduplicated data is distributed across the deduplication nodes. The deduplication nodes can be networked together and communicate with one another according using a light-weight, customized communication scheme (e.g., a scheme based on FTP or HTTP). In some cases, deduplication management information including deduplication signatures and/or other metadata is stored separately from the deduplicated data in deduplication management nodes, improving performance and scalability.

Abstract: Systems and methods for labeling text with alphanumeric identifiers are included. A logging string that includes a block of output text may be determined during program code execution. A computing device may generate a first alphanumeric identifier for the logging string using a hashing algorithm. The computing device may remove a portion of the logging string to determine a modified string. The computing device may generate a second alphanumeric identifier for the modified string using the hashing algorithm. The first alphanumeric identifier and the second alphanumeric identifier are presented with the logging string.

Abstract: In an example embodiment, a method, apparatus and computer program product are provided. The method includes determining an upper bound on a probability of error associated with a mapping of a data into binary codes. The mapping is performed based on a plurality of hash functions. The method further includes selecting a set of hash functions from among the plurality of hash functions associated with a minimization of the upper bound on the probability of error.

Abstract: A resilient hashing system leverages a fast, non-cryptographic hash/checksum function that has good diffusion properties while remaining reasonably efficient on modern central processing units (CPUs). The hash function uses random secret data so that hash keys for particular data are difficult to predict. Due to its internal structure, well-chosen random secret data is difficult for an attacker to counter without having access to the direct output of the hash function. At every stage of the block function, there are at least two operations that can be performed in parallel, increasing performance on modern superscalar CPUs. Thus, the resilient hashing system provides a hash table and checksum that can be used in Internet-facing or other vulnerable sources of input data to manage performance in the face of malicious attacks.

Abstract: Systems and methods are described for obtaining, for insertion into a schema-less database, a data object that comprises a plurality of key and value pairs. The method also includes hashing the keys associated with the plurality of key and value pairs. The hashing includes executing a hash function to generate a hashed data object. The method also includes comparing the hashed data object to a plurality of first hashed data objects and determining that the hashed data object does not match any of the first hashed data objects. Based on the determining, the method includes associating the hashed data object with the first hashed data objects to generate second hashed data objects and generating a schema for the database. The schema includes a hierarchy of keys that represent the second hashed data objects.