Abstract: A system for scanning a file system is presented. The system includes a memory storing one or more processor-executable routines; and a processor communicatively coupled to the memory. The processor is configured to execute the one or more processor-executable routines to execute a file system scan using a depth-first concurrent scan method; create one or more checkpoints during the file system scan based on one or more predefined time intervals; and restart a scan from a latest checkpoint of the plurality of checkpoints. A related method is also presented.

Abstract: A data management system manages concurrent readers and writers for large file scans. The data management system may read a plurality of data chucks of the file starting from different offsets and generate a bounded number of read requests, which causes a data chuck identifiable by a data offset to be loaded into a data buffer. The system may queue the loaded data chucks for generating write requests to release the loaded data chunks. One or more write requests are generated responsive to one or more data chunks being associated with a consecutive order of data offsets being successfully loaded to data buffers. The system may write data chucks released from the buffer-rounded reading stage to the data storage in a checkpointed writing stage. The checkpointed writing stage creates a checkpoint based on the data offset of the data chucks that have been completely transferred to the data storage.

Abstract: A system for context-aware application group management in a container deployment environment is presented. The system includes a memory storing one or more processor-executable routines and a processor. The processor is configured to execute the one or more processor-executable routines to receive an instance identifier (ID) corresponding to an application group based on a re-registration request for the application group by the user; identify an instance corresponding to the instance ID from an instance database and a context based on the instance from a context database; generate a re-registration token comprising a context ID corresponding to the context, and initiate a re-registration workflow to create a new instance of the application group based on the re-registration token and the context ID. A related method is also presented.

Abstract: A system for automatically identifying an application group in a container deployment environment is presented. The system includes a reference detection module configured to detect and store one or more reference paths corresponding to each resource type of a plurality of resources in the container deployment environment. The system further includes a resource classification module configured to assign a resource class to each resource type of the plurality of resources. The system moreover includes a resource grouping module configured to group the plurality of resources into one or more resource groups, for each namespace, based on the corresponding resource type, resource class, and one or more reference paths. The system furthermore includes an application group definition module configured to generate an application group definition based on the one or more resource groups. A related method is also presented.

Abstract: A file system and a related method are presented. The file system includes a data storage including a plurality of data blocks; a merge index including a plurality of namespace entries, wherein the plurality of namespace entries include a plurality of blockmap entries and a plurality of local reference entries; a deduplication database including a plurality of deduplication indices and a plurality of global reference entries for a plurality of datasets; and an indexing system configured to generate the plurality of namespace entries and the plurality of global reference entries.

Abstract: A file system and a related method are presented. The file system includes an object storage configured to store file data for one or more files and a plurality of namespace entries corresponding to file data and/or metadata of the one or more files as one or more objects. Each namespace entry of the plurality of namespace entries includes an operation type conducted on the file data and/or metadata captured in a particular snapshot and a version number corresponding to the particular snapshot. The file system further includes an indexing system configured to generate the plurality of namespace entries; store the plurality of namespace entries as one or more objects in the object storage; and identify, in response to a search query, one or more files for retrieval from the object storage based on a list of the plurality of namespace entries sorted on the version numbers.

Abstract: A system for identifying a plurality of resources that define an application in a container deployment environment is presented. The system includes a reference detection module configured to detect and store one or more reference paths corresponding to each resource type. The system includes a resource identification module configured to receive at least one information corresponding to an application definition from a user and identify each resource corresponding to at least one information. The system includes an application definition module configured to (a) scan one or more references at a reference path of each identified resource to identify one or more additional referenced resources; (b) repeat step (a) for the one or more additional referenced resources until all the resources of the plurality of resources that define the application are identified; and (c) generate an application definition based on all the resources identified. A related method is also presented.

Abstract: A system for secure recovery of an application group in a container deployment environment is presented. The system includes a backup controller configured to access an application group token and generate a corresponding backup token. The system further includes a backup module configured to initiate a backup based on the backup token and create a corresponding recovery point on a backup server. The system further includes a recovery access token module configured to access a recovery access token for a determined recovery point. The system further includes a recovery controller configured to generate a recovery token corresponding to the determined recovery point based on the recovery access token, and a recovery module configured to initiate a recovery of the application group from the backup server in a destination cluster based on the recovery token. A related method is also presented.

Abstract: A system for reference-aware application recreation in a container deployment environment is presented. The system includes a reference detection module configured to detect and store one or more reference paths corresponding to each resource type of a plurality of resources in the container deployment environment; a resource ordering module configured to generate a recreation sequence by ordering the plurality of resources based on corresponding references at the reference paths, such that a referenced resource is recreated before the referring resource; and an application recreation module configured to recreate an application based on the recreation sequence. A related method is also presented.

Abstract: A data storage system allows data to be encrypted and de-duplicated at the same system. By way of example, a server of the data storage system may request a client device which intends to upload a data block to transmit a first fingerprint of the data block to the server. The first fingerprint may be derived from the plaintext of the data block. The server may apply a one-way function to the first fingerprint to generate an encryption key and transmit the encryption key to the client device. The client device uses the encryption key to encrypt the data block and generates a second fingerprint which is derived from the ciphertext of the data block. The server uses both the first fingerprint and the second fingerprint to verify the data block and the legitimacy of the client attempting to upload the data block.

Abstract: A data storage system allows data to be encrypted and de-duplicated at the same system. By way of example, a server of the data storage system may request a client device which intends to upload a data block to transmit a first fingerprint of the data block to the server. The first fingerprint may be derived from the plaintext of the data block. The server may apply a one-way function to the first fingerprint to generate an encryption key and transmit the encryption key to the client device. The client device uses the encryption key to encrypt the data block and generates a second fingerprint which is derived from the ciphertext of the data block. The server uses both the first fingerprint and the second fingerprint to verify the data block and the legitimacy of the client attempting to upload the data block.

Abstract: Disclosed embodiments include a method (system and non-transitory computer storage readable medium) for load-balancing a distributed database. The distributed database includes one or more storage machines configured to store a plurality of partitions, where each partition includes key-value pairs. In one embodiment, the distributed database prepares for load-balancing by determining a partition to redistribute (or repartition) and generating smaller partitions of the determined partition. In one aspect, each of the smaller partitions is smaller than the determined partition. The redistribution of the partition can occur, when an amount of requests to access one or more key-value pairs stored in the database increases beyond a predetermined request level or when the size of a partition exceeds a predetermined size. Key-value pairs of the determined partition can be split into different sets of key-value pairs, and each set of key-value pair is copied to a corresponding smaller partition.

Abstract: Disclosed embodiments include a method (system and non-transitory computer storage readable medium) for load-balancing a distributed database. The distributed database includes one or more storage machines configured to store a plurality of partitions, where each partition includes key-value pairs. In one embodiment, the distributed database prepares for load-balancing by determining a partition to redistribute (or repartition) and generating smaller partitions of the determined partition. In one aspect, each of the smaller partitions is smaller than the determined partition. The redistribution of the partition can occur, when an amount of requests to access one or more key-value pairs stored in the database increases beyond a predetermined request level or when the size of a partition exceeds a predetermined size. Key-value pairs of the determined partition can be split into different sets of key-value pairs, and each set of key-value pair is copied to a corresponding smaller partition.