Abstract: Aspects of data management are described. Information for updating a cluster of storage nodes to a second version may be obtained. Based on obtaining the information, a cluster-level precheck procedure may be performed prior to initiating an update procedure for the cluster of storage nodes. Also, during the update procedure, a node-level precheck procedure may be performed prior to initiating an update of a storage node to determine whether the storage node is in a state for successful update. Based on a result of the node-level precheck, a determination of whether to delay the update of the storage node may be made.

Abstract: Systems and methods to throttle a universal backup host are described. The system receives responses, over a network, corresponding to one or more requests by utilizing one or more threads from a thread pool. The threads include a first thread. The requests include a first request. The receiving the responses includes receiving a first response corresponding to the first request by utilizing the first thread, the first response includes a first metadata item. The system processes the responses by generating a first latency based on the first response and incrementing a number for registering requests based on the first response. Next, the system aggregates samples. The samples include the first latency and the number of requests. Next, the system resizes the thread pool based on the aggregating and backs up the file set from the source host based on the metadata.

Abstract: A method for migrating a computing resource across cloud environments is described. According to the method, a data management system may interface with a first cloud environment and a second cloud environment. The data management system may receive a request to migrate a first computing resource stored in the first cloud environment to the second cloud environment. The data management system may generate, based on the request, a first compute job in the first cloud environment to cause the first cloud environment to extract data from a backup of the first computing resource and transfer the data to the second cloud environment. The data management system may instruct the second cloud environment to generate a second computing resource. The data management system may generate a second compute job in the second cloud environment to cause the second cloud environment to load the extracted data into the second computing resource.

Abstract: In some examples, a data management system processes snapshots of a distributed file system, the distributed file system having files, each file comprising multiple data chunks. The data management system performs operations including storing file-to-chunk mapping in file system metadata; creating, for each chunk, a chunk generation ID by associating each chunk with a file system generation ID; in a next-generation snapshot of the distributed file system, incrementing, for all chunks in the next-generation snapshot, the respective chunk generation IDs; and taking a snapshot of the file system metadata and storing an updated file-to-chunk mapping associated with the next-generation snapshot.

Abstract: Embodiments disclosed herein provide systems, methods, and computer readable media for infinite versioning by automatic coalescing. In a particular embodiment, a method provides determining an age range for a plurality of data versions stored in a secondary data repository and identifying first data versions of the plurality of data versions that are within the age range. The method further provides determining a compaction ratio for the first data versions and compacting the first data versions based on the compaction ratio.

Abstract: An application server may receive an input indicating a recovery priority for recovering data from a data backup environment to a data source environment and may receive data usage statistics indicating data access metrics and user access metrics corresponding to the data in the data source environment. The application server may generate, from the recovery priority and the data usage statistics, one or more data priority classifications for the data and may build a data model indicating an order for recovery of the data based on the one or more data priority classifications. The application server may then cause display of an indication of a progress of recovering the data from the data backup environment to the data source environment.

Abstract: Methods, systems, and computer programs are presented for providing backup services to a database. One method includes operations for installing a backup agent in a first database, receiving information about the first database, and executing, by the backup agent, queries to the first database to determine a topology of the first database. Further, the method includes configuring, based on the topology, a receiver service of a second database for backing up the first database in the second database. The backup agent configures an interface module of the first database to back up the first database to the second database. The configuration includes an interface to the receiver service of the second database and connection information for storing data in one or more nodes of the second database. The interface module streams updates from the first database to the second database based on the configuration of the interface module.

Abstract: A data management system may support techniques for immutable storage of snapshot data entities, which may each include data corresponding to one or more snapshots, in a cloud environment. The data management system may determine respective retention periods and respective immutability periods for the snapshot data entities. The data management system may extend the respective immutability period for a first snapshot data entity based on the respective retention period for the first snapshot data entity being greater than or equal to a threshold duration. Additionally or alternatively, the data management system may maintain (refrain from extending) the respective immutability period for a second snapshot data entity based at least in part on the respective retention period for the second snapshot data entity being less than the threshold duration.

Abstract: Systems, methods, and machine-storage mediums for optimizing snapshot image processing are described. The system receives a first read request to read data from optimized snapshot information including snapshot information and cached snapshot information. The first read request includes a first offset identifying a first storage location and a first length. The snapshot information includes a full snapshot and at least one incremental snapshot. The system identifies a first portion of the data is stored in the snapshot information responsive to identifying the first portion of the data is not stored in the cache snapshot information. The system identifies a second portion of data is stored in the optimized snapshot information, reads the first portion of data and the second portion of data from the optimized snapshot information, and communicates the data, including the first and second portions of the data, to the job.

Abstract: A data management system comprises: a storage appliance configured to store a snapshot of a virtual machine; and one or more processors in communication with the storage appliance. The one or more processors are configured to perform operations including: identifying a plurality of shards of the virtual machine; requesting a snapshot of each of the plurality of shards; receiving the shards asynchronously; ordering the received snapshot shards sequentially into a results queue; and storing a single snapshot of the virtual machine based on the ordered snapshot shards. Operations may further include maintaining a flow control queue that limits the number of snapshot shards requested.

Abstract: A file system in a user space partition of virtual memory may be mounted by a computing device that runs a virtual machine which includes a set of storage disks. The file system in user space may then expose one or more virtual files associated with one or more storage disks that correspond to one or more loop devices configured to map files of the virtual machine to the one or more virtual files. The computing device may then receive a request to read a data block stored at the virtual machine and may identify a file and corresponding virtual file that stores the requested data block based on a set of metadata provided by the loop devices. The computing device may then determine the location of the data block stored at the virtual machine, and may read the data block from the determined location.

Abstract: During a backup a procedure, a first storage entity may backup a set of database logs from a second storage entity and metadata associated with the set of database logs may be stored in a log deletion table. In accordance with a user configuration of the second storage entity and the backup procedure, a first command to delete one or more database logs of the set from the second storage entity may be transmitted to the second storage entity. The log deletion table may be updated based on a query of second storage entity about whether the one or more database logs were deleted from the second storage entity in response to the first command. Based on the updated log deletion table, a second command may be transmitted to the second storage entity to delete database logs that were not deleted in response to the first command.

Abstract: Methods and systems for managing, storing, and serving data within a virtualized environment are described. In some embodiments, a data management system may manage the extraction and storage of virtual machine snapshots, provide near instantaneous restoration of a virtual machine or one or more files located on the virtual machine, and enable secondary workloads to directly use the data management system as a primary storage target to read or modify past versions of data. The data management system may allow a virtual machine snapshot of a virtual machine stored within the system to be directly mounted to enable substantially instantaneous virtual machine recovery of the virtual machine.

Abstract: Techniques unmasking ransomware attacks are disclosed. In some embodiments, a computer system performs operations comprising: generating a first prediction that a file system comprising a plurality of files has been attacked by ransomware based on snapshot metadata of the file system using a snapshot-level machine learning prediction model, the snapshot metadata comprising a plurality of file change data indicating a plurality of file change events that have been performed on the file system; in response to the first prediction, generating a classification for each one of the files based on the file change data using a file-level machine learning prediction model, the classification indicating whether the files have been targeted by the ransomware for encryption; determining that one or more files have been targeted by the ransomware based on the classification; and displaying the classification for the one or more files on a computing device of a user.

Abstract: The technology disclosed herein enables the restoration of a database version across cloud environments. In a particular embodiment, a method provides receiving, in a second cloud environment from a first cloud environment, first metadata describing a first data version stored in the first cloud environment. The first data version includes first data items and the first metadata. After receiving the first metadata, the method provides receiving, in the second cloud environment, an instruction to restore the first data items to the second cloud environment. In response to the instruction, the method provides restoring the first data items to the second cloud environment using the first metadata.

Abstract: Techniques for data management are described. A monotonically increasing time source that indicates an elapsed time since a reference event may be activated. Multiple snapshots of a computing object may be generated in accordance with a schedule for backing up the computing object, where the schedule may include a retention duration for retaining snapshots. Based on generating the snapshots, timestamps for the snapshots may be stored, where the timestamps may indicate respective values of the monotonically increasing time source. As part of an expiration job, a reference value of the monotonically increasing time source may be identified based on the retention duration and a current value indicated by the monotonically increasing time source. Also, a snapshot of the snapshots may be expired based on a timestamp of the snapshot corresponding to a value of the monotonically increasing time source that is less than the reference value.

Abstract: A data storage infrastructure may establish a partition that includes a first data center and a second data center that is geographically separated from the first data center. The data storage infrastructure may replicate a full snapshot and one or more incremental snapshots of a virtual machine from a first data management platform to a second data management platform, where the virtual machine is migrated from a first host of the first host group to a second host of the second host group upon a failover event occurring at the first data center. The data storage infrastructure may then capture an incremental snapshot of the virtual machine based on linking a first instance of the virtual machine that was replicated from the first data management platform and a second instance of the virtual machine that is managed by the second data management platform.

Abstract: A method for recovering files from a filesystem stored across sparse files in a cloud environment is described. According to the method, a data management system may receive a request to read the files. The data management system may identify one or more target address ranges corresponding to the files indicated via the request. The data management system may read index information for the sparse files in the cloud environment. The index information may indicate respective address ranges for data blocks within the sparse files. The data management system may identify one or more data blocks within one or more sparse files as corresponding to address ranges that overlap with the one or more target address ranges based on the index information. The data management system may transmit, to the cloud environment, one or more read requests for the identified one or more data blocks.

Abstract: In some examples, a data management and storage (DMS) system comprises peer DMS nodes in a node cluster, a distributed data store comprising local and cloud storage, and an IO request scheduler comprising at least one processor configured to perform operations in a method of scheduling IO requests. Example operations comprise implementing a kernel scheduler to schedule a flow of IO requests in the DMS system, and providing an adjustment layer to adjust the kernel scheduler based on an IO request prioritization. A flow of IO requests is identified and some examples implement an IO request prioritization based on the adjustments made by the adjustment layer.

Abstract: Methods and systems for improving data back-up, recovery, and search across different cloud-based applications, services, and platforms are described. A data management and storage system may direct compute and storage resources within a customer's cloud-based data storage account to back-up and restore data while the customer retains full control of their data. The data management and storage system may direct the compute and storage resources within the customer's cloud-based data storage account to generate and store secondary layers that are used for generating search indexes, to generate and store shared space layers and user specific layers to facilitate the deduplication of email attachments and text blocks, to perform a controlled restoration of email snapshots such that sensitive information (e.g., restricted keywords) located within stored snapshots remains protected, and to detect and preserve emails that were received or transmitted and then deleted between two consecutive snapshots.