Abstract: Techniques for preventing recovery of specific data elements based on a recovery prevention configuration defined by a user are disclosed. In some embodiments, a computer system performs operations comprising: receiving a recovery prevention configuration from a first computing device of a first user, the recovery prevention configuration comprising at least one recovery prevention parameter specified by the first user via one or more user interface elements displayed on the first computing device, the at least one recovery prevention parameter being configured to identify one or more backup data elements stored in a secondary storage system; storing the recovery prevention configuration in a database in association with the secondary storage system; and filtering out the one or more backup data elements stored in the secondary storage system from a data recovery process based on the at least one recovery prevention parameter of the stored recovery prevention configuration.

Abstract: Examples relate generally to systems and methods for orchestrating a recovery in the event of a ransomware attack on a compute infrastructure. More specifically, some examples include techniques for application migration in cloud data management, ransomware recovery, and mitigation of lost data.

Abstract: Techniques for preventing recovery of specific data elements based on a recovery prevention configuration defined by a user are disclosed. In some embodiments, a computer system performs operations comprising: receiving a recovery prevention configuration from a first computing device of a first user, the recovery prevention configuration comprising at least one recovery prevention parameter specified by the first user via one or more user interface elements displayed on the first computing device, the at least one recovery prevention parameter being configured to identify one or more backup data elements stored in a secondary storage system; storing the recovery prevention configuration in a database in association with the secondary storage system; and filtering out the one or more backup data elements stored in the secondary storage system from a data recovery process based on the at least one recovery prevention parameter of the stored recovery prevention configuration.

Abstract: Techniques for preventing recovery of specific data elements based on a recovery prevention configuration defined by a user are disclosed. In some embodiments, a computer system performs operations comprising: receiving a recovery prevention configuration from a first computing device of a first user, the recovery prevention configuration comprising at least one recovery prevention parameter specified by the first user via one or more user interface elements displayed on the first computing device, the at least one recovery prevention parameter being configured to identify one or more backup data elements stored in a secondary storage system; storing the recovery prevention configuration in a database in association with the secondary storage system; and filtering out the one or more backup data elements stored in the secondary storage system from a data recovery process based on the at least one recovery prevention parameter of the stored recovery prevention configuration.

Abstract: Methods and systems for managing, storing, and serving data within a virtualized environment are described. In some embodiments, a data management system may manage the extraction and storage of virtual machine snapshots, provide near instantaneous restoration of a virtual machine or one or more files located on the virtual machine, and enable secondary workloads to directly use the data management system as a primary storage target to read or modify past versions of data. The data management system may allow a virtual machine snapshot of a virtual machine stored within the system to be directly mounted to enable substantially instantaneous virtual machine recovery of the virtual machine.

Abstract: Described herein are systems and methods that manage machine backups, including the creation of virtual machine packages sufficient to instantiate virtual machines corresponding to the backups. In one aspect, a compute infrastructure includes many machines, which may be either physical or virtual. From time to time, snapshots of the states of these target machines are pulled and saved. Virtual machine packages corresponding to these snapshots are also created. A virtual machine package can be used to instantiate a virtual machine (VM) emulating the target machine with the saved state on a destination virtual machine platform. At some point, the initial VM package for a target machine is created by converting the snapshot to a VM package. However, this may take a long time. Later VM packages can instead be created by updating a prior VM package according to differences between the corresponding snapshots, rather than performing the full conversion process.

Abstract: In one example, a method includes measuring an amount of physical storage space used, or expected to be used, by a portion of a dataset S of segments, and measuring the amount of physical storage space includes receiving information that identifies an ad-hoc group of size ‘n’ of files F1 . . . Fn that makes up a subset of the dataset S, determining a number of unique segments in the dataset S, identifying a respective unique segment set UF1 . . . UFN for each of the ‘n’ files in the ad-hoc group of files, performing a set union operation on the unique segment sets UF1 . . . UFN, and determining a sum of sizes of the unique segment sets UF1 . . . UFN, where the sum is the amount of physical storage space used or expected to be used by the ad-hoc group of size ‘n’ of files F1 . . . Fn.

Abstract: Methods and systems for improving the performance of a distributed job scheduler using job self-scheduling and job stealing are described. The distributed job scheduler may schedule jobs to be run among data storage nodes within a cluster. Each node in the cluster may make a localized decision regarding which jobs should be executed by the node by periodically polling candidate jobs from a table of candidate jobs stored using a distributed metadata store. Upon completion of a job, the job may self-schedule another instance of itself if the next instance of the job should be run before the next polling of candidate jobs by the node that ran the completed job. The node may attempt to steal one or more jobs from a second node within the cluster if a job queue length for a job queue associated with the node falls below a queue length threshold.

Abstract: Described herein are systems and methods that manage machine backups, including the creation of virtual machine packages sufficient to instantiate virtual machines corresponding to the backups. In one aspect, a compute infrastructure includes many machines, which may be either physical or virtual. From time to time, snapshots of the states of these target machines are pulled and saved. Virtual machine packages corresponding to these snapshots are also created. A virtual machine package can be used to instantiate a virtual machine (VM) emulating the target machine with the saved state on a destination virtual machine platform. At some point, the initial VM package for a target machine is created by converting the snapshot to a VM package. However, this may take a long time. Later VM packages can instead be created by updating a prior VM package according to differences between the corresponding snapshots, rather than performing the full conversion process.

Abstract: Methods and systems for allocating disk space and other limited resources (e.g., network bandwidth) for a cluster of data storage nodes using distributed semaphores with atomic updates are described. The distributed semaphores may be built on top of a distributed key-value store and used to reserve disk space, global disk streams for writing data to disks, and per node network bandwidth settings. A distributed semaphore comprising two or more semaphores that are accessed with different keys may be used to reduce contention and allow a globally accessible semaphore to scale as the number of data storage nodes within the cluster increases over time. In some cases, the number of semaphores within the distributed semaphore may be dynamically adjusted over time and may be set based on the total amount of disk space within the cluster and/or the number of contention fails that have occurred to the distributed semaphore.

Abstract: Methods and systems for backing up and restoring different point in time versions of a virtual machine, an application, a database, or an electronic file using independently managed snapshot chains are described. In some cases, different point in time versions of a virtual machine may be captured and stored using one or more snapshot chains. Each snapshot chain may correspond with a base image (e.g., a full image snapshot) and one or more incremental files (e.g., two forward incremental files) that derive from the base image. A snapshot chain may be split into a plurality of shards or sub-chains such that the width of each shard or the maximum data size of the files in each shard is less than a threshold data size (e.g., is less than 1 TB). Data operations (e.g., consolidation and reverse operations) may be performed on the individual shards or sub-chains in parallel.

Abstract: Methods and systems for improving the performance of a distributed job scheduler using job self-scheduling and job stealing are described. The distributed job scheduler may schedule jobs to be run among data storage nodes within a cluster. Each node in the cluster may make a localized decision regarding which jobs should be executed by the node by periodically polling candidate jobs from a table of candidate jobs stored using a distributed metadata store. Upon completion of a job, the job may self-schedule another instance of itself if the next instance of the job should be run before the next polling of candidate jobs by the node that ran the completed job. The node may attempt to steal one or more jobs from a second node within the cluster if a job queue length for a job queue associated with the node falls below a queue length threshold.

Abstract: A method of identifying historical snapshots for a virtual machine (VM) is provided. Some example operations include receiving a request for a historical snapshot of a VM, the request indicating an ID for the VM. A detection is made that the ID for the VM in the request received is a new ID assigned to a VM. A determination is made whether the new ID corresponds to a newly created VM or an existing VM that has been previously registered using a previous ID, wherein the determining includes accessing a property of the VM including a use case identifier associated with an instant recovery request for a specific VM. Based on identifying that the new ID corresponds to a newly created VM, a new VM Group (VMG) object is created for the newly created VM corresponding to the new ID.

Abstract: A method of identifying historical snapshots for a virtual machine (VM) is provided. Some example operations include receiving a request for a historical snapshot of a VM, the request indicating an ID for the VM. A detection is made that the ID for the VM in the request received is a new ID assigned to a VM. A determination is made whether the new ID corresponds to a newly created VM or an existing VM that has been previously registered using a previous ID, wherein the determining includes accessing a property of the VM including a use case identifier associated with an instant recovery request for a specific VM. Based on identifying that the new ID corresponds to a newly created VM, a new VM Group (VMG) object is created for the newly created VM corresponding to the new ID.

Abstract: Methods and systems for managing, storing, and serving data within a virtualized environment are described. In some embodiments, a data management system may manage the extraction and storage of virtual machine snapshots, provide near instantaneous restoration of a virtual machine or one or more files located on the virtual machine, and enable secondary workloads to directly use the data management system as a primary storage target to read or modify past versions of data. The data management system may allow a virtual machine snapshot of a virtual machine stored within the system to be directly mounted to enable substantially instantaneous virtual machine recovery of the virtual machine.

Abstract: Methods and systems for managing, storing, and serving data within a virtualized environment are described. In some embodiments, a data management system may manage the extraction and storage of virtual machine snapshots, provide near instantaneous restoration of a virtual machine or one or more files located on the virtual machine, and enable secondary workloads to directly use the data management system as a primary storage target to read or modify past versions of data. The data management system may allow a virtual machine snapshot of a virtual machine stored within the system to be directly mounted to enable substantially instantaneous virtual machine recovery of the virtual machine.

Abstract: A method of identifying historical snapshots for a virtual machine (VM) is provided. Some example operations include receiving a request for a historical snapshot of a VM, the request indicating an ID for the VM. A detection is made that the ID for the VM in the request received is a new ID assigned to a VM. A determination is made whether the new ID corresponds to a newly created VM or an existing VM that has been previously registered using a previous ID, wherein the determining includes accessing a property of the VM including a use case identifier associated with an instant recovery request for a specific VM. Based on identifying that the new ID corresponds to a newly created VM, a new VM Group (VMG) object is created for the newly created VM corresponding to the new ID.

Abstract: A method of identifying historical snapshots for a virtual machine (VM) is provided. Some example operations include receiving a request for a historical snapshot of a VM, the request indicating an ID for the VM. A detection is made that the ID for the VM in the request received is a new ID assigned to a VM. A determination is made whether the new ID corresponds to a newly created VM or an existing VM that has been previously registered using a previous ID, wherein the determining includes accessing a property of the VM including a use case identifier associated with an instant recovery request for a specific VM. Based on identifying that the new ID corresponds to a newly created VM, a new VM Group (VMG) object is created for the newly created VM corresponding to the new ID.

Abstract: A method of identifying historical snapshots for a virtual machine (VM) is provided. Some example operations include receiving a request for a historical snapshot of a VM, the request indicating an ID for the VM. A detection is made that the ID for the VM in the request received is a new ID assigned to a VM. A determination is made whether the new ID corresponds to a newly created VM or an existing VM that has been previously registered using a previous ID, wherein the determining includes accessing a property of the VM including a use case identifier associated with an instant recovery request for a specific VM. Based on identifying that the new ID corresponds to a newly created VM, a new VM Group (VMG) object is created for the newly created VM corresponding to the new ID.

Abstract: Methods and systems for allocating disk space and other limited resources (e.g., network bandwidth) for a cluster of data storage nodes using distributed semaphores with atomic updates are described. The distributed semaphores may be built on top of a distributed key-value store and used to reserve disk space, global disk streams for writing data to disks, and per node network bandwidth settings. A distributed semaphore comprising two or more semaphores that are accessed with different keys may be used to reduce contention and allow a globally accessible semaphore to scale as the number of data storage nodes within the cluster increases over time. In some cases, the number of semaphores within the distributed semaphore may be dynamically adjusted over time and may be set based on the total amount of disk space within the cluster and/or the number of contention fails that have occurred to the distributed semaphore.