Abstract: A data processing system maintaining a plurality of masking engines is configured to synchronize masking jobs between different masking engines, in order to ensure consistent outputs between the different engines for the same job. Upon identifying a masking job object maintained by a first masking engine to be exported, dependencies of the job object are traversed to identify all objects depended on by the job object, which are used to form an export payload. At least object of the payload is a reference object that references other objects maintained by the first masking engine. When importing the export payload onto a second masking engine, the export payload is analyzed to identify revision information of the referenced object. The identified revision information is compared with revision information of a corresponding object maintained by the second masking engine, and the referenced objects is imported based upon a result of the comparison.

Abstract: User interactions with a database storage system allow creation of virtual databases based on point-in-time copies associated with a source database. Multiple point-in-time copies are obtained for each source database. A point-in-time copy retrieves data changed in the source database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system and mounting the files on a database server allowing the database server to access the files. User interactions allow the user to specify the source database, a point in time associated with the source database and a destination server to create the virtual database. User input can specify other attributes associated with the virtual database including the file paths, database parameters etc. The user can specify schedules of various actions, including making and retention of point-in-time copies.

Abstract: An application development system allows developers of software system to manage infrastructure resources during the development and testing process. The application development system allows users to define application containers that comprise components including source code, binaries, and virtual databases used for the application. An application container can be associated with policies that control various aspects of the actions taken using the application container including constraints and access control. The application development system enforces the policies for actions taken by users for the application containers. The encapsulation of policies with the application containers allows users of the application containers to take actions including creating virtual databases, provisioning virtual databases, and the like without requiring system administrators to manage resource issues.

Abstract: A storage system performs transformations of data stored as source snapshots to obtain transformed snapshots. The storage system stores relations between source snapshots, for example, parent child relationships. The storage system analyzes relationships between source snapshots to determine relationships between transformed snapshots. The storage system creates a transformed snapshot based on a source snapshot. The storage system traverses the graph representing source snapshots to identify ancestor or descendants of the source snapshots that were previously transformed. The storage system determines the transformed snapshots corresponding to the ancestor and descendant source snapshots and relates them to the newly created transformed snapshot.

Abstract: A database storage system creates secure snapshots or virtual databases based on a source database that stores sensitive information. The database storage system applies a masking function to the sensitive data and stores the masked data in the secure snapshot. The database storage system creates secure snapshots that share masked database blocks with other secure snapshots. The database storage system shares masked database blocks by storing information identifying a previously created masked database block for a new secure snapshot. If a masked database block for the new secure snapshot is different from the previously created masked database block, the database storage system stores metadata including the difference between the two masked database blocks. The database storage system allows sharing any transformed database blocks across snapshot, if the transformed database blocks are obtained by applying a transformation function to data of the source database.

Abstract: A storage system maintains copies of a source database to allow virtual databases to be provisioned. Transaction logs received from the source database are applied to database blocks stored in the storage system to update the database blocks. The storage system checks if there are defect in the transaction logs of the source database. If a defect in transaction logs is detected, the storage system obtains a latest copy of the source database. The storage system matches database blocks of the received copy with database blocks already available in the storage system to determine whether a corresponding database block is already available in the storage system. If a received database block is different from database blocks already available at the storage system, the received database block is stored in the storage system. The database blocks may be matched using a hash value of the database blocks.

Abstract: A storage system executes transform scripts to transform data stored in a source snapshot to obtain transformed snapshot. A transform script may be used to upgrade data stored in a snapshot from a version of an application to a newer version of the application. The storage system stores intermediate snapshots during execution of a transform scripts. The storage system reuses data stored during execution of transform scripts to optimize execution of newer transform scripts. For example, the storage system identifies a previously executed transform script having a matching prefix of transform operations and reuses an intermediate snapshot obtained by executing the matching prefix. The storage system processes the intermediate snapshot by executing the remaining transform operations of the new transform script.

Abstract: Information from source databases is retrieved and stored on a database storage system. Multiple point-in-time copies are obtained for each database. A source database may contain partitions comprising sets of database tables. A partition is imported into a virtual database created. The database storage system validates the partition before importing, by checking whether the partition includes database tables that refer to database tables outside the partition. A partition imported in a virtual database may be rewinded to an older state or refreshed to a more recent state. Multiple partitions may be included in a virtual database and the state of each partition modified independent of the state of the remaining virtual database including the other partitions. Multiple versions of the same partition corresponding to different points in time are incorporated in the same virtual database, thereby allowing a query to process different versions of the same database table.

Abstract: Information from multiple databases is retrieved and stored on a database storage system. Multiple point-in-time copies are obtained for each database. A point-in-time copy retrieves data changed in the database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system. Each file in the set of files created for a VDB is linked to the database blocks on the database storage system associated with a point-in-time copy of the source database. The set of files associated with the VDB are mounted on a database server allowing the database server to read from and write to the set of files. Workflows based on VDBs allow various usage scenarios based on databases to be implemented efficiently, for example, testing and development, backup and recovery, and data warehouse building.

Abstract: User interactions with a database storage system allow creation of virtual databases based on point-in-time copies associated with a source database. Multiple point-in-time copies are obtained for each source database. A point-in-time copy retrieves data changed in the source database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system and mounting the files on a database server allowing the database server to access the files. User interactions allow the user to specify the source database, a point in time associated with the source database and a destination server to create the virtual database. User input can specify other attributes associated with the virtual database including the file paths, database parameters etc. The user can specify schedules of various actions, including making and retention of point-in-time copies.

Abstract: Virtual databases are provisioned using point-in-time copies of a source database. Transaction logs received from the source database are applied to a point-in-time copy of the source database to create recovery milestones comprising database blocks that can be used to provision virtual databases. The recovery milestones are created in advance of receiving a request to provision a virtual database. Each recovery milestone is validated to determine that a consistent database snapshot can be created using the recovery milestone. If a problem is identified that prevents creation of a consistent database snapshot, the problem is fixed before creating the next recovery milestone. In response to receiving a request to provision a virtual database corresponding to a point in time, a recovery milestone is identified and transaction logs applied to the recovery milestone to create a database snapshot corresponding to the requested point in time.

Abstract: A source storage system replicates snapshots stored on the source storage system to a target storage system. The source storage system stores a plurality of snapshots. The source storage system also stores parent child relationships between snapshots. The source storage system allows child snapshots of a parent snapshot to be modified independent of each other. The source storage system determines changed blocks of a snapshot compared to a previously transmitted snapshot and sends only the changed blocks to the target storage system. The source storage system determines the changed blocks of a snapshot compared to a previously replicated snapshot based on time of creation of a common ancestor of two snapshots. The source storage system transmits a data block of a snapshot if either the data block or a corresponding data block of the previously replicated snapshot was modified after the creation of the common ancestor snapshot.

Abstract: File system backups are performed by copying information describing changes in the file system since a previous point in time. To restore data, a virtual restored file system (VRFS) structure is created corresponding to a snapshot of data copied from the file system that is stored in the backup file system. A client can read the data stored in the backup file system pointed at by the VRFS structure as well as write to the backup file system. Multiple clients can share blocks of data stored on the backup file system via the same VRFS structure or via multiple VRFS structures. If a client writes to a data block pointed at by a VRFS, a copy of the data block is made to which the client makes changes so that the data in the original data blocks is saved for other clients.

Abstract: User interactions with a database storage system allow creation of virtual databases based on point-in-time copies associated with a source database. Multiple point-in-time copies are obtained for each source database. A point-in-time copy retrieves data changed in the source database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system and mounting the files on a database server allowing the database server to access the files. User interactions allow the user to specify the source database, a point in time associated with the source database and a destination server to create the virtual database. User input can specify other attributes associated with the virtual database including the file paths, database parameters etc. The user can specify schedules of various actions, including making and retention of point-in-time copies.

Abstract: User interactions with a database storage system allow creation of virtual databases based on point-in-time copies associated with a source database. Multiple point-in-time copies are obtained for each source database. A point-in-time copy retrieves data changed in the source database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system and mounting the files on a database server allowing the database server to access the files. User interactions allow the user to specify the source database, a point in time associated with the source database and a destination server to create the virtual database. User input can specify other attributes associated with the virtual database including the file paths, database parameters etc. The user can specify schedules of various actions, including making and retention of point-in-time copies.

Abstract: Allocation of resources across multiple consumers allows efficient utilization of shared resources. Observed usages of resources by consumers over time intervals are used to determine a total throughput of resources by the consumers. The total throughput of resources is used to determine allocation of resources for a subsequent time interval. The consumers are associated with priorities used to determine their allocations. Minimum and maximum resource guarantees may be associated with consumers. The resource allocation aims to allocate resources based on the priorities of the consumers while aiming to avoid starvation by any consumer. The resource allocation allows efficient usage of network resources in a database storage system storing multiple virtual databases.

Abstract: Information from multiple databases is retrieved and stored on a database storage system. Multiple point-in-time copies are obtained for each database. A point-in-time copy retrieves data changed in the database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system. Each file in the set of files created for a VDB is linked to the database blocks on the database storage system associated with a point-in-time copy of the source database. The set of files associated with the VDB are mounted on a database server allowing the database server to read from and write to the set of files. Workflows based on VDBs allow various usage scenarios based on databases to be implemented efficiently, for example, testing and development, backup and recovery, and data warehouse building.

Abstract: Information from multiple databases is retrieved and stored on a database storage system. Multiple point-in-time copies are obtained for each database. A point-in-time copy retrieves data changed in the database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system. Each file in the set of files created for a VDB is linked to the database blocks on the database storage system associated with a point-in-time copy of the source database. The set of files associated with the VDB are mounted on a database server allowing the database server to read from and write to the set of files. Workflows based on VDBs allow various usage scenarios based on databases to be implemented efficiently, for example, testing and development, backup and recovery, and data warehouse building.

Abstract: User interactions with a database storage system allow creation of virtual databases based on point-in-time copies associated with a source database. Multiple point-in-time copies are obtained for each source database. A point-in-time copy retrieves data changed in the source database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system and mounting the files on a database server allowing the database server to access the files. User interactions allow the user to specify the source database, a point in time associated with the source database and a destination server to create the virtual database. User input can specify other attributes associated with the virtual database including the file paths, database parameters etc. The user can specify schedules of various actions, including making and retention of point-in-time copies.

Abstract: A storage system maintains copies of a source database to allow virtual databases to be provisioned. Transaction logs received from the source database are applied to database blocks stored in the storage system to update the database blocks. The storage system checks if there are defect in the transaction logs of the source database. If a defect in transaction logs is detected, the storage system obtains a latest copy of the source database. The storage system matches database blocks of the received copy with database blocks already available in the storage system to determine whether a corresponding database block is already available in the storage system. If a received database block is different from database blocks already available at the storage system, the received database block is stored in the storage system. The database blocks may be matched using a hash value of the database blocks.