Abstract: Optimizing data movement from a source data center to a target data center by grouping metadata of data objects into a dataset that encompasses data processed identically by a data processing operation, where the dataset defines a single data access unit for these data objects, and the data processing operation processes them as a single unit based on data content rather than physical or logical data location of the data objects. A mobility process correlates an increase in a number of metadata elements in the dataset with a growth rate of the dataset, and compares the dataset growth rate with historical or similar dataset growth rate data. The comparison is used to determine when and where to move data from the source to the target data center based on a forecast of accelerated growth indicated by the comparing step, and a consideration of target data center resources, data move costs, and streaming data effects.

Abstract: Providing content based data protection for data stored in a large-scale data storage system by scanning data stored in one or more databases for discovery of metadata, and extracting the discovered metadata, for storage in a data catalog, the data catalog having a scanning function performing the scanning step, and comprising a database storing the metadata in one or more tables. A protection policy is defined to commonly protect content data referenced by metadata in the data catalog, and applied to the referenced content data to perform a data protection operation the content data. Datasets stored in the catalog are generated by running queries on the catalog, where a query comprises metadata selectors as tags applied to the catalog, where the tags define at least one of a file type, name, location, creation time, or file characteristic.

Abstract: Embodiments of monitoring data assets in a system to apply rules to optimize storage and access of the data assets based on data content rather than data location, by defining rules based on the monitoring attributes, wherein a rule dictates a storage location of selected data or access permissions to the data by one or more persons or groups in the system. The selected data is tagged with a defined metadata tag, and a dataset is created by running a query against a data catalog to derive the dataset. A component monitors data usage and access of data elements referenced by the dataset to detect any violations of the defined rules, and provides a notification of any violation to facilitate remedial action by a user or process. The dataset can span multiple storage devices of different types to define a single processing unit for the monitoring attributes.

Abstract: Providing content based data protection for data stored in a large-scale data storage system by creating a dataset by grouping metadata for unstructured data objects that are grouped together by one or more filters. The dataset can span multiple storage devices of different types, so that it defines a single data protection unit for the corresponding content data. A user initiated query generates the one or more filters, and a protection policy is defined that protects the dataset as the single unit based on data content rather than data location. Datasets are stored in a catalog, and are generated by running queries on the catalog, where a query comprises metadata selectors as tags applied to the catalog, where the tags define at least one of a file type, name, location, creation time, or file characteristic.

Abstract: Providing content based data access protection for data stored in a system by creating a dataset by grouping metadata for data objects that are grouped together by one or more filters. The dataset can span multiple storage devices of different types to define a single data access protection unit for the corresponding content data. A user query generates the one or more filters, and an access rule is defined that allows or denies access to the dataset by users or processes as the single unit based on data content rather than location. The access rule can comprise at least one of an Access Control List (ACL) rule or a Role-Based Access Control (RBAC) rule, where the ACL lists permissions associated with certain data elements that grant access to specific users or processes, and the RBAC rules allow or deny access on the basis of role-permissions within the system.

Abstract: Providing content-based encryption to content data in a data processing system by creating datasets by grouping metadata for data objects that are intended to be encrypted with a common encryption key, where each dataset spans multiple storage devices of different storage types, and defines a single data encryption unit for the data objects referenced by a respective dataset. Each dataset is tagged with an encryption tag to enable or disable use of a self-selected encryption key. Encryption keys stored in or made available to the system are accessed for encrypting the data objects using an encryption process. A key management component maps each dataset to a corresponding encryption key of the encryption keys, and an encryption component encrypts, for each dataset, referenced data objects using a corresponding mapped encryption key.

Abstract: Organizing data for data protection based on content data stored in a large-scale data storage system. Embodiments create a dataset by grouping metadata for unstructured data objects that are grouped together by one or more filters. Datasets can be static or dynamic and can span multiple storage devices of different types, so that it defines a single data protection unit for the corresponding content data. A user initiated query generates the one or more filters, and a protection policy is defined that protects the dataset as the single unit based on data content rather than data location. Datasets are stored in a catalog, and are generated by running queries on the catalog, where a query comprises metadata selectors as tags applied to the catalog, where the tags define at least one of a file type, name, location, creation time, or file characteristic.

Abstract: Classifying data objects for the content-based protection and process control in a system and modifying a classification based on evolving data. Data objects stored in the system are scanned to identify data objects to be processed similarly with respect to data protection or access control. A dataset comprising metadata for corresponding data objects is generated, and a classifier labels the dataset with a classifier tag to indicate an ownership group. Data objects that belong to the dataset are similarly tagged with the classifier so that the same operations are performed on this data regardless of location. A monitor tracks changes in the dataset based system evolution to determine a change in the classifier for the dataset based on the change. A classifier behavior tag is appended to the dataset specify an operation of the classifier to accommodate the tracked change.

Abstract: Managing versioning of data objects for a project revised from a first version to a revised version by producing a dataset representing the data objects as a group by scanning the data objects to identify metadata of the grouped data to be processed similarly within a current version of the lifecycle, and storing the identified metadata in the dataset. Data object changed from the first version to the revised version are identified, and the corresponding metadata for changed data objects in the dataset is updated. A version control operation is then performed on the dataset to update all data objects referenced by the dataset from the first version to the revised version. A commit-map and commit-tree are stored in a repository, and version control operations including commit, checkout, merge, branch and merge-branch are performed on the dataset snapshot.

Abstract: Managing a lifecycle of data by identifying data objects that are subject to same control rules in each stage of the lifecycle as grouped data, where the control rules allow only authorized access to or authorized operations on the grouped data based on a current stage of the lifecycle. A dataset is generated for the grouped data by identifying metadata of the grouped data to be processed similarly within the lifecycle, and storing the metadata in the dataset. The control rules associated with the grouped data as stage tags for the dataset. Actions performed on the data referenced by the dataset are monitored to ensure that the monitored actions comply with control rules using the stage tags of the dataset.

Abstract: Providing content-based protection and process control to data objects in a multi-network system, by scanning content data to identify metadata associated with data objects to be protected by a defined protection policy. The content data is stored in storage devices comprising network attached storage (NAS), object storage, local storage, or cloud networks, and where the storage devices are deployed in different networks including core networks, edge networks, and cloud networks. The gathered metadata is stored in a catalog, and a user entered query is executed against the catalog to generate a dataset. The defined protection policy is then applied to the dataset to operate on the corresponding data objects referenced by the dataset as a single unit based on data content rather than data location in a file directory or physical location within the multi-network system.

Abstract: Providing content-based sensitivity classification to content data in a data processing system, by defining sensitivity designations for data objects stored in the system, and creating datasets by grouping metadata for data objects that are intended to classified with a same sensitivity designation, wherein each dataset spans multiple storage devices of different storage types, and wherein each dataset defines a single data sensitivity unit for the data objects referenced by a respective dataset. A sensitivity classifier component tags each dataset with a sensitivity tag to specify a protection or control operation on the data objects referenced by the respective dataset, and a processing component then processes data objects for each tagged dataset in accordance with the specified protection or control operation.

Abstract: Embodiments applying data protection and control policies using content-based datasets by scanning data objects stored in the system to determine grouped data that is processed similarly with respect to data protection and access control operations defined by a policy. A dataset is produced comprising metadata the scanned data objects of the grouped data. The actions performed on the dataset will affect only the corresponding data objects referenced by the metadata. A policy attribute derivation (PAD) process determines a change in the policy affecting a subset of data objects of the dataset and dictating changed data protection and access control operations applied to this subset, and tags the change in the policy as a PAD tag to the dataset to affect the application of the changed data protection and access control operations only to the subset and not any remaining data objects of the dataset.

Abstract: Performing charge/showback operations in a large-scale data system by scanning multiple data sources to identify data objects for processing as a unitary group with respect to common characteristics that have an impact on finances within the system. Metadata of the identified data objects are stored in a dynamic dataset that defines a single data access unit for the referenced data objects. The system processes a user query regarding cost allocations, cost forecasts, and resource usage of respective groups within an organization. The query initiates a charge-back or show-back operation that allocates costs associated with each respective usage of resources by a department or cost center, and accesses the referenced data objects through the dataset as a single unit based on data content rather than data location in a file directory of the system.

Abstract: Enforcing a legal hold procedure in a system by scanning multiple data sources to identify data objects for processing as a unitary group with respect to common access and control processes of the legal hold to preserve the data for a defined period of time and protected against modification and unauthorized access. The metadata is stored in a static dataset that defines a single data access unit for the referenced data. A user query regarding a referenced data object is processed, and accesses the data through the dataset as a single unit based on data content rather than data location in a file directory of the system. The data may be sensitive data and the legal hold procedure may be implemented as court rules in accordance with Federal Rules of Civil Procedure.

Abstract: Providing content based data protection for data stored in a large-scale data storage system by creating a dataset by grouping metadata for unstructured data objects that are grouped together by one or more filters. The dataset can span multiple storage devices of different types, so that it defines a single data protection unit for the corresponding content data. A user initiated query input through a search engine interface generates the one or more filters, and a protection policy is defined that protects the dataset as the single unit based on data content rather than data location. Datasets are stored in a catalog, and are generated by running queries on the catalog, where a query comprises metadata selectors as tags applied to the catalog, where the tags define at least one of a file type, name, location, creation time, or file characteristic.

Abstract: Embodiments help timely remediation of issues in a data protection system by automatically evaluating scripts configured to address the issues. The system has a bounded list of possible attributes that are deemed important by system administrators. Each attribute is assigned a System Impact Score (SIS) along a defined scale. A self-healing processing component monitors the state of each attribute over time. The scripts are evaluated through repeated execution and use of the attribute monitoring to determine which attributes are affected by a script. Weights are assigned to each attribute affected by a script to aid in the selection of scripts most likely to remediate an actionable issue. Regularly performing script evaluation and attribute weighting allows for updating of scripts with an accurate list of attributes to overcome problems associated with manual updates.

Abstract: A portable, self-contained data mover container with all required packages for data management, and which can be hosted on a physical host, VM, cloud or appliance, such as a storage array. The data mover container protects multiple workloads such as NAS, native OS filesystem, applications, and so on. An automated process spins up and tears down containers without any impact on the host. If a container fails to run on a host if desired resources are not available, it automatically runs on other hosts for same data set. Minimal user intervention to achieve scale and performance is required, as the system automatically computes the desired number of containers required depending upon dataset and available resources. Containers can be patched and upgraded independently on a host without making any change to the hosting environment.

Abstract: A three-phase full quorum commit method enabling backing up of network devices that do not offer direct hooks in order to have application consistent protection. Devices are verified to be ready to perform a backup, and a condition of reaching and maintaining a full quorum of devices within a maximum time period is required before the system can be backed up. The three phase backup process reduces the chance of changes to network devices from corrupting consistency among the saved states of the different and disparate network devices. Multiple devices of different makes and models participate together as a unified backup as a network partition and all devices are verified as being in a ready state. The device configuration data is moved from device memory to local disk, and can then be tiered to secondary storage.

Abstract: Embodiments automate the timely remediation of issues by matching failed operations to a script or set of scripts that is configured to fix the problem. A method detects a backup operation alert message sent from a data manager or storage system that encodes one or more attributes affected by the backup operation, wherein the attributes are weighted to reflect a percentage importance of the a corresponding script to fix the problem. A script library is scanned to identify scripts that match the alert message. A component generates a score for each script, where the score factors the relevance of the script to the backup operation based on the weighted attributes. Scores for each single script each possible combination of multiple scripts are computed and compared, and the script or script combination with the highest score is executed by the self-healing process.