Abstract: Check point recovery based on identifying changed blocks for block-based backup files is described. At least one data block is identified that changed for a system since a point in time in response to receiving a request to restore the system based on the point in time. At least one data block, corresponding to the identified at least one data block, is recovered from at least one backup file for the system, without reading each data block backed up via the at least one backup file for the system. The system is restored based on the recovered at least one data block.

Abstract: A method begins by processing modules in a dispersed storage network (DSN) identifying a memory device having a legacy slice storage format (SSF) to a second SSF and that includes a first encoded data slice (EDS) of a set of EDSs. When at least a predetermined threshold number of EDSs of the set of EDSs are included within one or storage units (SUs) excluding the first EDS the method continues by transitioning the first SSF of the memory device to the second SSF, and performing a rebuilding process using the at least the decode threshold number of EDSs of the set of EDSs to generate a rebuilt first EDS. The method continues by storing the rebuilt first EDS within the memory device to replace the first EDS that was deleted during the transitioning.

Abstract: Embodiments for data deduplication of an initial snapshot of a data set in a storage system by a processor. An intra-snapshot similarity index, inclusive of representations of the data inside the initial snapshot, is built. The intra-snapshot similarity index is used for deduplication of the initial snapshot. The intra-snapshot similarity index is merged with a global similarity index.

Abstract: A database management tool may migrate a portion of a constrained database by transferring the portion to a constraint-free database from a first constrained database, then populating a second constrained database using the constraint-free data. The constraint-free data may be transformed prior to populating to update internal database identifiers. In some cases, the transformation may also update the data to a different schema of the target database. The database management tool may also include a load balancer that may monitor the loads on a distributed database and may separate portions of the database to different database instances to more effectively load balance the larger database.

Abstract: A distributed storage system called StorFS that performs distributed data deduplication is described. In an exemplary embodiment, a storage controller server determines if there is duplicative data in a distributed storage system. In this embodiment, the storage controller server determines if an entity is duplicated in the distributed storage system in line with an incoming input/output operation. The storage controller server determines if the entity is duplicated in the distributed storage system by receiving the entity and looking up the entity in a data deduplication table. If the entity exists in the data deduplication table, the storage controller server updates the metadata for the entity to point to the duplicate entity.

Abstract: A data transform method and a data transformer. The method includes: importing a data transform rule; acquiring from the data transform rule a source data definition, a destination data definition and a data transform rule definition; predicting resource energy consumption parameters of a data transform node server according to the source data definition, the destination data definition and the data transform rule definition; and deploying a resource energy consumption optimization policy of the data transform node server according to the predicted resource energy consumption parameters of the data transform node server.

Abstract: A method for pre-fetching objects from an object store and storing them in an object cache is disclosed. In one embodiment, such a method includes monitoring access to objects in an object store. The objects have metadata tags, such as user-defined or pre-defined metadata tags, associated therewith. The method further maintains a database for metadata tags associated with the objects, and keeps track of a number of object accesses per metadata tag. In the event a threshold number of object accesses for a particular metadata tag is reached, the method promotes objects having the metadata tag and/or a related metadata tag to an object cache. A corresponding system and computer program product are also disclosed.

Abstract: Processing backup data based on file system authentication is described. A system request authentication information from an application in response to a receipt of a request from the application to process backup data. The system receives encrypted authentication information associated with an authentication system corresponding to a file system. The system requests a permission level from the authentication system based on the encrypted authentication information. The system receives the permission level from the authentication system. The system determines whether the permission level permits the request from the application to process the backup data. The system processes the backup data for the application in response to a determination that the permission level permits the request from the application to process the backup data.

Abstract: One or more techniques and/or computing devices are provided for directory level incremental replication. For example, a first storage controller may evaluate a base snapshot and an incremental snapshot of a source subdirectory to generate a set of operations that can be used by a second storage controller for reconstructing a mirror of the source subdirectory as reflected by the incremental snapshot. Accordingly, the first storage controller may send the set of operations and/or source data to the second storage controller for constructing a destination directory structure mirroring the source subdirectory. In this way, replication may be achieved at an arbitrary level of granularity, such as to replicate a particular subdirectory of a volume.

Abstract: According to certain aspects, an information management system may instruct a first secondary storage computing device in a set of secondary storage computing devices to initiate a restore of a first group of a plurality of files from a first storage device associated with the first secondary storage computing device, the first group including a first file. The computing system may instruct a second secondary storage computing device in the set to initiate a restore a second group of the plurality of files from a second storage device associated with the second secondary storage computing device, the second group including the first file. The computing system can, upon receiving notification that the restore of the first file from the first storage device has completed prior to completion of the restore of the first file from the second storage device, cancel restoration of the first file from the second storage device.

Abstract: A system and method on one computer records actions on objects performed on a second computer. If an object is deleted from the second computer, the first computer can restore to a point in time the state of the object on the second computer, including assignments made between it and other objects, using the recorded actions.

Abstract: A system and method for managing data transfer operations includes at least one data server including data stored in a plurality of data fields, at least one transaction server operatively coupled to the at least one data server and configured to provide at least one customizable business rule and a customer relationship application, an information server operatively coupled to the at least one transaction server, and at least one workstation operatively coupled to the information server. The least one workstation includes local memory accessible to the customer relationship application and configured to provide web communication and presentation services to a user of the at least one workstation that are deployed on the information server. The at least one customizable business rule specifies a mapping between the plurality of data fields and the local memory, the mapping having been provided by the user interacting with the web communication and presentation services.

Abstract: Systems and methods for performing file-level restore operations for block-level data volumes are described. In some embodiments, the systems and methods restore data from a block-level data volume contained in secondary storage by receiving a request to restore one or more files from the block-level data volume, mounting a virtual GUID Partition Table (GPT) disk to the block-level data volume, accessing one or more mount paths established by the virtual GPT disk between the data agent and the block-level data volume, and browsing data from one or more files within the block-level data volume via the established one or more mount paths provided by the virtual GPT disk.

Abstract: A method begins by a first computing device of a dispersed storage network (DSN) of a plurality of DSNs receiving a data segment for storage, encoding the data segment using a dispersed storage error encoding function to produce a set of encoded data slices, where each DSN of the plurality of DSNs utilizes the dispersed storage error encoding function, and generating a set of DSN addresses for the set of encoded data slices using a deterministic function. The method continues with a set of storage units of the DSN storing the set of encoded data slices using the set of DSN addresses. The method continues with storage units of the set of storage units and of another set of storage units of another DSN coordinating to store copies of encoded data slices of the set of encoded data slices in corresponding storage units of both sets of storage units.

Abstract: A system and method for managing data transfer operations includes at least one data server including data stored in a plurality of data fields, at least one transaction server operatively coupled to the at least one data server and configured to provide at least one customizable business rule and a customer relationship application, an information server operatively coupled to the at least one transaction server, and at least one workstation operatively coupled to the information server. The least one workstation includes local memory accessible to the customer relationship application and configured to provide web communication and presentation services to a user of the at least one workstation that are deployed on the information server. The at least one customizable business rule specifies a mapping between the plurality of data fields and the local memory, the mapping having been provided by the user interacting with the web communication and presentation services.

Abstract: A first data set is obtained specifying configuration information associated with elements of a computing system. A second data set is also obtained specifying dependency information associated with the elements of the computing system. A third data set is also obtained specifying deployment constraint information associated with the elements of the computing system. A plan for migrating one or more of the elements of the computing system is automatically generated based on at least a portion of the first data set, at least a portion of the second data set, and at least a portion of the third data set. The automatic generation of the migration plan checks for one or more conflicts between configuration information, dependency information and deployment constraint information and generates the migration plan to at least substantially eliminate the one or more conflicts.

Abstract: Systems and methods are provided for data migration. For example, a data packet of a first platform is acquired; the data packet is parsed to obtain one or more data structure fields of the data packet; the data structure fields of the data packet are matched with a predetermined first field mapping table; in response to the data structure fields of the data packet being matched with the predetermined first field mapping table, the data packet is associated with a second platform based at least in part on one or more predetermined association rules; the data packet is converted based at least in part on the association of the data packet and the second platform; and the converted data packet is migrated to the second platform.

Abstract: In various embodiments, a data integration system is disclosed which enables users to create a logical design which is platform and technology independent. The user can create a logical design that defines, at a high level, how a user wants data to flow between sources and targets. The tool can analyze the logical design, in view of the user's infrastructure, and create a physical design. The logical design can include a plurality of components corresponding to each source and target in the design, as well as operations such as joins or filters, and access points. Each component when transferred to the physical design generates code to perform operations on the data. Depending on the underlying technology (e.g., SQL Server, Oracle, Hadoop, etc.) and the language used (SQL, pig, etc.) the code generated by each component may be different.

Abstract: Changes in a multi-RAID configuration to metadata of a source production volume in a source RAID are monitored since taking a most recent FC backup on the source production volume. The incremental copies of the metadata are stored on a target production volume in an target RAID. An incremental FC backup of the metadata are periodically created or created if the monitored changes exceed a change rate threshold.

Abstract: In one embodiment, a method includes accessing a mapping that represents a correspondence between a column in a source database and a column in a temporary storage table, in response to receiving a data operation request to execute a data operation on a column of the source database. The method further includes changing the temporary storage table by executing the data operation on the temporary storage table and performing a reconstruction operation using data from the temporary storage table and the source database.

Abstract: Disclosed are some examples of systems, methods and storage media for associating a group of users to a record and facilitating collaboration on the record by the users via a group feed of an enterprise network. In some implementations, a system includes first data associating each of a plurality of group identifiers to one or more record identifiers, and second data associating each of a plurality of feed item identifiers to a respective group identifier or record identifier. In one implementation, the system is configured to receive a request for a first group feed associated with a first group identifier. Based on the request, the system identifies one or more first record identifiers associated with the first group identifier, identifies one or more first feed item identifiers associated with the first group identifier or the first record identifiers, and generates the first group feed to include the corresponding feed items.

Abstract: Technology is disclosed for subpartitioning a namespace region. In various embodiments, the technology creates at least two subpartitions from a partitioned namespace, wherein the partitioned namespace corresponds to at least two different name nodes of the large scale data storage service; and stores data corresponding to each subpartition as a separate file, e.g., so that it can be easily mounted by an operating system executed by a different computing device.

Abstract: Methods, systems, and apparatus, including computer program products, for discovering entities in a first portion of a data center network, examining information associated with the discovered entities to determine relationships that exist between pairs of entities in the first portion of the data center network, and generating a specification of the relationships that exist between pairs of entities in the first portion of the data center network based on results of the examining.

Abstract: A database system may include a database service and a separate distributed storage service. The database service (or a database engine head node thereof) may be responsible for query parsing, optimization, and execution, transactionality, and consistency, while the storage service may be responsible for generating data pages from redo log records and for durability of those data pages. For example, in response to a write request directed to a particular data page, the database engine head node may generate a redo log record and send it, but not the data page, to a storage service node. The storage service node may store the redo log record and return a write acknowledgement to the database service prior to applying the redo log record. The server node may apply the redo log record and other redo log records to a previously stored version of the data page to create a current version.

Abstract: A hybrid-sharded index includes document-sharded posting lists and term-sharded posting lists. Implementations include systems and methods for updating a hybrid-sharded index. For example, a method may include receiving updates to the hybrid-sharded index and generating, at a first leaf node, replacement posting lists and change information for a respective second leaf node. The method may also include dividing the replacement posting lists into portions, a portion having associated change information and being associated with a respective one of the second leaf nodes and sending the portions to respective leaf nodes. At a particular leaf node of the second leaf nodes, the method includes merging a received portion into an updated posing list portion, swapping the updated posting list portion into memory. During the swap, the change information and the updated posting list portion are used to respond to a query with an older version of the hybrid-sharded index.

Abstract: Various embodiments for improving hash index key lookup caching performance in a computing environment are provided. In one embodiment, for a cached fingerprint map having a plurality of entries corresponding to a plurality of data fingerprints, reference count information is used to determine a length of time to retain the plurality of entries in cache. Those of the plurality of entries having a higher reference counts are retained longer than those having lower reference counts.

Abstract: This disclosure relates to a method, article of manufacture, and apparatus of restoring a database allowing simultaneous full and incremental backups. In some embodiments, this includes receiving, by a server, a request for restoring as of a restore point-in-time, in response to the request, retrieving by the server a set of backup objects, sorting the set of backup objects in descending order, locating a first incremental backup in the set of backup objects based on the restore point-in-time and a first incremental backup start time associated with the first incremental backup, locating a full backup in the set of backup objects based on the first incremental backup start time, a full backup start time and a full backup end time associated with the full backup, storing the full backup and the first incremental backup in a list of restoring objects, and sending, by the server, the list of restoring objects.

Abstract: A data migration tool is disclosed for allowing a data migration project to be more efficient to execute from the client standpoint. The data migration tool is able to accomplish this by offering an efficient process for receiving the client's data migration requirements and generating a migration schedule based on the client's data migration requirements. The client's data migration requirements may identify one or more migration rules relating the data to be migrated as well as the client's scheduling conflicts. Based on the data migration requirements, the data migration tool can generate a data migration schedule. The data migration schedule may include a sequence in which data entities are migrated, as well as provide migration activities that describe the specific instructions for migrating a data entity.

Abstract: Systems and methods for item-level restoration from and verification of an image level backup without fully extracting it. The method receives backup parameters and selection of an image level backup to restore or verify and initializes virtual storage. The method attaches the virtual storage to a hypervisor to launch a virtual machine (VM) to test and restore data objects. The method stores VM virtual disk data changes resulting from restoration and verification in a changes storage. The method optionally reconfigures VMs to use an isolated network. The method optionally uses a routing appliance to provide access to VMs running in the isolated network from a production network. The method determines if the VM operating system (OS) is able to start using restored copies of selected data objects and tests applications associated with selected data objects. The method displays restoration and test results in an interface and automatically delivers the results.

Abstract: Mechanisms are provided for performing efficient compression and deduplication of data segments. Compression algorithms are learning algorithms that perform better when data segments are large. Deduplication algorithms, however, perform better when data segments are small, as more duplicate small segments are likely to exist. As an optimizer is processing and storing data segments, the optimizer applies the same compression context to compress multiple individual deduplicated data segments as though they are one segment. By compressing deduplicated data segments together within the same context, data reduction can be improved for both deduplication and compression. Mechanisms are applied to compensate for possible performance degradation.

Abstract: Techniques are described for expressing a constraint in a host language. A constraint can be defined in the host language as an event handler. The event handler can be registered to a built-in event associated with an object of the database that the constraint is intended to validate. When the object triggers the built-in event, the event handler is called to execute the constraint. Exemplary built-in events include onSave (trigger the event when the object is saved), onValidate (trigger the event when an explicit call to validate the object is detected), onInsert (trigger the event when the object is inserted), onUpdate (trigger the event when the object is updated), and onCheckBeforeSave (trigger the event when the object is checked prior to saving).

Abstract: This disclosure relates to a method, article of manufacture, and apparatus for backing up data. In some embodiments, this includes a backup server locating one or more instances of data associated with a list of identifiers, wherein each instance of data comprises one or more target volumes and associated journal volume; the enablement of replication; and replication.

Abstract: Service level representations of applications being made highly available are automatically created and published, based on their operational representations. A repository containing multiple application service templates is maintained. Each application service template defines the attributes of an application type to include in corresponding service level representations. To create a service level representation for a specific application instance, an appropriate application service template is selected from the repository. The attributes of a copy of the selected template are populated with values describing the application instance and its associated resources (including a service identifier), thereby automatically creating a service level representation of the application instance. The attributes can be populated with data read from resources of the operational representation of the application instance.

Abstract: Assignment of files to a de-duplication domain. Address space of data files is divided into multiple containers. For each of the containers, a file metadata scan is performed to obtain file system metadata, which is aggregated and summarized in a content feature summary. A content feature summary prediction measurement is measured between containers from the generated content feature summary, and files from each container are assigned to a de-duplication domain based upon the content similarity predication measurement.

Abstract: Assignment of files to a de-duplication domain. Address space of data files is divided into multiple containers. For each of the containers, a file metadata scan is performed to obtain file system metadata, which is aggregated and summarized in a content feature summary. A content feature summary prediction measurement is measured between containers from the generated content feature summary, and files from each container are assigned to a de-duplication domain based upon the content similarity predication measurement.

Abstract: A data storage system according to certain aspects can archive database data associated with different database applications. The data storage system according to certain aspects may provide database archiving modules that include logic incorporating and/or based on the native schema and/or native commands specific to particular database applications. The database archiving modules according to certain aspects may determine the relationship between tables associated with corresponding database applications and archive selected database data based on the native schema and native commands.

Abstract: A system and method for linking parent and child data records during data migration is disclosed. One embodiment comprises sequentially receiving a plurality of data records from an arbitrary source of records without running a query on the source of records, generating a destination store of records that contains the plurality of data records, and locating any records in the destination store of records that has a dangling key. In this way, the embodiment may correct dangling keys to generate a destination store of records with referential integrity.

Abstract: A data transform method and a data transformer. The method includes: importing a data transform rule; acquiring from the data transform rule a source data definition, a destination data definition and a data transform rule definition; predicting resource energy consumption parameters of a data transform node server according to the source data definition, the destination data definition and the data transform rule definition; and deploying a resource energy consumption optimization policy of the data transform node server according to the predicted resource energy consumption parameters of the data transform node server.

Abstract: Various embodiments for improving hash index key lookup caching performance in a computing environment are provided. In one embodiment, for a cached fingerprint map having a plurality of entries corresponding to a plurality of data fingerprints, reference count information is used to determine a length of time to retain the plurality of entries in cache. Those of the plurality of entries having a higher reference counts are retained longer than those having lower reference counts.

Abstract: Methods, systems, and apparatus, including computer program products, for discovering entities in a first portion of a data center network, examining information associated with the discovered entities to determine relationships that exist between pairs of entities in the first portion of the data center network, and generating a specification of the relationships that exist between pairs of entities in the first portion of the data center network based on results of the examining.

Abstract: Techniques for computer system recovery which remotely restore a default partition to a recent state even when an operating system is functioning abnormally. In an example embodiment, a service center computer establishes a first network connection to a monitored computer system. The service center computer configures the monitored computer system to boot from a bootable image file in the monitored computer system and reboots the monitored computer system into an alternate operating system environment of the bootable image file. The service center computer establishes a second network connection to the monitored computer system to restore a recent backup image of the default partition from a diagnostic partition to a default partition. The service center computer establishes a third network connection to the monitored computer system and reboots the monitored computer system to the default partition.

Abstract: A method enables cluster-level backup and restore functionality of all Virtual Input/Output Server (VIOS) configuration data within a VIOS cluster and the data of a shared VIOS cluster database. The method comprises: performing, via a backup/restore utility of a VIOS partition, a cluster level backup, which creates a first VIOS cluster configuration backup file having configuration information about hardware, logical and virtual devices of each VIOS partition within a VIOS cluster and all cluster data from the shared VIOS database of the VIOS cluster; storing the VIOS cluster configuration backup file within a storage location; and responsive to receipt of a VIOS restore command at a VIOS partition: retrieving the configuration backup file from the storage location; restoring a configuration of the hardware, logical and virtual devices of each VIOS within the VIOS cluster to prior state; and restoring the shared VIOS database with the backed-up cluster data.

Abstract: A system for restoring a backup data comprises a processor and a memory. The processor is configured to: 1) receive an indication to restore a backup data to a virtual storage device location; 2) determine a map between a virtual storage device volume name and a virtual storage device backup serial number; and 3) retrieve backup data from the virtual storage device backup. The memory is coupled to the processor and configured to provide the processor with instructions.

Abstract: A versioned file system comprises a set of structured data representations. Each structured data representation corresponds to a version, and each version comprises a tree of write-once objects. Each version in the versioned file system has associated therewith a borrow window. When it is desired to reconstruct the file system to a point in time, i.e., to perform a “restore,” it is only required to walk a single structured data representation. During a restore, metadata is pulled back from the cloud first, so users can see the existence of needed files immediately. The remainder of the data is pulled back from the cloud when the user goes to open the file. As a result, the entire file system is restored to a previous time nearly instantaneously. A fast restore is performed if an object being restored exists within a borrow window of the version from which the system is restoring.

Abstract: The description is directed to systems and methods for restoring a program state retained from a prior execution session on a virtual machine. On receiving a request to execute a program an image of user-independent files are mounted to a virtual machine. Specified user-modifiable files are copied from a particular user storage location to put the program in a condition to execute the program so that it begins from the previously-existing program state. During the session at least some of the files are modified and on a session end the user-modifiable files are saved to the particular user storage location to retain the updated program state.

Abstract: Systems and methods for restoring file system objects from image level backups of servers. The method receives a selection of an image level backup of a source, file system object(s) to be restored, and restoration parameters identifying a destination. The method initializes and attaches a virtual storage to a virtual appliance and an associated hypervisor. The method stores virtual storage data changes resulting from the file system object restoration in changes storage. The method preserves file system object information, wherein the file system object information corresponds to the source operating system (OS) and file system. The method preserves file system object information such as permissions and ownership through business logic that ensures that the file system object information for restored file system object(s) on the destination match file system object information from the source, even when the method operates on a host running a different OS than the source server.

Abstract: A method and apparatus for processing a transform function and parameter information to represent data files is described. In one embodiment, the method includes processing at least one data file to identify a reference file, a transform function and parameter information and storing the transform function and the parameter information, wherein the transform function and the parameter information is applied to the reference file to create the at least one data file.

Abstract: Systems and methods are disclosed for rapidly restoring client data set for a computer by storing the client data and one or more pat sets required to revert to one or more version of the client data on a remote server; storing a local copy of the replicated client data on a local data storage device coupled to the computer; receiving a request to revert to a predetermined version of the client data; using the local copy as a seed, receiving a patch set corresponding to a predetermined version; and updating the local copy using the patch set to generated the predetermined version.

Abstract: Architecture for restoring nodes. After restoring a node, fix-up occurs to make the node appear as a different node than before the restore operation. The node appears as a new node, which new node knows the data up to a certain point from when the new node had the prior identity. This enables new changes generated by the new node to flow to the other nodes in the topology, as well as have the changes that the prior identity sent to other nodes flow back to the new node. In other words, the architecture maintains information to create the new node in the topology while maintaining prior data knowledge. Additionally, item level metadata of associated data items is updated to correlate with the updated data items so that changes can be correctly enumerated and applied. This metadata update occurs across scopes of which the data items are included.

Abstract: Systems and methods for restoring files and metadata records associated with files are provided herein. Some of the methods include identifying a version of the file, evaluating one or more reverse incremental delta increments, and restoring an arbitrary version of the file based on an arbitrary time reference by assembling the arbitrary version of the file based on the one or more reverse incremental delta increments.