Abstract: In asynchronous processing, processing of a low-order system is checked. A high-order system 1 includes a high-order workflow advancing unit 21 that transmits a processing command for asynchronous processing to a low-order system 2, and a notice processing unit 23 that receives, from the low-order system 2, a notice related to a device command to be executed by a device 3 in the asynchronous processing, and updates progress data 13. The low-order system 2 includes a low-order workflow advancing unit 61 that, upon receiving the processing command from the high-order system 1, transmits a device command to the device, and a notification unit 62 that, when transmission of the device command to the device 3 is detected, notifies the high-order system 1 of the identifier of the device to which the device command was transmitted and the device command.

Abstract: A method of processing an input task in a processing system involves duplicating the input task so as to form a first task and a second task; allocating memory including a first block of memory configured to store read-write data to be accessed during the processing of the first task; a second block of memory configured to store a copy of the read-write data to be accessed during the processing of the second task; and a third block of memory configured to store read-only data to be accessed during the processing of both the first task and the second task; and processing the first task and the second task at processing logic of the processing system so as to, respectively, generate first and second outputs.

Abstract: Systems, methods, and non-transitory computer readable media are provided for using linked documents. A system may receive, from a computing device, a request for a document. Content of the document may be defined based on state information and stateless information. A system may determine a local replica of the document in a local database. The local replica of the document may be linked to a primary replica of the document. The local replica of the document may include a snapshot of the primary replica of the document. The primary replica of the document may be stored in a remote database which may be accessible through a remote server. The system may subscribe to the primary replica of the document through the remote server, and may provide access to the document to the computing device based at least in part on the subscription to the primary replica of the document.

Abstract: Embodiments of the present invention assist in the development, management, and deployment of aggregated data attributes for multiple data sources. One embodiment provides a development interface that allows for elements of attributes, including filters, to be moved into a coding area in which an attribute or an attribute element is being edited. In another embodiment, the user interface presents data fields to assist in the development of filters for multiple data sources with divergent formats. The application further provides a validation interface through which users can validate attributes and trace the results returned by various elements referenced by the attributes under validation. Another embodiment provides a system for managing attributes and deploying them to various systems by creating a deployment file that is used by an attribute calculation system. In one embodiment, the attribute calculation system is a scalable system that dynamically calculates attributes for multiple data sources.

Abstract: Change data capture is performed asynchronously for direct transfer to a stream processor. When updates to a database as part of database transactions are performed, a transaction log is generated that describes the database transactions. The transaction log may be evaluated to determine indications of changes included in the database transactions. The indications of the changes may be stored in memory according to the database transactions that include the indications of the changes. When a database transaction is determined to be committed to the database, the change indications may be sent directly from the memory to the stream processor.

Abstract: At least a portion of data of a tree data structure is serialized into a first set of flat data. At least a portion of a structure of the tree data structure is serialized to reproduce at least the portion of the structure in a second set of flat data. It is determined to access a desired data of the tree data structure from an archive. The second set of flat data is read to reconstitute at least the portion of a structure of the tree data structure. The reconstituted structure of the tree data structure is used to determine a data offset associated with the first set of flat data, wherein the data offset corresponds to the desired data. The desired data is accessed using the data offset associated with the first set of flat data.

Abstract: Supporting multiple replication schemes across distinct network layers, including: replicating, over a first type of network messaging layer, data between a first storage system and a second storage system; selecting a different messaging layer for data replication; and replicating, over a second type of network messaging layer, data between the first storage system and the second storage system.

Abstract: A method and apparatus for replicating a data container of a source storage server at the logical level in an unordered stream of individual data units are described. In certain embodiments, the replication operations can be performed without regard to the differences in geometry of physical persistent storage of the source and destination storage servers and without regard to the differences in data container format of the source and destination storage servers. An initial replication operation of the source data container is performed by transferring the data objects within the source data container to a replica data container in the destination storage server as an unordered stream of individual data units while preserving data object identifiers of the source data objects. Afterwards, incremental replication operations can be performed to capture modifications to the source data container over time after the initial replication operation.

Abstract: The present concepts relate to contiguously writing data blocks of a data group in storage media in a one-step writing operation without defragmenting. A host may assign a group identifier to the data group, tag the data blocks in the data group with the group identifier, and send the data blocks to a storage controller. In turn, the storage controller may receive the data blocks that are tagged with the common group identifier, and contiguously write the data blocks in contiguous free space in the storage media. As such, reading the data group from the storage media may be performed faster than if the data group has been allowed to be written with fragmentation. The present concepts may also avoid the costs associated with a two-step writing operation that involves defragmenting.

Abstract: A computer-implemented method according to one embodiment includes identifying data to be transferred as part of a point-in-time snapshot copy operation; setting a data consistency application in an idle state; marking the data while the data consistency application is in the idle state; restarting the data consistency application; identifying, by the data consistency application, the marked data; creating, by the data consistency application, a consistency group including the marked data; and performing replication of the consistency group.

Abstract: The present invention provides for employing SQL to introduce blockchain technologies into a relational database, and thereby leverage the inherent tamper-resistant properties of blockchain, without the need to completely rewrite existing or legacy relational database software. The invention creates a relational database inside of a blockchain and uses a conventional SQL interface for standard database operations. This reduces the burden of introducing blockchain technologies, while providing the benefits of the intrinsic security and verification features of blockchain technology. The invention provides a rich historical record of every transaction thereby greatly reducing, if not eliminating, the relational database's susceptibility to tampering. This allows for temporal queries on arbitrary records within the database and the generation of reports and audits for any point in the history of the database.

Abstract: Data is replicated on a backup node, where the granularity of the replication can be less than a full volume. A data consistency group comprising a subset of data for a volume is defined for a primary node. A set of differences for the data consistency group is sent to a backup node. The backup node creates change logs in response to receiving the set of differences. In response to receiving a request to access a file having data in the data consistency group, the backup node creates a clone of the file. The backup node determines whether an update to a data block of the file exists in the change logs. In response to determining that the update to the data block exists in the change logs, the backup node updates a copy of the data block for the cloned file with data in the change logs.

Abstract: Systems and methods are described herein for stopping a data replication process on database systems. The method includes identifying a data replication process corresponding to a downstream database system and an upstream database system. The method also includes determining a first status of the data replication process at the downstream database system and upstream data corresponding to the upstream database system. The method also includes determining a second status of the data replication process at the upstream database system based on the upstream data. The method also includes stopping the data replication process at the upstream database system based on the second status and stopping the data replication process at the downstream database system based on the first status. The method also includes generating a data log notification including data corresponding to the stopped data replication process.

Abstract: A system establishes spools that have spool regions in first memory. The system communicates first spool insertions into the spools, and from the spools to the storage devices, to write into a first partition of second memory of the storage devices. The first spool insertions include data and are optimized towards a first spool insertion size. The system communicates second spool insertions into the spools, and from the spools to the storage devices, to write into a second partition of the second memory of the storage devices. The second spool insertions include metadata and are optimized towards a second spool insertion size.

Abstract: Systems, methods, and non-transitory computer readable media are provided for using linked documents. A system may receive, from a computing device, a request for a document. Content of the document may be defined based on state information and stateless information. A system may determine a local replica of the document in a local database. The local replica of the document may be linked to a primary replica of the document. The local replica of the document may include a snapshot of the primary replica of the document. The primary replica of the document may be stored in a remote database which may be accessible through a remote server. The system may subscribe to the primary replica of the document through the remote server, and may provide access to the document to the computing device based at least in part on the subscription to the primary replica of the document.

Abstract: A bulk data distribution system in which, instead of multiple data consumers contending to access the same data sets from a primary data store, one or more producers capture snapshots (states of the data sets at particular points in time) and upload the snapshots to an intermediate data store for access by snapshot consumers. The snapshot consumers may download the snapshots to generate local versions of the data sets for access by one or more data processing applications or processes. A snapshot producer may periodically generate full snapshots of a data set, and may generate one or more incremental snapshots of the data set between full snapshots. A snapshot consumer may bootstrap a local data set from a full snapshot and one or more incrementals, and may maintain state of the local data set by accessing new snapshots uploaded by the producer.

Abstract: The exemplary embodiments described herein are related to techniques for automatically generating narratives about data based on communication goal data structures that are associated with configurable content blocks. The use of such communication goal data structures facilitates modes of operation whereby narratives can be generated in real-time and/or interactive manners.

Abstract: Data refresh in a replication environment is provided by: reading, by a data services component of a replication environment as part of a data refresh operation, a data store and identifying refresh data, from a source object of source objects of the data store, that is to be provided to a target system, where the data services component is further configured to read a replication log into which changes to source object(s) of the source objects are recorded, and send the changes as change data records to a capture service; and retrieving, by the data services component, the refresh data from the source object and sending the refresh data as refresh data records to the capture service, where the capture service is configured with data record handling routines for applying to received change data records and to the received refresh data records.

Abstract: Embodiments of the present invention assist in the development, management, and deployment of aggregated data attributes for multiple data sources. One embodiment provides a development interface that allows for elements of attributes, including filters, to be moved into a coding area in which an attribute or an attribute element is being edited. In another embodiment, the user interface presents data fields to assist in the development of filters for multiple data sources with divergent formats. The application further provides a validation interface through which users can validate attributes and trace the results returned by various elements referenced by the attributes under validation. Another embodiment provides a system for managing attributes and deploying them to various systems by creating a deployment file that is used by an attribute calculation system. In one embodiment, the attribute calculation system is a scalable system that dynamically calculates attributes for multiple data sources.

Abstract: Peer storage systems share the workload of asynchronously replicating a shared logical storage unit (LSU) to a target system. Peer storage systems (S1 and S2) that synchronously replicate a shared LSU share the workload of asynchronously replicating the shared LSU to a target system (S3) by dividing ownership of the replication cycles in an alternating manner without a strict synchronization of cycles between the peer storage systems. Rather, a given cycle number (e.g., Cycle 1) on S1 may not start and end with a same write operation as a same cycle number (Cycle 1) on S2, such that cycles on S1 (e.g., Cycle 1 and Cycle 2) overlap with cycles having the same number (Cycle 1 and Cycle 2) on S2. S1 may asynchronously replicate to S3 only the cycles it owns, and S2 may asynchronously replicate to S3 only the cycles it owns to S3.

Abstract: The data of data objects stored in a DDOS are converted from one data format to another. The conversion is performed while the system is live and available to process requests for access to the data objects being converted. The conversion techniques do not require taking the DDOS offline. The conversion techniques also do not require locking a data object for the entire conversion of the data object.

Abstract: A computer-implemented method for seamlessly performing a maintenance operation on a stateful system includes mapping a network address of the stateful system to a primary server that uses a primary database to respond to incoming data requests. In response to receiving a maintenance request, the primary database is replicated to a secondary database of a secondary server. The secondary server is updated according to the maintenance request. The method further includes caching, in a replay buffer of the primary server, incoming data requests during the replicating. After the replicating, the data requests from the replay buffer are executed by the secondary server. Write operations to the primary server are disabled during the replicating, and the network address of the stateful system is mapped to the secondary server. Subsequently, the primary server is updated and reinstated by mapping the network address, and enabling the write operations.

Abstract: A method, system, and computer readable medium for asynchronously processing write operation on a volume having copy-on-write snapshots. In one embodiment, the method comprises the steps of: updating a normal mirror with write data associated with a write operation for the volume; asynchronously copying the write data from an asynchronous mirror to at least one copy-on-write snapshot; and, once the at least one copy-on-write snapshot is updated, updating the asynchronous mirror with the write data from the normal mirror.

Abstract: Described are techniques executed in a data storage system in connection with processing an I/O request. The I/O request is received. It is determined whether the I/O request is a write request. If the I/O request is a write request, write request processing is performed. The write request processing includes: copying write request data of the write request to cache; destaging the write request data from the cache to a primary storage device; and copying, in accordance with a heuristic, the write request data from the primary storage device to an asynchronous mirror device including an asynchronous copy of data from the primary storage device, wherein the asynchronous mirror device is write disabled with respect to received write requests requesting to write data thereto, the asynchronous mirror device used for servicing data requested in connection with read requests upon the occurrence of a cache miss.

Abstract: A method, system, and computer readable medium for asynchronously processing write operation on a volume having copy-on-write snapshots. In one embodiment, the method comprises the steps of: updating a normal mirror with write data associated with a write operation for the volume; asynchronously copying the write data from an asynchronous mirror to at least one copy-on-write snapshot; and, once the at least one copy-on-write snapshot is updated, updating the asynchronous mirror with the write data from the normal mirror.

Abstract: Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.