Abstract: The present disclosure involves systems, software, and computer implemented methods for efficiently authorizing parameterized query views. An example method includes parsing a received query to generate a global query parse tree. In response to determining that the query includes a parameterized query view, the parameterized query view is parsed to generate a view parse tree which is then attached to the global query parse tree. In response to determining that an object in the global query parse tree is a parameterized query view, a view parse tree portion of the global query parse tree is traversed to identify objects associated with the parameterized query view. The parameterized query view and the identified objects are authorized in a single authorization step. For objects in the global query parse tree that are not parameterized query views, the object is authorized. In response to all objects being authorized, the query is executed.

Abstract: Computer-implemented systems and methods for synchronizing data for dataset execution. The system includes a source database that stores a canonical dataset, a secondary database that stores a processed dataset, and a synchronization server that comprises a processor and a memory. The processor is configured to monitor for a publication of one or more source tables and when the publication is detected, identify the processed tables, corresponding to the source tables, to be updated in the processed dataset. The processor determines a tolerance level corresponding to each processed table and updates the processed tables in the processed dataset. The processor determines whether the processed tables in the processed dataset were successfully updated within the tolerance levels and transmits a notification based on determining whether the processed tables in the processed dataset were successfully updated within the tolerance levels.

Abstract: The present disclosure provides a data reading method, including: in response to receiving a read response request generated by a replication group for an application and sent by a storage terminal, setting, in a dedicated mapping table corresponding to the application, a commit index corresponding to the replication group as a commit index carried by the read response request; searching for a target replication group corresponding to a first read request generated by a target application based on the first read request; determining, in a dedicated mapping table corresponding to the target application, a commit index corresponding to the target replication group as a target commit index; sending a second read request carrying the target commit index to the storage terminal; and obtaining the data read by the storage terminal.

Abstract: A data dictionary generation system automatically populates and updates a data dictionary for listings offering shared data. The data listing distribution component distributes the data dictionaries to various remote deployments in a data exchange by using a global messaging framework and replication method. For example, the data listing distribution component replicates a data dictionary generated for the listing and its shared data from a source deployment to one or more destination deployments associated with various geographic regions. The data listing distribution component distributes the listing to the various remote deployments to allow for the listing, including its shared data and data dictionary, to be accessed by users within the geographic region associated with the remote deployment.

Abstract: One example method includes by a data source entity, querying a data storage entity to determine if it has adequate under-utilized resources to perform a portion of a data seeding process. When the data storage entity indicates to the data source entity that the data storage entity has adequate unused resources to perform the portion of the data seeding process, initiating, by the data source entity, the data seeding process, and the data seeding process includes transmitting data from the data source entity to the data storage entity for storage in a temporary file at the data storage entity, and the data that is transmitted is data that was targeted to be protected by a data protection process. Finally, the data protection process may be performed so that only data not already present at the data storage entity is transmitted by the data source entity to the data storage entity.

Abstract: The present disclosure is directed to a centralized control policy for multicast replicator selection.

Abstract: A first or second computer viewing a virtual canvas may be a host computer for a live source object requiring authorization between a host and a non-host computer by creating a live source window for the live source object requiring authorization (ARLSW) within the virtual canvas on the host computer, sending metadata information for the ARLSW to the non-host computer so that the ARLSW is at the same location on the virtual canvas for the first and second computers, and streaming the live source object requiring authorization to the ARLSW from the host computer to the non-host computer. When the host computer for the ARLSW leaves the virtual canvas and no other non-host computer for ARLSW in the canvas is known to have authorization to view the ARLSW, a placeholder screen is displayed in the ARLSW on the virtual canvas.

Abstract: Systems and methods are provided for conducting continuous changelog-based replication of modifications made to a file system to a target storage system. An export system can monitor a changelog on a file system to detect changes made to the file system. The exporter system can further filter and reformat the changes for replication to the target storage system by storing the changes in a pending changes data store. To reduce duplicative traffic, the export system can store changes for a configurable delay period, writing all pending changes to a file system object after the delay while aggregating and deduplicating the changes, thus improving performance relative to immediate replication.

Abstract: Example distributed storage systems, replication managers, and methods provide replication barriers for dependent data transfers between data stores. An object data store may include a barrier object and be configured to identify dependencies between a dependency set of data objects and the barrier object. When replicating data objects to another data store, the dependency set of data objects may be transferred first, delaying the transfer of the barrier object while the dependency set is being transferred.

Abstract: A system for a decentralized cloud-storage solution operated within a P2P network. The decentralized cloud-storage solution includes a system and specific operations that not only performs data-classification analysis on an end-user's data, but also enhances a data-distribution mechanism with route distances related to geographical factors and facilitates a block-chained technology implemented for decentralized cloud storages; thereby reducing the risks on data storage and facilitating a user's big data transactions through a shared P2P network.

Abstract: In a method used for maximizing performance of a storage system, saturation points of the storage system for I/O requests of different types and sizes are identified. Normalized tokens are determined based on the saturation points. Unique numbers of normalized tokens are associated with the I/O requests of different types and sizes. A number of normalized tokens for a queue depth limit of a storage device is determined. From the queue depth limit, normalized tokens are allocated to each node in the storage device. I/O requests are processed according to the allocations of normalized tokens.

Abstract: A system can determine to perform a replication of data from first computing equipment to second computing equipment, the data being stored as part of a path in a file system. The system can determine that a change log of file operations on files in the path identifies a file operation on a file that is made by a user account that has priority for replications. The system can replicate the file. The system can, after processing the change log for file operations made by first user accounts that have priority for replications, replicating other files in the path associated with second user accounts that do not have priority for replications.

Abstract: A method of resolving an identity of an entity is disclosed. Entity data is ingested. Identity fragments observed in the entity data are recorded. All or a subset of identifiers that are connected across the identity fragments are discovered. A directed graph of the identity fragments is built. An entity profile is produced by applying a set of rules to the directed graph.

Abstract: Systems and methods are provided to implement a fast recovery process in a partitioned replicated data store. In some embodiments, the data store is configured to store an object in a plurality of partitions and replicate data in each partition in a group of replica nodes to satisfy a durability model. In response to a replica failure, the data store performs a split operation to create a plurality of new partitions. The partition's data is split into subsets corresponding to the new partitions. The subsets are transmitted, in parallel, from the surviving replica nodes of the partition to new replica nodes in the new partitions. The new partitions then replicate respective subsets of data in their respective replication groups using a chained replication technique. The recovery process allows the data store to return into compliance with the durability model more quickly, by parallelizing the copying of data.

Abstract: Methods and apparatuses are described for automated latency-aware purging of database records. A server determines a synchronization latency between a storage database and an online transaction processing (OLTP) database. When the synchronization latency is below a predetermined threshold, the server identifies a first database object in the OLTP database and selects database records to be purged from the first database object based upon a data purge instruction set. The server identifies a second database object in the storage database that corresponds to the first database object and selects database records in the second database object based upon the data purge instruction set. The server purges the database records from the first database object in the OLTP database when a characteristic of the database records to be purged from the first database object matches a characteristic of the database records selected in the second database object.

Abstract: Application-level data in a storage system are evaluated. For example, a backup analysis tool retrieves a backup object stored in the storage system. The backup analysis tool reconstructs an application object from the backup object. The backup analysis tool accesses the first application object according to a native application format associated with the first object.

Abstract: Systems, methods, and computer program products for testing new software are provided. Multiple payloads that correspond to scenarios in a production computing environment are identified. From the multiple payloads unique payloads are identified. User data that corresponds to the unique payloads is created. A first testing environment conducts a test using software components in the production environment, the unique payloads, and the user data to generate expected results. A second testing environment conducts a test using new software that replaces at least one of the software components in the production environment, the unique payloads, and the user data, to generate actual results. The one or more attributes in the expected results are compared to the one or more attributes in the actual results to determine if the new software causes an error.

Abstract: Disclosed is a data synchronization method performed by a computer device, the method including: receiving a piece of transaction data, the piece of transaction data carrying an update operation identifier and a predefined phrase corresponding to the update operation identifier; obtaining, in a case that a target block including the piece of transaction data is stored into a blockchain of a blockchain system, the update operation identifier and the predefined phrase in the target block; and updating a predefined phrase database according to the update operation identifier and the predefined phrase in the target block.

Abstract: In an implementation, a method includes causing a source application data object to be embedded in a host application file and recording information associated with the source application data object in a common data store. It further includes identifying a modification made to the embedded source application data object, sending a notification to the source application to update a corresponding source copy of the source application data object based on the modification made to the embedded source application data object, and recording the modification in the common data store. The method also includes identifying a change made to the source copy of the source application data object, and causing the host application to update the embedded source application data object in the host application data store according to access permissions of the host application file, and recording the change in the common data store.

Abstract: Systems and methods are provided for adaptive data replication. A first data object is stored. One or more first network characteristics of a communication network are detected. A first replication mode is triggered (e.g., a single-master replication mode) based on the one or more first network characteristics of the communication network. The single-master replication may be configured to facilitate real-time synchronizations (e.g., for critical updates). One or more second network characteristics of the communication network are detected. A second replication mode (e.g., a multi-master replication mode) is triggered based on the one or more second network characteristics. The second replication may be configured to facilitate a delayed synchronization (e.g., for non-critical updates) of the first data object and the second data object using the second update file.

Abstract: Graphics processors for implementing multi-tile memory management are disclosed. In one embodiment, a graphics processor includes a first graphics device having a local memory, a second graphics device having a local memory, and a graphics driver to provide a single virtual allocation with a common virtual address range to mirror a resource to each local memory of the first and second graphics devices.

Abstract: A technique provides efficient data failover by creation and deployment of a protection policy that ensures maintenance of frequent common snapshots between sites of a multi-site data replication environment. A global constraint optimizer executes on a node of a cluster to create the protection policy for deployment among other nodes of clusters at the sites. Constraints such as protection rules (PRs) specifying, e.g., an amount of tolerable data loss are applied to a category of data designated for failover from a primary site over a network to a plurality of (secondary and tertiary) sites typically located at geographically separated distances. The optimizer processes the PRs to compute parameters such as frequency of snapshot generation and replication among the sites, as well as retention of the latest common snapshot maintained at each site to create a recovery point and configuration of the protection policy that reduces network traffic for efficient use of the network among the sites.

Abstract: Systems and methods for geographically distributed node replication include a first node which receives a message from a client based on a proximity of the first node to the client, the message transmitted to the first node via anycast routing from the client. The first node may replicate the message to a first subset of the geographically distributed system of nodes based on a geographic proximity of nodes within the first subset. The first node may publish the message to a data feed of a message bus for the system of nodes, to cause at least one node of the system of nodes to receive the message from the first node, the at least one node outside the first subset and subscribing to the data feed of the first node.

Abstract: In response to receiving a request for an identity key from a first entity, an identity key for the first entity is generated. A first request from the first entity to replicate a set of data is received. The generated identity key for the first entity is added to the metadata of the set of data requested to be replicated. A determination is made whether a replication rule exists for the first entity. In response to determining that a replication rule exists for the first entity, the set of data is replicated according to the replication rule for the first entity.

Abstract: Various embodiments for a data management system for managing inferences are described herein. An embodiment operates by detecting a signal to perform a replication task. Parameters corresponding to the replication task are identified. The replication task is configured to at least partially replicate a source data table from a source repository to a target data table in a target repository, based on the parameters. It is determined that a columnar structure of a first column of the source data table including data to be replicated matches a columnar structure of a second column of the target data table where the data is to be replicated to. The replication task is executed responsive to the determination of the matching structures, the replication task including performing one or more arithmetic operations on the data.

Abstract: Provided methods and systems allow dynamic rendering of a reflexive questionnaire based on a modifiable spreadsheet for users with little to no programming experience and knowledge. Some methods comprise receiving a modifiable spreadsheet with multiple rows, each row comprising rendering instructions for a reflexive questionnaire from a first computer, such as a data type cell, statement cell, logic cell, and a field identifier; rendering a graphical user interface, on a second computer, comprising a label and an input element corresponding to the rendering instructions of a first row of the spreadsheet; receiving an input from the second computer; evaluating the input against the logic cell of the spreadsheet; in response to the input complying with the logic cell of the spreadsheet, dynamically rendering a second label and a second input element to be displayed on the graphical user interface based on the logic of the first row.

Abstract: Disclosed herein are various embodiments a code simplification system. An embodiment operates by determining that a software version of a coordinator node is different from a software version of one or more worker nodes, Commits by the one or more worker nodes to a database are disabled based on the determination that the software versions differ. An update is performed on each of the one or more worker nodes. An acknowledgement that the update on each of the one or more worker nodes has completed is received, and the commits to the database by the one or more worker nodes is enabled.

Abstract: Techniques are provided for timestamp consistency. An operation targeting a first storage object having a synchronous replication relationship with a second storage object is intercepted. A timestamp is assigned to the operation. A replication operation is created as a replication of the operation. The same timestamp is assigned to the replication operation. The operation is implemented upon the first storage object and the replication operation is implemented upon the second storage object.

Abstract: In one embodiment, a method comprises maintaining state information regarding a data replication status for a storage object of the storage node of a primary storage cluster with the storage object being replicated to a replicated storage object of a secondary storage cluster, temporarily disallowing input/output (I/O) operations when the storage object has a connection loss or failure. The method further includes initiating a resynchronization between the storage object and the replicated storage object including initiating asynchronous persistent inflight tracking and replay of any missing I/O operations that are missing from one of a first Op log of the primary storage cluster and a second Op log of the secondary storage cluster, and allowing new I/O operations to be handled with the storage object of the primary storage cluster without waiting for completion of the asynchronous persistent inflight tracking and replay at the secondary storage cluster.

Abstract: A file data access method and apparatus, and a medium. The method includes: when a file copy instruction is received, creating a backup file; recording, in metadata of an original file, identification information of the backup file, and recording, in metadata of the backup file, identification information of the original file; when a read instruction for a first target file is received, determining whether there is a data object corresponding to the first target file; if YES, reading the data object; and if NO, determining a first target original file associated with the first target file according to identification information recorded in metadata of the first target file, and reading a data object of the first target original file. During data backup, an original file may be backed up by merely recording, in metadata, identification information of an associated file, without copying a data object.

Abstract: With multiple software applications, multiple electronic devices for a user to access and work upon, and multiple potential storage locations for the files it is difficult for a user to remember the identity of a file worked upon, its storage location, or which application they employed. Whilst some applications provide a user with a listing of recently opened files these are only those accessed by that specific application upon a specific device. Embodiments of the invention provide users with a synthesized centralized view of all files accessed irrespective of where the user has stored the electronic content, with which electronic device the electronic content was accessed with, or with which software application the electronic content was accessed with.

Abstract: With multiple software applications, multiple electronic devices for a user to access and work upon, and multiple potential storage locations for the files it is difficult for a user to remember the identity of a file worked upon, its storage location, or which application they employed. Whilst some applications provide a user with a listing of recently opened files these are only those accessed by that specific application upon a specific device. Embodiments of the invention provide users with a synthesized centralized view of all files accessed irrespective of where the user has stored the electronic content, with which electronic device the electronic content was accessed with, or with which software application the electronic content was accessed with.

Abstract: Disclosed in embodiments of the present disclosure are methods and apparatuses for realizing a globally unique index, which are applicable to a distributed database. One of the methods includes: receiving a structured query language, SQL, request, obtaining a conflict detection SQL statement corresponding to the SQL request and calculating an expected value; issuing the conflict detection SQL statement to each database node for execution; obtaining an execution result of the conflict detection SQL statement, and comparing the execution result of the conflict detection SQL statement with the expected value; and when the execution result of the conflict detection SQL statement is less than or equal to the expected value, processing an original SQL statement of the SQL request, and issuing the original SQL statement to the each database node for execution.

Abstract: Techniques for providing horizontally scaled caching of versioned data are provided. In some aspects, the techniques described herein relate to a method including initializing a first version cache (VC) object based on a version of data stored in a data storage device; replicating the first VC to generate a second VC; receiving a write operation at the first VC; generating a delta for the write operation, the delta representing a change in the version of data; writing the delta to a persistent replication log, the persistent replication log storing an ordered set of deltas including the delta; writing data in the write operation to the data storage device; and applying the ordered set of deltas at the second VC to update data stored by the second VC.

Abstract: A method may include: receiving a notification of a triggering event, the triggering event identifying an associated product; accessing a baseline configuration for the product; transmitting a notification to a recipient responsible for maintaining the baseline configuration for the product; receiving an approved update to the baseline configuration from the recipient; generating an updated baseline configuration for the product based on the approved update; querying a database to identify an application subscriber for the triggering event for the product; and transmitting the updated baseline configuration to the identified application subscriber.

Abstract: Systems and methods for moving, reconciling, and aggregating data from mainframe computers to hybrid cloud are disclosed.

Abstract: A method for validating a query result for a query of a database uses an index of the database. A selection of a set of source data from the database is received and a first hash operation is performed on the source data in the database resulting in a database hash value for the source data. A second hash operation is performed on the source data in the index resulting in an index hash value. The index hash value is compared with the database hash value, and a guarantee indication is provided for the source data in the index.

Abstract: Database synchronization by initiating a transaction to synchronize data from a source database to a target database. The source database issues a trigger that provides a sequence identification (ID) and calculates a thread number that is employed for syncing the rows from the source database to the target database based on a primary key hash. The sequence identification (ID) and the thread number from the source database is inserted into a synchronization (SYNC) table. Data synchronization may also include picking the rows that are allotted their thread number from the SYNC table using polling threads. The rows from the SYNC table may be replicated into target database using a sequence provided by the sequence identification and thread number.

Abstract: The present disclosure describes storage techniques for hybrid transactional and analytical processing. Data captured by a first processing engine may be received. The first processing engine may be configured to perform online transactional processing). Multiple replicas of logical logs generated based on the data may be distributed to a Delta Store by applying a quorum protocol on the multiple replicas. Data in the Delta Store are stored in a row format and are visible to a query for online analytical processing performed by a second processing engine. Data may be flushed from the Delta Store to a Base Store based on one or more predetermined rules. Data in the Base Store are stored in a columnar format and may be accessible by the second processing engine.

Abstract: The disclosed technology includes systems and methods for automatically generating a dynamic system context diagram based on machine-readable code. A method can include receiving, at a rules engine, machine-readable code describing interactions among a plurality of applications in software architecture, evaluating, with the rules engine in communication with a system of record (SoR), compliance of the interactions among the plurality of applications according to the SoR, identifying, with the rules engine, and based on compliance evaluation, one or more dependencies among the plurality of applications, generating, with an output engine, a system context diagram image comprising a graphical representation of the plurality of applications with associated interactions and dependencies, and outputting, for display, the system context diagram image.

Abstract: Replicas may be added to a multi-leader replica group for a data set. A new replica may be added to a group of replicas by copying a version of the data set according to a point in time to the new replica. As updates may occur at different ones of the replicas when the new replica is added, updates that occurred at a source replica for the snapshot and other replicas may be replicated by the different replicas according to the point in time at which the snapshot is created without blocking updates at any of the replicas in the replica group.

Abstract: An incremental backup is made of a source volume on a production host to on premise secondary storage. The incremental backup includes a bitmap specifying locations of blocks that have changed since a last backup of the source volume, and data of the changed blocks. First checksums are calculated from data of the changed blocks. The incremental backup, including the changed blocks and the bitmap, are moved to cloud storage. The changed blocks are merged, at the cloud storage, with the last backup of the source volume to generate a synthetic full backup of the source volume. The bitmap is consulted to identify locations of the changed blocks. Second checksums are calculated from data written to the locations on the synthetic full backup. The first and second checksums are compared. If the first and second checksums do not match, an indication is generated that the synthetic full backup is corrupt.

Abstract: Techniques are provided for neutralizing replication errors. An operation is executed upon a first storage object and is replicated as a replicated operation for execution upon a second storage object. A first error may be received for the replicated operation. Instead of transitioning to an out of sync state and aborting the operation, a wait is performed until a result of the attempted execution of the operation is received. If the first error is the same as a second error returned for the operation, then the operation and replicated operation are considered successful and a synchronous replication relationship is kept in sync. If the first error and the second error are different errors, then an error response is returned for the operation and the synchronous replication relationship is transitioned to out of sync.

Abstract: A data dictionary generation system automatically populates and updates a data dictionary for listings offering shared data. The data listing distribution component distributes the data dictionaries to various remote deployments in a data exchange by using a global messaging framework and replication method. For example, the data listing distribution component replicates a data dictionary generated for the listing and its shared data from a source deployment to one or more destination deployments associated with various geographic regions. The data listing distribution component distributes the listing to the various remote deployments to allow for the listing, including its shared data and data dictionary, to be accessed by users within the geographic region associated with the remote deployment.

Abstract: A method, computer system, and a computer program product for managing a multi-vendor environment is provided. The present invention may include receiving a request to make a change to an aggregate computer system from a participant of the aggregate computer system. The present invention may include creating, in response to the request, a change block. The present invention may include receiving approval for the change block in accordance with an endorsement policy. The present invention may include adding the change block to a distributed ledger. The present invention may include monitoring a performance of the aggregate computer system.

Abstract: In one aspect, a method includes forming, at a database platform, a file stage in a user account on the database platform, the user account being associated with a client, forming, at the database platform, a hidden stage directory table and a data pipeline for the file stage, and binding the data pipeline to the file stage. The method also includes receiving a notification of an event of the file stage. The method also includes identifying the data pipeline corresponding to the file stage. The method also includes updating the hidden stage directory table of the identified data pipeline.

Abstract: Direct storage loading may be used to add data to a database. New data may be added to a database, using nodes different than a database engine to access a database. The addition of the new data may be assigned to different nodes. The nodes may obtain the data and store the data to storage locations according allocated space in the database by the database engine. The new data can then be made available for access at the database engine.

Abstract: A data replication method includes obtaining differential data information corresponding to differential data, where the differential data information includes a storage address of the differential data, and a determining value of the differential data, replicating the differential data from the primary volume to the secondary volume according to the storage address of the differential data that is located in the primary volume when the determining value is not less than a preset threshold, and taking a snapshot for the primary volume when the determining value is less than the preset threshold and replicating the differential data to the secondary volume.

Abstract: Methods and systems disclosed herein allow data to be transferred from a data source to a target database with little to no offline period or data corruptions. The methods and systems describe a server that generates a temporary data repository having a similar configuration as the target data repository; transmits the set of new data records from the data source to the temporary data repository; identifies dependency relationship attributes among the data records stored within the target data repository; and when the server identifies that a predetermined number of data records and their respective dependent data records are stored within the temporary data records, the server merges the set of data records and the set of new data records. The server also stores a pre/post merger record of data such that the server can revert to a previous version of data or roll forward to another version.

Abstract: Described embodiments provide systems and methods of providing accessing to a file. A client device may store a copy of a file. The copy may be generated during active use of the file in which content of the file is accessible by the client device via an application hosted on a remote computing device. Responsive to a loss in connectivity that disrupts the active use of the file, the client device may identify a local application on the client device based at least on a type of the file. The client device may use the identified local application to access the copy of the file stored on the client device and continue the active use of content of the file. Subsequent to restoring the connectivity, the client device may provide the file to the remote computing device for synchronization of the file.