Abstract: Provided herein are systems and methods for network intrusion event response and remediation. A file encryption key is generated based on a first private key of a user and a root key associated with a primary deployment of a database system. File data is encoded at a source account of the user using the file encryption key. The file data is replicated from the source account into a target account of the user to generate replicated file data. Notification of a detected network intrusion event is encoded for transmission to the user. A failover of the source account to the target account is performed based on the notification. The failover grants the user access to the replicated file data based at least on a second private key of the user.

Abstract: Proxy-based scaling may be performed for databases. A proxy may be implemented for a database that can establish a connection between the proxy and a database engine to perform a database queries received from a client at the proxy. A scaling event may be detected for the database responsive to which the proxy may establish a connection with a new database engine which may, in some embodiments, have different capabilities or resources that address the features or criteria that triggered the scaling event. Session state may be copied from the database engine to the new database engine so that the new database engine may be able to provide access to the database on behalf of requests received from the client through the proxy.

Abstract: In a Byzantine computing environment in which a database is sharded or partitioned among multiple clusters of computing nodes, consensus for and execution of data transactions (e.g., transactions that require and/or affect data of one or more shards) are achieved in a resilient manner Within some clusters, multiple primary replicas concurrently propose transactions for processing in parallel by all replicas. For some multi-shard transactions, shards involved in the transactions may be logically ring-ordered; each shard in turn achieves consensus among its nodes to commit the transactions, and then executes its portion of the operation after consensus is obtained among all shards. For some other multi-shard transactions, involved shards first determine whether local data constraints are satisfied, after which data modifications are made in parallel.

Abstract: An event trigger to flush key-value data from a volatile memory device is detected. In response to detection of the event trigger, a determination is made whether the key-value data comprises uncommitted data for one or more active database transactions. If the key-value data comprises uncommitted data for one or more active database transactions, the uncommitted data is moved to a hold aside buffer in the volatile memory device, and committed data in the key-value data is stored on a non-volatile memory device.

Abstract: A method is provided for performing transaction processing in a system. The system includes a plurality of nodes. Each node in the system includes a processor, an application executing in the processor, and a local queue. Each node executes multiple user transactions. Each transaction includes a BEGIN step, and one or more data manipulation language (DML) or data definition language (DDL) operations. In operation, two of the nodes are selected to receive the same user request to process the transaction. The processor at each of the two nodes executes in the application the same user request to process the transaction. Local indicia is created at each of the two nodes from the outcome of the transaction steps and operations of the transaction and is then exchanged with the other of the two nodes. The processor at each of the two nodes puts on their respective local queues the local indicia created at the same node as the processor, and the local indicia received from the other of the two nodes.

Abstract: The disclosed application relates to a tool by which a user may create a cloud workspace that includes a data memory space, as well as a tool for automatically identifying ad-hoc analyses on that data. The solution allows a user to connect to data sources using SQL or GUI tools, combine data from different data sources, prepare and clean the data, mine the data for insights, and move that data into downstream reporting tools for visualization. The system is linked to a code repository to allow data scientists to execute code from the code repository in trial data spaces, investigate that data, and prepare more in-depth analytics for downstream reporting tools.

Abstract: The disclosed techniques reduce a responsiveness time for a secondary node state of a database in switching from a second computing node to replace a first computing node acting in a primary node state, with both computing nodes performing the same database queries. The second node receives information regarding queries performed by the first node while in the primary state. In some embodiments, the second node retrieves, from a transaction log, log records detailing operations performed for database transactions. In some embodiments, the second node inserts, based on the log records, data records of the transactions into an in-memory cache of the second node that stores chains of database records from different transactions. Upon receiving sufficient information to switch to the primary state, the second node changes a mode of operation during failover making a committed transaction available for reads by subsequent database queries prior to record reordering.

Abstract: Transaction statements may be forwarded for a database transaction. A transaction begun at a read-only node may include a write statement. The read-only node may forward the write statement to the read-only node to perform as a transaction. The read-only node may update a read view for the transaction to include the write statement before performing a read statement submitted after the write statement at the read-only node.

Abstract: Methods and systems are described herein for synchronizing communication records in computer networks. For example, the methods and systems may determine whether or not a first communication relates to a second and generate a recommendation that the communications relate to a single communication. In particular, the methods and systems described herein describe synchronizing communication records in computer networks based on detecting patterns in categories of metadata. For example, the methods and systems retrieve specific types of metadata and compare this metadata between communications in order to synchronize and/or deduplicate them.

Abstract: In a Byzantine computing environment in which a database is sharded or partitioned among multiple clusters of computing nodes, consensus for and execution of data transactions (e.g., transactions that require and/or affect data of one or more shards) are achieved in a resilient manner. Within some clusters, multiple primary replicas concurrently propose transactions for processing in parallel by all replicas. For some multi-shard transactions, shards involved in the transactions may be logically ring-ordered; each shard in turn achieves consensus among its nodes to commit the transactions, and then executes its portion of the operation after consensus is obtained among all shards. For some other multi-shard transactions, involved shards first determine whether local data constraints are satisfied, after which data modifications are made in parallel.

Abstract: A transaction processing protocol for serverless database management systems can use a transaction scheduler to guarantee consistent serializable execution though analysis of the access pattern of transaction types and appropriate ordering of the transaction's events at runtime. A transaction topology is determined for each type of transaction and these are combined and used to generate a serialization graph. Cycles in the serialization graph are identified and breaking transaction types which may break the cycles are determined. When transaction requests are received, a breaking type of transaction is scheduled as a last transaction in the current epoch and later transactions not having the breaking transaction type are scheduled to execute in the next epoch.

Abstract: This application discloses a data replication method and apparatus, a computer device, and a storage medium, and belongs to the field of database technologies. The method includes: adding, when a commitment operation of a transaction is detected, historical state data of the transaction to a data queue; adding at least one piece of historical state data in the data queue to a transmission buffer; and replicating, when a first preset condition is met, the at least one piece of historical state data in the transmission buffer to a cluster device. In this application, a node device does not need to convert an original historical state data format into a log format, and the cluster device does not need to parse a log into an original data format for storage, thereby avoiding a cumbersome log replay procedure, and improving the efficiency of the data replication process.

Abstract: A method, apparatus, and computer program product for executing a relational database management system (RDBMS) in a computer system, wherein the RDBMS manages a relational database comprised of at least one column-partitioned base table storing data. Column values from at least one column of the column-partitioned base table are stored in one or more containers spread across one or more data blocks. Metadata comprising summarized information about the column values in the containers is stored in a metadata index subtable. A query with a filtering condition on the column is applied to the metadata index subtable before the column-partitioned base table is accessed, so that only qualified containers and data blocks are accessed, and unqualified containers and data blocks are eliminated, when responding to the query.

Abstract: Methods, computer program products, computer systems, and the like are disclosed that provide for scalable deduplication in an efficient and effective manner. For example, such methods, computer program products, and computer systems can include determining, at a source site, whether metadata has been received from a target site, and, in response to a determination that the metadata has been received at the source site, retrieving the at least one unit of the source data from the source data store using the metadata and sending, from the source site, the at least one unit of source data to the target site.

Abstract: Techniques are provided for utilizing a log to free pages from persistent memory. A log is maintained to comprise a list of page block numbers of pages within persistent memory of a node to free. A page block number, of a page, within the log is identified for processing. A reference count, corresponding to a number of references to the page block number, is identified. In response to the reference count being greater than 1, the reference count is decremented and the page block number is removed from the log. In response to the reference count being 1, the page is freed from the persistent memory and the page block number is removed from the log.

Abstract: A node includes storage that stores a portion of high accessibility data corresponding to a shard that is divided into revisions. The portion of the high accessibility data is managed based on an activity state model including a writing state, a resolving state, a metadata update state, and a reading state. The node further includes a repository manager that obtains a cross-shard read request for the high accessibility data; makes a determination that the portion of the high accessibility data is not in the reading state; in response to the determination, adds the cross-shard read request to a delayed action queue at the revision of a last delayed action in the delayed action queue; and after processing the last delayed action in the delayed action queue and while in the resolving state: converts the cross-shard read request to a read request; and processes the read request.

Abstract: Methods, systems, and computer-readable media for automatic archiving of data store log data are disclosed. One or more operation records in a log are selected for archival. The one or more operation records comprise data indicative of operations performed on one or more data objects of a data store. The one or more operation records are selected for archival prior to deletion from the log. The one or more operation records are replicated from the log to an archive. Based at least in part on the replicating, the one or more operation records in the log are marked as archived. Based at least in part on the marking as archived, the deletion of the one or more operation records from the log is permitted.

Abstract: Computing platforms, methods, and storage media for data movement are disclosed. Exemplary implementations may: obtain a data transfer command including a table-based characterization of a set of data to be transferred from an on-premise platform to a cloud platform, the data transfer command further including a date specification; automatically determine, based on the table-based characterization of the data to be transferred and based on the date specification, a set of files to be transferred; and initiate transfer of the set of files from the on-premise platform to the cloud platform. In an implementation, data is moved at the table level, and the files associated with the specified tables/date range are automatically determined. A single command, which may reference a CSV file, may be used to efficiently and reliably transfer a large amount of data without a user having to specify the specific files to be transferred.

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize a plurality of data validation processes to merge or replicate data across databases for downstream operations. For example, in response to a request to merge data from one or more source servers to one or more destination servers, a disclosed system determines database events to merge based on a plurality of database tables accessed in connection with a card account transaction. The disclosed system validates subsets of database events for the accessed database tables based on the quantities and event identifier sequencing of the database events. Additionally, the disclosed system replicates the database events from the source servers to the destination servers in response to validating the completeness and ordering of the database events for the card account transaction.

Abstract: The present disclosure involves systems, software, and computer implemented methods for aligned purpose disassociation in a multi-system landscape. One example method includes receiving, from multiple systems, a can-disassociate status for a purpose for an object instance. The status from a respective system can be an affirmative status that indicates that the system can disassociate the purpose from the instance or a negative status that indicates that the system cannot disassociate the purpose from the instance. The received statuses are evaluated to determine a central disassociate purpose decision for the purpose for the instance. The central disassociate purpose decision can be to disassociate the purpose from the instance when no system has the negative status and to not disassociate the purpose from the instance when at least one system has the negative status. The central disassociate purpose decision is provided to at least some of the multiple systems.

Abstract: An automated method is provided for use when replacing a currently operating data replication engine in a first system with a new data replication engine in the first system in a bidirectional data replication environment. The currently operating data replication engine in the first system and the new data replication engine in the first system replicates first database transactions from an audit trail of a first database in the first system to a second database in a second system. The new data replication engine in the first system generating a list of active database transactions in the first system, and sends the list of active database transactions to the new data replication engine in the second system as a first token.

Abstract: A computer implemented method and system for a selectively replicated trustless persistent store is provided using a bilateral distributed ledger. The selectively replicated trustless persistent store synchronizes current state data stores shared among multiple parties. Data modifications may be made in any shared store locally and then are automatically replicated across other permissioned stores. The selectively replicated trustless persistent store is responsible for getting the data validated and agreed upon before committing locally.

Abstract: The invention provides for a cloud-based solution that saves all the data in the cloud storage. The peer devices synchronize data among each other independent of the operating system since the data is synced via web services. Synchronization of data among peer devices is possible even when cloud service is unavailable via a router, Wi-Fi, Bluetooth, NFC or any other mechanism. The peer devices form a hierarchical structure, which designates a master, and the master communicates with the cloud-based service to synchronize data. The master then synchronizes data with the other peer devices in the hierarchy. New devices can be added to the peer devices and can join the hierarchy.

Abstract: Systems and devices can include a controller and a command queue to buffer incoming write requests into the device. The controller can receive, from a client across a link, a non-posted write request (e.g., a deferred memory write (DMWr) request) in a transaction layer packet (TLP) to the command queue; determine that the command queue can accept the DMWr request; identify, from the TLP, a successful completion (SC) message that indicates that the DMWr request was accepted into the command queue; and transmit, to the client across the link, the SC message that indicates that the DMWr request was accepted into the command queue. The controller can receive a second DMWr request in a second TLP; determine that the command queue is full; and transmit a memory request retry status (MRS) message to be transmitted to the client in response to the command queue being full.

Abstract: Transaction consistency query support is available for replicated data from recovery log to external data stores. An external data store is populated with records using entries of a change data table. The change data table has entries for each transaction that has committed and is to be replicated, and each of the entries stores information for each log entry in a recovery log from a database management system. Each log entry identifies a transactional change of data and a transaction completion indicator of one of commit and abort. In response to receiving a query about a transaction of the transactions, a set of records are retrieved from the external data store for the transaction. From the set of records, records whose sequence identifier values are larger than a maximum transaction commit sequence identifier are removed. From the set of records, remaining records having transaction consistency are returned.

Abstract: Transaction consistency query support is available for replicated data from recovery log to external data stores. An external data store is populated with records using entries of a change data table. The change data table has entries for each transaction that has committed and is to be replicated, and each of the entries stores information for each log entry in a recovery log from a database management system. Each log entry identifies a transactional change of data and a transaction completion indicator of one of commit and abort. In response to receiving a query about a transaction of the transactions, a set of records are retrieved from the external data store for the transaction. From the set of records, records whose sequence identifier values are larger than a maximum transaction commit sequence identifier are removed. From the set of records, remaining records having transaction consistency are returned.

Abstract: A method, system and computer program product for copying data from a source database to a target database. A first database engine of the source database assigns a data-change-ID to each changed row in the source table. The data rows of the source table are stored in a plurality of source pages. Each source page comprises a source-page-ID and a highest-data-change-ID indicating its most recently changed data row. The target table comprises a source-page-ID-column. A second database engine of the target database maintains a metadata table comprising a source-page-ID column and a highest-data-change-ID column. The highest-data-change-IDs in the metadata table are compared with the highest-data-change-IDs of the source pages for selectively copying source table rows of source pages whose highest-data-change-ID is higher than their highest-data-change-ID in the metadata table to the target table.

Abstract: Disclosed herein are system, method, and computer program product embodiments for utilizing look-ahead-staging (LAS) to guarantee the ability to rollback and reconstruct a package while minimizing locking duration and enabling multiple packages to be processed in a data pipeline simultaneously. An embodiment operates by receiving a package from a source system for processing through a data pipeline. The embodiment stores the package in a persistent storage together with a respective package status. The embodiment transmits the package to the data pipeline in response to the storing. The embodiment receives a commit notification for the package from a target system in response to the transmitting. The embodiment then removes the package from the persistent storage in response to receiving the commit notification for the package.

Abstract: Transaction recovery in a cloud computing environment. Transaction recovery includes determining whether a first container of a plurality of containers in the cloud computing environment is crashed. In response to the first container being crashed, the method may include determining at least one transaction log entry from a first log file corresponding to the first container. Transaction recovery includes retrieving at least one in-doubt transaction corresponding to the first container from a first database corresponding to the first container. Transaction recovery includes comparing the at least one transaction log entry with the at least one in-doubt transaction for transaction recovery.

Abstract: Disclosed embodiments are related to Master Data Management (MDM) technologies. Each DB entity (record) in component databases (DB) is represented as a judgment, and an MDM system unifies judgments obtained from the component DBs into a unified set of judgments. In the unified set of judgments, linkages are judgments asserting that particular DB entities from different DBs are the same/similar, and a golden record comprises field values describing each of the DB entities. In making judgments, the MDM system consolidates judgments (or records) describing the same subject into a single judgment, and performs updates to field values in a manner that does not violate referential integrity. Each update is associated with an MDM consistent state. Updates in the form of judgments are provided to the relevant component DBs, which are converted into serializable transactions associated with respective MDM consistent states. Other embodiments may be described and/or claimed.

Abstract: An augmented sleepable read-copy update implementation (PREEMPT_SRCU) combines elements of a tree-based sleepable read-copy update environment (Tree-SRCU) with elements of a preemptible read-copy update environment (Preemptible-RCU). The elements of Tree-SRCU may be used to manage PREEMPT_SRCU grace periods and handle PREEMPT_SRCU callbacks. The elements of Preemptible-RCU may be used to drive existing PREEMPT_SRCU grace periods to completion.

Abstract: The present disclosure provides a method and a device for data processing. The method includes acquiring at least two pathways of communication messages, where the at least two pathways of communication messages are messages intercepted in a bypass manner from messages transmitted by a service processing system to an external system; and the service processing system does not execute logic of record storage; processing the at least two pathways of communication messages, and determining communication messages to-be-stored from the at least two pathways of communication messages processed; and according to the at least two pathways of communication messages, storing the communication messages to-be-stored in a database.

Abstract: A method is provided for performing transaction processing in a system that includes a plurality of nodes. Each of the nodes include a processor, an application executing in the processor, and an instance of a database used by the application. Each node executes transactions. At least two of the nodes are selected to receive the same request to process a transaction, and the processor at each of the at least two nodes executes the same request to process a transaction. Each of the at least two nodes replicates the steps and operations of the transactions to one or more other nodes in the system. The processor at the one or more other nodes compares the replicated steps and operations of the transactions that were received from each of the at least two nodes and executes one or more remediation actions when the result of the comparison indicates that the replicated steps and operations of the replicated transactions do not match each other.

Abstract: A system includes a multi-layer capacity configuration optimization (CCO) stack to generate a token containing prescriptions for optimize capacity configuration of a database container in a NoSQL database cloud service. The system may aggregate the capacity utilization data; predict, based on the aggregated capacity utilization data, respective prediction-based processing capacity utilizations for the database container; determine a target processing capacity utilization value from the prediction-based processing capacity utilizations; calculate respective provisioned processing capacity utilizations based on the target processing capacity utilization value; evaluate a consumption metric based on the prediction-based processing capacity utilizations and the provisioned processing capacity utilizations; select one of the predetermined capacity modes as a recommended capacity mode for the database container based on the consumption metric.

Abstract: An embodiment includes extracting, after a first specified time interval, recovery data from a recovery log for a data table corresponding to a set of database operations on the data table that occurred during the first specified time interval. The embodiment also generates index data for a progress record associated with the first time interval. The progress record includes a first data field for storing data representative of a first net change to the data table resulting from a first subset of the set of database operations. The first subset of database operations are associated with a first transaction. The embodiment also detects a commit operation in the first subset of database operations that committed the first transaction. The embodiment also recovers the first transaction in a target data table using the recovery log and the first net change data associated with the first transaction in the progress record.

Abstract: Offline building of a projected data subset may be performed. A request to create a data set that is a projected subset of data from a source data set may be received. A data store separate from the data store storing the source data set may store a copy of the source data set that is used to replicate items to the projected subset of data according to a schema for the projected data subset. Updates made to the source data set may also be replicated to the projected data subset according to the schema. Conflicts between replicated items and replicated updates to the projected data set may be resolved by comparing a version identifier for the replicated update and replicated item to determine what to store in the projected data subset.

Abstract: A streaming distributed decentralized database task system can perform multiple tasks on clusters of nodes without overloading the clusters' computational resources, such as disk, memory, processors, and network bandwidth. A cluster master can manage a job and add items to node queues. A node manager accepts or rejects queue items based on streaming task limits that are applied at the node level.

Abstract: Systems, methods, and software for management of partitioned data storage spaces is provided herein. An exemplary method includes storing sets of structured data records among partitioned data storage spaces, with data fields of the structured data records correlated among the sets by relational associations. The method includes, in a data center that receives change actions related to the structured data records, selectively placing the change actions into a plurality of change feeds, where the change feeds can be commutatively executed in parallel to implement the change actions. The method further includes implementing the change actions and propagating replication data comprising the change actions in the plurality of change feeds.

Abstract: Data is replicated between primary and secondary storage systems using a data pulling process or a log shipping process. If data is to be replicated through the log shipping process, a transaction and its associated data get stored in a transaction log in the secondary storage system. Subsequently, when the transaction gets executed, actual data gets persisted from the transaction log to a data log in the secondary storage system. If the data is to be replicated through the data pulling process, the transaction is stored in the transaction log. However, the associated data for the transaction is retrieved from the first primary storage system and stored directly in the data log in the secondary system.

Abstract: Disclosed is a computer program stored in a computer readable storage medium, which includes encoded commands, wherein executing the computer program by one or more processors of a computer system allows the one or more processors to perform steps for change data capture (CDC) between a source database and a target database.

Abstract: A system synchronizes change-data in a multi-tenant system with one or more external service provider systems.

Abstract: Systems, methods, and devices for storing database data in journal tables comprising a snapshot and a log table. A method includes defining a journal table comprising a snapshot and a log table, the snapshot comprising an up-to-date representation of data in the journal table at a point in time. The method includes assigning a timestamp to the snapshot indicating when the snapshot was generated. The method includes receiving a request to execute a transaction on the journal table to modify the data in the journal table, the transaction comprising one or more of an insert, a delete, an update, or a merge. The method includes inserting a new row into the log table in lieu of executing the transaction on the snapshot of the journal table, the new row comprising an indication of a change requested to be made to the journal table based on the transaction.

Abstract: A computer implemented method and system for a selectively replicated trustless persistent store is provided using a bilateral distributed ledger. The selectively replicated trustless persistent store synchronizes current state data stores shared among multiple parties. Data modifications may be made in any shared store locally and then are automatically replicated across other permissioned stores. The selectively replicated trustless persistent store is responsible for getting the data validated and agreed upon before committing locally.

Abstract: A streaming distributed decentralized database task system can perform multiple tasks on clusters of nodes without overloading the clusters' computational resources, such as disk, memory, processors, and network bandwidth. A cluster master can manage a job and add items to node queues. A node manager accepts or rejects queue items based on streaming task limits that are applied at the node level.

Abstract: Technologies are described for performing synchronization within a database environment. A source host stores multiple database tables and a replica host stores copies of the multiple database tables. During synchronization, replication log replay is blocked at the replica node, and a multithreaded process locks the multiple database tables of the source hosts, reactivates replication log generation for each of the multiple database tables, and then unlocks the database tables. A synchronization timestamp is acquired and used to synchronize the multiple database tables with the copies of the multiple database tables. After synchronization, replication log replay is reactivated at the replica host for the copies of the multiple database tables.

Abstract: A database session in an active standby server on which an active standby database resides receives a DML statement. The session is suspended while the statement is redirected over a database link to a primary database on which the statement is executed. Information associated with execution of the statement is communicated to the session in the active standby server. Redo records describing changes to the contents of the primary database are applied to the active standby database and control is returned to the session. Prior to commitment of a transaction including the statement, a query directed to data to which the statement was directed is received at the active standby server from a client and executed on the active standby database absent use of a database link based on whether information associated with a database session associated with the client matches the information associated with execution of the statement.

Abstract: Disclosed herein are system, method, and computer program product embodiments for a database recovery and optimization with batch processing system. An embodiment operates by retrieving a database log that includes a plurality of operations for modifying data of a database stored across a plurality of tables. From the database log, a plurality of consecutive insert operations for inserting data into the database are identified. The consecutive insert operations are sorted by table. The sorted insert operations are grouped into a batch message. The batch message is transmit to the database for replay. An acknowledgement is received that the replay has completed.

Abstract: Optimized location information is provided for a location based service. An event is detected which is indicative of an error in the location based service which degrades the performance of the location based service. A cause of the error is determined, and resolution of the error is initiated based on the determined cause.

Abstract: In accordance with embodiments, there are provided techniques for implementing batch processing in a multi-tenant database system. These techniques for implementing batch processing in a multi-tenant database system may enable embodiments to provide great flexibility to a tenant of the architecture to perform desired functions on content of the database while allowing the owner of the architecture schedule the processing of the functions on the content.

Abstract: A request to store a new value to a data entity to be fault tolerant across a plurality of nodes is received. The data entity being in an intermediate state is indicating to at least one less than a majority of the plurality of nodes. A request to read a stored value of the data entity is received. A latest value instance of the data entity among a majority nodes of the plurality of nodes is determined. A value of the latest value instance in a majority nodes of the plurality of nodes is stored as a next value instance of the data entity.