Abstract: A technique for retaining a context in which data resides independently of a data store from which the data originates is disclosed. In relation to a method aspect of the technique, a computer-implemented method provides data with related first metadata, both originating from a data store and extracts the data and the related first metadata independently from the data store. A universal unique identifier of a portion of the data to which portion specific first metadata exists is created as part of the related first metadata. The universal unique identifier of the portion of the data is integrated into the related first metadata, thereby creating modified first metadata as an independently manageable and linkable representation of the related first metadata.

Abstract: Embodiments of the present invention provide methods, computer program products, and systems. For each dataset in the system, embodiments of the present invention can identify at least one stack of hierarchically-dependent system components on which that dataset is dependent, and generate a system graph having nodes interconnected by directed edges. Dependency of each dataset on a stack, and between successive components in each stack, is represented by a directed edge between nodes of the graph. Embodiments of the present invention can then calculate for each dataset, a stack-dependent compliance score as a function of the compliance scores for a set of nodes reached by traversing directed edges in the graph from the node corresponding to that dataset, and generate an alert if the stack-dependent compliance score for a dataset breaches a predetermined risk threshold for that dataset.

Abstract: A technique for retaining a context in which data resides independently of a data store from which the data originates is disclosed. In relation to a method aspect of the technique, a computer-implemented method provides data with related first metadata, both originating from a data store and extracts the data and the related first metadata independently from the data store. A universal unique identifier of a portion of the data to which portion specific first metadata exists is created as part of the related first metadata. The universal unique identifier of the portion of the data is integrated into the related first metadata, thereby creating modified first metadata as an independently manageable and linkable representation of the related first metadata.

Abstract: A method includes obtaining a first snapshot of a source system and deriving a set of key-value pairs reflecting the first snapshot. A mirror operation of the source system is performed to obtain CDC change operations. The method obtains a first CDC log as a first sequence of key-value pairs. A second snapshot of the source system is obtained and a set of key-value pairs is derived, which reflects the second snapshot. The first sequence of key-value pairs is compared with the set of key-value pairs to derive corrective CDC operations, captured as a set of key-value pairs. The corrective CDC operations represent corrections to be performed with respect to the first sequence of key-value pairs. A second CDC log is obtained as a second sequence of key-value pairs. The corrective CDC operations ensure that the second sequence of key-value pairs are coherent with the set of key-value pairs.

Abstract: An approach to finding data locations may be provided. A first synopses, which corresponds to first tabular data may be provided. An auxiliary data storage system may be scanned to identify second tabular data stored in the auxiliary data storage system. A second synopses may be obtained, in which the synopses correspond to the second columns of second tabular data. The synopsis may be computed for each second column of the second columns, according to a numeric representation of contents of cells of said each second column. The computed synopsis may include a vector of m descriptors. The two sets of one or more descriptors may be compared. A subset of the descriptors of the second synopses may be compared with corresponding descriptors of the first synopses, to identify potential matches between the second tabular data and the corpus of first tabular data.

Abstract: A computer-implemented method for transferring changes made to data from a data source to a data destination may be provided. The method involves using a message broker with streaming channels for transferring data changes from a data source to a data consumer. To determine if data has changed, a portion of data labeled with a key is sent through a hash function. A hash value with the corresponding key that has already been sent to the data consumer is retrieved from a streaming channel and compared to the newly created hash value. If the newly created hash value is different the data is sent to data consumers by the message broker via one streaming channel, while the new hash value is stored in the other streaming channel.

Abstract: A server system includes a database for storing data, a memory device for storing data selected from the data in the database, and a processing device that selects data from the database, stores the selected data in the memory device, and transmits over a communication network an availability signal to a client, the availability signal including an identifier of the selected data and indicating that the selected data is stored in the memory device and is available to be accessed by the client.

Abstract: The present disclosure relates to a method for processing data of a database system, the database system being configured to connect to a centralized cache shared with other database systems. The method includes determining a unique identifier of the content of a table of the database system using a common identification method of the database systems. A database command may be generated for processing data of the table. The command references the identifier. It may be determined if the cache comprises results of a same command with the same identifier. The command may be processed at the database system or the results of the command may be retrieved from the cache depending on whether the cache comprises the results.

Abstract: An approach to finding data locations may be provided. A first synopses, which corresponds to first tabular data may be provided. An auxiliary data storage system may be scanned to identify second tabular data stored in the auxiliary data storage system. A second synopses may be obtained, in which the synopses correspond to the second columns of second tabular data. The synopsis may be computed for each second column of the second columns, according to a numeric representation of contents of cells of said each second column. The computed synopsis may include a vector of m descriptors. The two sets of one or more descriptors may be compared. A subset of the descriptors of the second synopses may be compared with corresponding descriptors of the first synopses, to identify potential matches between the second tabular data and the corpus of first tabular data.

Abstract: A method includes obtaining a first snapshot of a source system and deriving a set of key-value pairs reflecting the first snapshot. A mirror operation of the source system is performed to obtain CDC change operations. The method obtains a first CDC log as a first sequence of key-value pairs. A second snapshot of the source system is obtained and a set of key-value pairs is derived, which reflects the second snapshot. The first sequence of key-value pairs is compared with the set of key-value pairs to derive corrective CDC operations, captured as a set of key-value pairs. The corrective CDC operations represent corrections to be performed with respect to the first sequence of key-value pairs. A second CDC log is obtained as a second sequence of key-value pairs. The corrective CDC operations ensure that the second sequence of key-value pairs are coherent with the set of key-value pairs.

Abstract: A computer implemented method for enabling data coherency between a first data source and a second data source of different structure may be provided. The first data source has a first format and a related first log file. Similar conditions apply to the second data source. The method comprises also determining a first entry in the first log file at which the data have a semantic consistency and storing this as first read barrier. The same applies for an entry in the second data source. Additionally, the method comprises building a third data set derived from data of the first data source and data of the second data source and being coherent with them, using data from the first data source up to the first read barrier and data from the second data source up to the second read barrier.

Abstract: A computer-implemented method for transferring changes made to data from a data source to a data destination may be provided. The method involves using a message broker with streaming channels for transferring data changes from a data source to a data consumer. To determine if data has changed, a portion of data labeled with a key is sent through a hash function. A hash value with the corresponding key that has already been sent to the data consumer is retrieved from a streaming channel and compared to the newly created hash value. If the newly created hash value is different the data is sent to data consumers by the message broker via one streaming channel, while the new hash value is stored in the other streaming channel.

Abstract: A computer implemented method for enabling data coherency between a first data source and a second data source of different structure may be provided. The first data source has a first format and a related first log file. Similar conditions apply to the second data source. The method comprises also determining a first entry in the first log file at which the data have a semantic consistency and storing this as first read barrier. The same applies for an entry in the second data source. Additionally, the method comprises building a third data set derived from data of the first data source and data of the second data source and being coherent with them, using data from the first data source up to the first read barrier and data from the second data source up to the second read barrier.

Abstract: The present disclosure relates to a method for processing data of a database system, the database system being configured to connect to a centralized cache shared with other database systems. The method includes determining a unique identifier of the content of a table of the database system using a common identification method of the database systems. A database command may be generated for processing data of the table. The command references the identifier. It may be determined if the cache comprises results of a same command with the same identifier. The command may be processed at the database system or the results of the command may be retrieved from the cache depending on whether the cache comprises the results.

Abstract: A computerized method for ingesting data from a relational database into a data lake is provided, wherein a user-defined function (UDF) is associated with a standard operation of extract, transform, load, or ETL, of an ETL pipeline. This UDF is triggered upon performing the standard operation and thereby allow a code associated with the UDF to be executed. Upon migrating data from one or more data sources into the relational database, the standard operation is executed, which triggers the UDF and, in turn, an execution of the code. As per the execution of this code, an entity running on the data lake is notified that a set of data migrated to the relational database is to be ingested according to given ingestion modalities specified by the code. Finally, the set of data can be ingested into the data lake according to the modalities. Related computer program products are also provided.

Abstract: A server system includes a database for storing data, a memory device for storing data selected from the data in the database, and a processing device that selects data from the database, stores the selected data in the memory device, and transmits over a communication network an availability signal to a client, the availability signal including an identifier of the selected data and indicating that the selected data is stored in the memory device and is available to be accessed by the client.

Abstract: A computer implemented method includes storing data in a server, selecting a block of data from the stored data, storing the selected block of data in a cache memory of the server, and transmitting a block availability signal to a client, the block availability signal including an identifier of the selected block of data and indicating that the selected block of data is stored in the cache memory and is available to be accessed by the client.

Abstract: A mechanism for declarative service domain federation uses a declarative approach to both the visibility aspect and the management aspect of service domain federation. Instead of manually exporting services, that is, selecting individual services from source domains to be visible in the target domain, using the present system, a federation architect uses a federation console to specify federation intent. The federation intent may include federation rule, query-based service group, and quality-of-service (QoS) policy. Based on the declared federation intent, a planning engine resolves the individual services based on the group query, generates the service visibility configuration, and creates the necessary service proxies and/or mediation objects to satisfy the goal. At runtime, a service monitor collects the performance metrics of federated services and dynamically adjusts the mediation/proxy configurations to maintain the QoS objectives specified by the federation architect.

Abstract: A method, system and computer program product are disclosed for maintaining a membership view of a distributed computing system comprising a multitude of members, wherein each of said members sends out a message indicating that said each of the members is available in the system. In one embodiment, the method comprises maintaining a first list of at least some of the members in the distributed system; and over a period of time, when one of the members sends out the message indicating that said one of the members is available in the distributed system, adding said one of the members to a second list. At defined times, the first list is replaced with the second list. In an embodiment, the method further comprises at said defined times, starting a new, empty second list.

Abstract: A computerized method for ingesting data from a relational database into a data lake is provided, wherein a user-defined function (UDF) is associated with a standard operation of extract, transform, load, or ETL, of an ETL pipeline. This UDF is triggered upon performing the standard operation and thereby allow a code associated with the UDF to be executed. Upon migrating data from one or more data sources into the relational database, the standard operation is executed, which triggers the UDF and, in turn, an execution of the code. As per the execution of this code, an entity running on the data lake is notified that a set of data migrated to the relational database is to be ingested according to given ingestion modalities specified by the code. Finally, the set of data can be ingested into the data lake according to the modalities. Related computer program products are also provided.