Abstract: Techniques for log storage in distributed data streaming systems are described. A cluster of brokers receive log records from publishers and send log records to subscribers. The log is represented as a group of segments, each segment subdivided into chunks. Metadata describes the log structure. Log records are stored in chunks at least in a remote storage location shared amongst the brokers in the cluster.

Abstract: A computer implemented method can receive a query for a database table. The query has a filter having a first predicate and a second predicate. The first predicate specifies a first condition evaluating values in a first column of the database table and the second predicate specifies a second condition evaluating values in a second column of the database table. The method can obtain a projection histogram including base statistics of the first column and projected statistics of the second column and determine an output size of the filter based on the projection histogram. The base statistics includes a plurality of intervals and respective counts of cells in the first column whose values are within corresponding intervals. The projected statistics includes a plurality of ranges corresponding to the plurality of intervals, respectively. Related systems and software for implementing the method are also disclosed.

Abstract: Certain aspects of the disclosure provide for a difference analysis method. In certain aspects, a difference analysis method may include embedding a set of source documents into a knowledge graph, wherein each source document is embedded in the knowledge graph as a set of segments and a set of associations connecting two or more segments. A difference may be determined between a first segment in the set of segments of a first source document and a second segment in the set of segments of a second source document. In response to determining the difference between the first segment in the set of segments of the first source document and the second segment in the set of segments of the second source document, determining a significance of the difference on the second source document based on one or more associations of the set of associations connected to the second segment.

Abstract: A system and method of blockchain-based data management of multi-source, multi-format data objects. The method including obtaining a plurality of data objects from a plurality of sources in a plurality of formats. The method including storing the plurality of data objects on one or more blockchains in a blockchain format. The method including generating, by a processing device using natural language processing (NLP), one or more keywords for each data object of the plurality of data objects. The method including respectively linking each of a plurality of metadata tags to a single data object of the plurality of data objects on the one or more blockchains, each metadata tag comprising the one or more keywords for the single data object.

Abstract: In several aspects for storing a table, in which the table is configured for comprising records having values of one or more attributes, a processor creates a pseudo table representing the table, the creating comprising defining an empty version of the table in a program. One or more records may be inserted in the pseudo table by encoding attribute values of the one or more records as inline data in the program. In response to determining that the pseudo table satisfies a materialization criterion, the pseudo table may be materialized by creating the table with the records encoded in the program and removing the program.

Abstract: The present invention envisages a system (100) for data archival in a blockchain network (20). The blockchain network (20) comprises a plurality of participating nodes, each categorized either as an archiver node (A1-A3) or a non-archiver node (N1-N3). The archiver nodes (A1-A3) maintain a database (102) comprising transaction data corresponding to all block creation events and corresponding transaction IDs. Each of the non-archiver nodes (N1-N3) comprises a data management module (104) which maintains an archival ledger (106a) comprising transaction data corresponding to a partial list of block creation events and corresponding transaction IDs. The data management module (104) receives a request for querying a transaction from a requester-non-archiver node, checks the archival ledger (106a) to determine whether the queried block is discarded or not, and retrieves and returns the transaction data from one of the archival ledger (106a) and the database (102) based on the determination.

Abstract: A computer-implemented method comprises creating a plurality of access group identifiers each associated with one or more user account identifiers, and a plurality of classification markings each representing a different level of security classification of an electronic document; associating each account identifier with at least one classification marking; indexing, for each particular electronic document, one or more access group identifiers and one or more classification markings; receiving a search query specifying one or more document properties; obtaining a first set of classification markings associated with the search query; executing a search of the indices using a covering query that requires a specified minimum number of matches between the first set of classification markings and a second set of classification markings associated with a particular document satisfying the one or more document properties to obtain a result set of electronic documents; providing the result set in response to the searc

Abstract: An apparatus and method for determining a probability datum is provided. The apparatus includes a processor and a memory connected to the processor. The memory contains instructions configuring a processor to receive a first datum that describes actions of the user, and to receive a second datum that describes a target attainment value, and to classify the first datum and the second datum to various labels, each label identified by a label value. Classifying includes generating a gap datum by as a function of the first datum and the second datum and categorizing the first datum with a label based on the gap datum and the label value of each label. The processor may determine the probability datum by using each label to select a respective probability generation model configured to determine the probability datum of the first datum matching the second datum per label based on the gap datum.

Abstract: Systems and methods for preprocessing large inference files in a cluster environment prior to transmission to one or more downstream applications. The inference files are processed using templates that correspond to particular downstream applications, allowing for optimized transmission and optimized processing by each downstream application.

Abstract: The present disclosure relates to a method. A main transaction may be received. The main transaction comprises a set of change records of the at least one source table. Each change record represents at least one data record of a source table of the at least one source table. A least one subset of change records of the set of change records may be determined. Each subset of change records is determined such that it represents a distinct source partition of the at least one source table. A sub-transaction may be determined per subset of change records of the at least one subset of change records. The main transaction may be executed by at least concurrently executing the set of sub-transactions, wherein the execution of each sub-transaction of the sub-transactions comprises concurrently adapting a data store and adapting a metadata store in accordance with the sub-transaction.

Abstract: A spatial-temporal storage method, system, and non-transitory computer readable medium include dynamically managing a plurality of region servers for querying spatiotemporal data in noSQL databases.

Abstract: A computer-implemented method for suggesting keywords as a search term of a content item includes receiving, from a content provider, information about the content item in a database of content items. The method further includes generating a set of seed keywords related to the content item, and expanding the set of seed keywords to a plurality of candidate keywords. The plurality of candidate keywords are then scored based, at least in part, on an engagement metric measuring a user engagement with the content item in response to being presented with results from a search query comprising the candidate keyword. A candidate keyword is then selected from the plurality of candidate keywords based on the scoring, and stored relationally to the content item to define an audience for a recommendation about the content item, providing a suggestion to the content provider.

Abstract: A method can include determining, via a translatability classifier module, a translatability class label for a cross-lingual search query received, via a computer network, from a user device for a user. The translatability classifier module can be trained to determine the translatability class label among multiple translatability class labels associated with user intentions for the cross-lingual search query based on a respective probability of the cross-lingual search query being associated with each of the multiple translatability class labels. The method further can include determining, via a language translator module, a class-associated search query for the cross-lingual search query based on the translatability class label, as determined. The method additionally can include transmitting, via the computer network, the class-associated search query to a monolingual search engine. Other embodiments are disclosed.

Abstract: A computer implemented method can receive a write request to store sensor data generated by a sensor group including a plurality of sensors. The sensor data includes at least a structure identifier and one or more pairs of sensor parameters and sensor values. The method can identify a metadata structure based on the structure identifier. The metadata structure includes a plurality of tuples corresponding to the plurality of sensors in the sensor group. The tuples define pairs of field names and data types of the corresponding sensors. The method can pack the one or more pairs of sensor parameters and sensor values into one or more data containers, and store the one or more data containers respectively into one or more columns of a sensor data table.

Abstract: Changes can be synchronized in a distributed system with intermittent connectivity. A synchronization coordinator on a server and an application on mobile devices can be configured to synchronize a shared dataset using file transfers. By transferring files, the application on the mobile device can synchronize large, shared datasets via a long-running task without needing to remain active on the mobile device until the task is completed. The synchronization coordinator and applications can use unique identifiers to identify storage locations by which the files are transferred.

Abstract: Metadata of a file system is maintained in a key value store. The metadata includes a namespace represented as a B+ tree having pages written to the key value store. Identifiers to the pages include a snapshot identifier and a page number. The identifiers are stored as keys in the key value store. The snapshot identifiers are used to track pages that are shared between first and second snapshots. Upon a request to write to an existing page, a determination is made from the snapshot identifiers as to whether the existing page belongs to the first or second snapshot. If the existing page belongs to the second snapshot, the write to the existing page is allowed. If the existing page belongs to the first snapshot, a copy on write (COW) is performed to generate a new page for the write.

Abstract: The application discloses a method, apparatus, electronic device and storage medium for data aggregation. The method includes: obtaining an aggregation basis dimension and an aggregation calculation dimension, wherein the aggregation basis dimension and the aggregation calculation dimension are respectively fields in a data source table to be aggregated; obtaining an aggregation result by aggregating field values corresponding to the aggregation calculation dimension of at least one target row, wherein field values of the aggregation basis dimension of the at least one target row are the same; and recording a correspondence relationship between a target aggregation path and the corresponding aggregation result, wherein the target aggregation path is determined according to field values of the aggregation basis dimension of the target row.

Abstract: One example method includes generating in a shared protection namespace a first point-in-time copy of a first working frozen copy of an active namespace that includes first files associated with the active namespace that are stacked in a first horizontal file. A retention lock is applied to the first horizontal file. A second point-in-time copy of a second working frozen copy of the active namespace that includes second files associated with the active namespace that are stacked in a second horizontal is generated. A retention lock is applied to the second horizontal file. The second horizontal file is generated by copying files from the first horizontal file into the second horizontal file and copying files from the second working frozen copy of the active namespace into the second horizontal file.

Abstract: A blockchain system is enabled to participate in distributed transactions that uses a two-phase commit protocol (“2PC”). In a 2PC, a computer system, such as a DBMS or blockchain system, commits a transaction that changes data (e.g. database, world state) using two phases. To participate in a distributed transaction using 2PC, a blockchain system executes a “staged transaction”. A staged transaction transitions through the 2PC phases. In the prepare phase, the new values for world state records are staged in staging records as staged values. In the second phase, if the distributed transaction is to be committed, the world staging records are set to the staged values.

Abstract: Columnar storage provides many performance and space saving benefits for analytic workloads, but previous mechanisms for handling single row update transactions in column stores suffer from poor performance. A columnar data layout facilitates both low-latency random access capabilities together with high-throughput analytical access capabilities, simplifying Hadoop architectures for use cases involving real-time data. In disclosed embodiments, systems may perform read and scan features through which users may have consistent reads of the columnar storage at any given point in time. These scans (e.g., snapshots) allow for analytics to be performed on the data, consistent backups of the data, for incremental backups, perform cross-cluster synchronization, and/or for offline audit analysis.