Abstract: A deduplicated storage system is provided according to certain embodiments that uses one or more mechanisms to update the deduplication database and remove records corresponding to data blocks that have been or will be erased from the secondary copies, without using or tracking reference counting values. Some embodiments described herein use a secondary table (for tracking archive file contents) and a bitmap to mark which primary records are present in the secondary table. In another embodiment, once the marking phase is completed, the deduplication system uses the marked-up bitmap to identify the corresponding records from the primary table that can be moved to another table for storing “zero-reference” data blocks. In other embodiments, the system will then traverse the “zero-reference” table and remove those primary data blocks from secondary storage devices.

Abstract: In an example, a piece of content is obtained. The piece of content is segmented into a plurality of segments. Each of the plurality of segments is divided into a plurality of units. Then, for each of the plurality of units for each segment, a quality model is used to pass the unit through a long short-term memory (LSTM) corresponding to the unit, causing an embedding of the unit and generating one or more parameters. For each of the plurality of segments for the piece of content, the quality model is used to pass the segment, and one or more parameters obtained from LSTMs corresponding to units within the segment, through an LSTM corresponding to the segment, causing an embedding of the segment. The piece of content is then classified as spam based on the embeddings of the plurality of segments for the piece of content, using the quality model.

Abstract: A method is provided that stores, in a second memory, an index structure including, for each given word from among words included in documents, a group of document IDs of documents including the given word. The method stores an index structure subset in a main memory which is faster than secondary memory. The method acquires a keyword and identifies any documents including the keyword. The method finds top-K frequent words among the words included in the identified documents by: identifying, for each given group in descending order of the number of the documents IDs therein, the number of documents IDs of the identified documents in the given group, from the subset when the number of document IDs in the given group is within the range, and from the index structure otherwise; and presenting words of top-K groups with a largest amount of the document IDs identified.

Abstract: Computer program products, methods, systems, apparatus, and computing entities are provided for creating and using heterogeneous pools of validators to implement blockchain consensus. The specialized validators can be segmented and pooled based on their roles and functionalities in the consensus mechanism.

Abstract: A server receives a query identifying one or more attributes of an entity. The server accesses a set of candidate templates for answering the query based on the one or more attributes of the entity, each candidate template having one or more fields, wherein each field is associated with at least one constraint. The server obtains a set of information that answers the query and selects a template from the set of candidate templates, where the selected template has a maximum number of fields with constraints that are satisfied by the set of information. The server then generates a phrase by adding the set of information to the one or more fields of the selected template, such that the phrase comprises an answer to the query. Finally, the server communicates the phrase to a client device.

Abstract: An apparatus for guaranteeing the integrity of a state database includes a memory for storing one or more instructions, a communication interface for communicating with one or more blockchain nodes and a processor. The processor is configured, by executing the stored one or more instructions, to receive a request to query for a state value, obtain a first state value by querying a state database in response to the request, forward the request to the one or more blockchain nodes to receive a second state value from the one or more blockchain nodes and compare the first state value with the second state value to determine integrity of a state value.

Abstract: One or more assignments (e.g., one or more maps) can be used for assigning data to processing units of a database system with multiple processing. In other words, assignments can be provided as a more effective solution for assigning data to the processing units of database systems that can operate with multiple processing units. Generally, an assignment can be used to assign data to the processing units for processing, virtually in any desired manner (e.g., virtually any desired function). By way of example, maps can associate data to containers (e.g., buckets) and associate the containers to processing units of database system with multiple processing units.

Abstract: Methods, systems, and devices supporting data archiving are described. A device (e.g., an application server) may identify a set of archiving jobs for a first database (e.g., a relational database), where each archiving job indicates a set of data records for archiving. The device may receive a data record of the set of data records from the first database, where the data record is filtered by an abstraction layer (e.g., removing some information from the data record such that the filtered data record corresponds to a user's view of the data record). For example, the filtered data record may include a subset of the total information for the stored data record. The device may transform this received, filtered data record from a first format to a second format supported by a second database. The device may then send the transformed data record to the second database for storage.

Abstract: Various examples are directed to systems and methods for identifying database records in a database table. A database management system receives a search request comprising a first set of strings associated with a first column of the database table and a second set of strings associated with a second column of the database table. The database management system selects a set of first column keywords using the first set of strings and executes a first tier query at the database table. Responsive to determining that no database record returned by the first tier query has a relevance score greater than a threshold value, the database management system executes a second tier query at the database table.

Abstract: Systems, methods, and devices for batch ingestion of data into a table of a database. A method includes determining a notification indicating a presence of a user file received from a client account to be ingested into a database. The method includes identifying data in the user file and identifying a target table of the database to receive the data in the user file. The method includes generating an ingest task indicating the data and the target table. The method includes assigning the ingest task to an execution node of an execution platform, wherein the execution platform comprises a plurality of execution nodes operating independent of a plurality of shared storage devices collectively storing database data. The method includes registering metadata concerning the target table in a metadata store after the data has been fully committed to the target table by the execution node.

Abstract: A question is analyzed to determine a set of categories, a category corresponding to a type of possible answers responsive to the question. A set of opinions is extracted from social media data. Each opinion is from a corresponding responder on a social media platform to which the question is sent. An opinion from the set of opinions is categorized into a category from the set of categories. A strength of the opinion is computed using a subset of a set of strength parameters. The opinion is ranked according to the strength of the opinion. The opinion, the strength, and the category are presented in a filterable presentation.

Abstract: The present application discloses a data configuration query method and device. The method includes: determining at least two target data sets required to be queried by a query instruction and an ordered association between the target data sets, wherein the ordered association at least includes a unidirectional association and/or a bidirectional association; generating, on the basis of the at least two target data sets and the ordered association between the target data sets, a target association path; and outputting an OLAP model conforming to the target association path in a database. According to the present application, the technical problems of a large demand quantity of OLAP models included in an OLAP query system and a low utilization rate of the OLAP models in the related art can be solved.

Abstract: The described technology is generally directed towards incremental aggregation of metadata for a file storage system. According to an embodiment, a system can comprise a memory and a processor that can execute the components stored in the memory. The components can comprise a scanner component that can accessing a data structure storage component that can store a first data structure, and a branch of the first data structure can comprise a node that comprises at least one descendent link to a descendant node. The scanner component can further traverse from a first node to a second node by employing a first descendent link. Further, the method comprises a data collector that can collect node data from the first node and the second node. The system can further comprise a rollup data generator to aggregate, upon occurrence of a condition, the node data, resulting in aggregated node data.

Abstract: The disclosed technology includes ranking entities in real-time to show the relative importance of those entities. The ranking is based on attributes of the entities that vary in real-time. An example of an entity is a process (e.g., an executing computer program) and the associated attributes can include the process' current CPU memory consumption. While the process runs, its CPU and memory consumption vary in real-time.

Abstract: A streaming ingest platform can improve latency and expense issues related to uploading data into a cloud data system. The streaming ingest platform can organize the data to be ingested into per-table chunks and per-account blobs. This data may be committed and may be made available for query processing before it is ingested into the target source tables. This significantly improves latency issues. The streaming ingest platform can also accommodate uploading data from various sources with different processing and communication capabilities, such as Internet of Things (IOT) devices.

Abstract: A method for generating a universal distributed data storage includes: receiving a plurality of data values; generating a key for each of the plurality of data values, wherein the respective key is generated via application of a hashing algorithm to the associated data value; storing, in a database, each of the plurality of data values and the associated generated key as a plurality of key-value pairs; receiving a plurality of additional key-value pairs, wherein each value of the plurality of additional key-value pairs is not included in the plurality of data values; and storing, in the database, each of the plurality of additional key-value pairs, wherein each key included in the plurality of additional key-value pairs is generated via application of the hashing algorithm to the associated value and is not included in the keys generated for each of the plurality of data values.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable storage media for implementing a scalable, secure, efficient, and adaptable distributed digital ledger transaction network. Indeed, the disclosed systems can reduce storage and processing requirements, improve security of implementing computing devices and underlying digital assets, accommodate a wide variety of different digital programs (or “smart contracts”), and scale to accommodate billions of users and associated digital transactions. For example, the disclosed systems can utilize a host of features that improve storage, account/address management, digital transaction execution, consensus, and synchronization processes. The disclosed systems can also utilize a new programming language that improves efficiency and security of the distributed digital ledger transaction network.

Abstract: The subject technology performs a locking operation on a first set of keys by a first statement of a first transaction. The subject technology determines that a conflict occurred between the first statement and a second transaction. The subject technology determines that the second transaction has yet to complete after a predetermined period of time. The subject technology performs a deadlock detection process where the subject technology stores a key and value in a table indicating the first transaction and the second transaction, detects, based at least in part on a graph traversal of the table starting from the first transaction, a cycle between the first transaction and the second transaction, and determines that the first transaction is a youngest transaction in the detected cycle. The subject technology ceases execution of the first transaction in response to the first transaction being a youngest transaction in a detected cycle.

Abstract: Techniques are described for materializing computations in memory. In an embodiment, responsive to a database server instance receiving a query, the database server instance identifies a set of computations for evaluation during execution of the query. Responsive to identifying the set of computations, the database server instance evaluates at least one computation in the set of computations to obtain a first set of computation results for a first computation in the set of computations. After evaluating the at least one computation, the database server instance stores, within an in-memory unit, the first set of computation results. The database server also stores mapping data that maps a set of metadata values associated with the first computation to the first set of computation results.

Abstract: Embodiments of the present invention provide a system for refreshing data within the testing environment by sanitizing production data. In particular, the system may process real-world data obtained in the production environment and automatically replace sensitive or confidential information with realistic substitute data, then provide the sanitized data to the systems within the low level testing environment. The system may account for differences in table schema amongst a plurality of disparate databases within the production environment and the low level environment, thereby allowing the system to provide sanitized data only to the extent that it is needed for testing purposes. Furthermore, the system selectively provides sanitized test data on an as-needed basis, removing the need to continuously generate testing data.