Abstract: The present disclosure describes systems, non-transitory computer-readable media, and methods for utilizing hash partitions to determine local densities and distances among users (or among other represented data points) for clustering sparse data into segments. For instance, the disclosed systems can generate hash signatures for users in a sparse dataset and can map users to hash partitions based on the hash signatures. The disclosed systems can further determine local densities and separation distances for particular users (or other represented data points) within the hash partitions. Upon determining local densities and separation distances for datapoints from the dataset, the disclosed systems can select a segment (or cluster of data points) grouped according to a hierarchy of a clustering algorithm, such as a density-peaks-clustering algorithm.

Abstract: Methods and systems are presented for facilitating a data migration process between two data centers in an automated and secured manner. Based on detection of an event, a migration server initiates a data migration process for migrating data from a source data center to a destination data center. The migration server transmits instructions to a first migration application of the source data center, which causes the first migration application to retrieve the data, encrypt the data within an attested enclave of the source data center, and transfer the encrypted data to a pipeline. The migration server also transmits instructions to a second migration application of the destination data center, which causes the second migration application to retrieve the encrypted data from the pipeline, decrypt the encrypted data in an attested enclave of the destination data center, and store the decrypted data in a data storage of the destination data center.

Abstract: Systems and methods are disclosed for mapping search nodes to a search head in a data intake and query system based on a tenant identifier in order to execute a query received by the data intake and query system. The mapping may allow same or similar search nodes to be used to execute queries that are associated with a particular tenant identifier, in order to take advantage of caching and local data stored with those search nodes. In some cases, search nodes can be mapped based on the tenant identifier using a hashing algorithm, such as a consistent hashing algorithm.

Abstract: Dictionary-based compression is performed to compress data units using a similar data unit as the base unit (i.e., dictionary) for each candidate data unit. Similarity may be determined between data units by applying a locality-sensitive hashing scheme to each candidate data unit to produce a hash value, and by determining whether there is a matching value in a hash index of hash values for existing data units on the system. If there is a matching hash value, the candidate data unit may be compressed using the data unit corresponding to the matching hash value as the dictionary. Only a representative portion of the data unit may be hashed to produce the hash value, the portion comprised of chunks of the data unit, where each chunk is a continuous, uninterrupted section of data. The chunks themselves may not be (in some embodiments likely are not) contiguous to one another.

Abstract: A quality check apparatus, a quality check method, and a quality check program can check the quality of input data output to a processing module. A device outputs the input data and first metadata indicating an attribute regarding the quality of the input data to the processing module. The quality check apparatus includes a first obtaining unit and a check unit. The first obtaining unit obtains the first metadata. The check unit checks the quality of the input data based on the first metadata.

Abstract: The present disclosure provides systems, methods, and computer program products for providing efficient embedding table storage and lookup in machine-learning models.

Abstract: Object service receives communication of fingerprints stream, corresponding to file segments, from file source, and identifies sequential fingerprints in fingerprints stream as fingerprints group. Object service identifies group identifier for fingerprints group, and communicates fingerprints group to deduplication service associated with group identifier range including group identifier. Deduplication service identifies fingerprints in fingerprints group which are missing from fingerprint storage, and communicates identified fingerprints to object service, which communicates request for file segments, corresponding to identified fingerprints, to file source. Deduplication service receives communication of requested segments from file source, and stores requested segments.

Abstract: Systems, methods and apparatus are provided for AI-based generation of data warehouse quality protocols. An attribute classifier may quantify relationships between source data and target data from an enterprise data warehouse. A data quality engine may apply these relationships to identify specific data quality concerns and generate customized data quality metrics. A user interface may enable a user to enter parameters for the classification protocols and corresponding rule-based generation of data quality metrics.

Abstract: Herein are techniques for dynamic aggregation of results of a database request, including concurrent grouping of result items in memory based on quasi-dense keys. Each of many computational threads concurrently performs as follows. A hash code is calculated that represents a particular natural grouping key (NGK) for an aggregate result of a database request. Based on the hash code, the thread detects that a set of distinct NGKs that are already stored in the aggregate result does not contain the particular NGK. A distinct dense grouping key for the particular NGK is statefully generated. The dense grouping key is bound to the particular NGK. Based on said binding, the particular NGK is added to the set of distinct NGKs in the aggregate result.

Abstract: Method and apparatus for stress management in a searchable data service. The searchable data service may provide a searchable index to a backend data store, and an interface to build and query the searchable index, that enables client applications to search for and retrieve locators for stored entities in the backend data store. Embodiments of the searchable data service may implement a distributed stress management mechanism that may provide functionality including, but not limited to, the automated monitoring of critical resources, analysis of resource usage, and decisions on and performance of actions to keep resource usage within comfort zones. In one embodiment, in response to usage of a particular resource being detected as out of the comfort zone on a node, an action may be performed to transfer at least part of the resource usage for the local resource to another node that provides a similar resource.

Abstract: In some implementations, a data cleaning platform may determine a respective entity key for each data record in a cleansed dataset based on a combination of fields, in each data record, that contain information that uniquely identifies an entity associated with a respective data record. The data cleaning platform may generate a delta dataset based on a set of uncleansed data records related to transactions that occurred after a time when the cleansed dataset was first generated. For example, in some implementations, each uncleansed data record in the delta dataset may be associated with a corresponding entity key based on the combination of fields. The data cleaning platform may perform a data join to update the cleansed dataset to include data records related to the transactions that occurred after the time when the cleansed dataset was first generated.

Abstract: Tracking changes to a document by defining a document record having a unique document record and comprising an index and a file name of the document, and defining a backup record for the document in a series of backups, which includes a timestamp for each backup, and a bitmask for the document. The bitmask has a single bit position for each document in the container which is set to a first binary value to indicate that the corresponding document is unchanged and a second binary value to indicate whether the document is changed or deleted. A primary query is received and resolved for the document by analyzing the document record to find the file name. A secondary query using the document record ID is resolved to find all tracked versions of the document, and the results are returned to the user in the form of a version history list.

Abstract: Systems and methods for indexing geological features are disclosed. In one embodiment, a method for indexing geological features includes accessing a database storing a plurality of map objects that originate from documents. Each map object includes a map defined by a geographical boundary and a text caption. The method includes, for each map object, determining a plurality of geohashes within the geographical boundary, and includes, for each map object, comparing terms of the text caption with a list of geological keywords. For each map object, the method includes identifying one or more geological noun phrases within the text caption that match one or more geological noun phrases of the list. The method includes determining, for each geological noun phrase, one or more geohashes associated with the geological noun phrase and, for each geohash, determining a frequency that the geohash is associated with the geological noun phrase.

Abstract: Systems and methods provide micro-credential accreditation. The systems and methods analyze, using one or more prediction models, received text submissions received from applicants via interaction with an applicant device. The prediction model(s) fit one or more micro-credentials to the received text submission, which may collectively or independently qualify the applicant for one or more accreditation credits. By processing the received text submission, the systems and methods allow for consistent and standard output of micro-credentials by the prediction model(s). Furthermore, the systems and methods provide for monitoring the prediction model output(s) to ensure ethical fairness across varying demographic groups of applicants.

Abstract: A system and method is described for database split generation in a massively parallel or distributed database environment including a plurality of databases and a data warehouse layer providing data summarization and querying functionality. A database table accessor of the system obtains, from an associated client application, a query for data in a table of the data warehouse layer, wherein the query includes a user preference. The system obtains table data representative of properties of the table, and determines a splits generator in accordance with one or more of the user preference or the properties of the table. The system generates, by the selected splits generator, table splits dividing the user query into a plurality of query splits, and outputs the plurality of query splits to an associated plurality of mappers for execution by the associated plurality of mappers of each of the plurality of query splits against the table.

Abstract: Systems and methods of computing classifications for and migrating digital content that includes accessing a digital content corpus within a source data storage system; in response to accessing the digital content corpus, for each distinct item of digital content of the plurality of distinct items of digital content: computing, via one or more digital content machine learning classification models, a content classification inference; identifying automated digital content handling tasks of a plurality of distinct digital content handling tasks based on the content classification inference; executing the automated content handling tasks identified for each distinct item of digital content, wherein executing the automated content handling tasks includes: designating a storage location within a target data storage system based on the in-migration content classification inference; and migrating a respective item of digital content from the source data storage system to the designated storage location within the ta

Abstract: A method, system, and computer-usable medium are disclosed for answering general background questions on a topic from documents with glossary sections, A set of documents with glossaries is received from which a set of terms and associated glossary entries are extracted, where each term has a corresponding glossary entry. Association is performed of related glossary entries. The associations is based on a similarity algorithm to form glossary clusters where each glossary cluster refers to one or more glossary entries. A query with query terms tailored to general information is received. The glossary clusters are ranked relevance to the query terms to form a ranked set. A set of glossary clusters meeting a high ranked threshold is selected and provided.

Abstract: Described herein is a system and method for initiating a bulk insert in a distributed database. The system described herein provides for a client to identify target index servers of the data records by examining the data records. The system identifies a partition criterion of the target index servers and partitions the data records based on a partition criterion, into subsets of data records. The system builds buckets including the subsets of data records that correspond to a partition. The system inserts each bucket including the subset of the data corresponding to a specific partition to the respective target index server where the specific partition is physically located and inserts the subset of data records in the table locally. The system inserts each bucket in each respective target index server in parallel.

Abstract: In an embodiment of the present invention, a checkpoint is received, the checkpoint indicating at least one source position of unprocessed object(s) of a plurality of objects on at least one source shard in a database system. In response to detecting a shard change related to the checkpoint to the at least one source shard, the shard change resulting in that the at least one source shard is updated to at least one destination shard, a mapping view is generated for representing position change of the unprocessed object(s) related to the shard change. An updated checkpoint is determined for indicating the at least one destination position of the unprocessed object(s) of the plurality of objects on at least one destination shard in the database system based on the mapping view. With these embodiments, the checkpoint may be managed in a more effective way.

Abstract: An index associates fingerprints of file segments to container numbers of containers within which the file segments are stored. At a start of migration, a boundary is created identifying a current container number. At least a subset of file segments at a source storage tier are packed into a new container to be written to a destination storage tier. A new container number is generated for the new container. The index is updated to associate fingerprints of the at least subset of file segments to the new container number. A request is received to read a file segment. The index is queried with a fingerprint of the file segment to determine whether the request should be directed to the source or destination storage tier based on a container number of a container within which the file segment is stored.