Abstract: A database export system exports data using a plurality of nodes that process the data to generate structured result files that are partitioned by an export parameter in an export request. The database export system distributes the data and merges the files to avoid small file creation and increase processing speed via parallelism. The database export system generates the result files of a specified maximum size in a final format, where the files are processed merged in a temporary file format. The parallel processing is optimized and constrained per the amount of processing nodes, available memory, requested final file sizes, and operation based ordering to complete data exports in a scalable multi-stage approach.

Abstract: A mechanism for storing indexing updates is described. A secondary storage system is configured to store a backup of a primary storage system. A plurality of different indexing updates of a data repository associated with the secondary storage system is received. An indexing update may include metadata and/or updates to the metadata for a data item in the backup. The indexing updates may be obtained from an indexing process. At least a portion of the indexing updates are sent for storage in an intermediate store. An indication to commit the indexing updates is received. In response, the indexing updates including the at least the portion of the indexing updates are requested to be committed. The indexing updates batched together into a batch to be committed to a metadata store.

Abstract: A system and method matches data from a first set of data with that of an other set of data.

Abstract: Methods, apparatus, and processor-readable storage media for efficient synchronization of a cloud enabled file system database during a snapshot restore operation are provided herein. An example computer-implemented method includes intercepting a user request to initiate a snapshot restore operation on a file system associated with a local storage system, wherein the file system comprises one or more stub files that are indicative of locations of cloud objects comprising files that were previously sent from the local storage system to a cloud storage platform; prior to the snapshot restore operation being performed, creating a current snapshot of the file system; obtaining an indication that the file system was successfully restored on the local storage system; and using the current snapshot to perform a synchronization operation, wherein the synchronization operation deletes one or more orphan cloud objects in the cloud storage platform that resulted from the snapshot restore operation.

Abstract: An identification of a new primary snapshot created for a primary storage system is received. A threshold time window based on a capture time associated with the new primary snapshot is determined. Entries of a storage log of the primary storage system occurring within the threshold time window are analyzed to identify any objects of the primary storage system that have changed during the threshold time window. Changed objects to capture in a new secondary backup stored at a secondary storage system and corresponding to the new primary snapshot are identified. Metadata of the new primary snapshot and metadata of a previous secondary backup are compared to determine for each of the objects of the primary storage system identified as having changed during the threshold time window whether a change to an object since the previous secondary backup is captured in the new primary snapshot.

Abstract: Techniques for query execution against relational databases using connection pooling are described. According to some embodiments, a query processing service is disclosed that enables users to execute queries against target relational databases implemented by a relational database service. The service receives query requests originated by a client application at a web service endpoint and identifies a connection to a particular target database. In some examples, the query request is a Hyper Text Transfer Protocol (HTTP) message comprising a query to be executed by a target database instance within the provider network. The service transmits the query for execution at the target database via the connection and obtains a query result based on the execution of the query. The service transforms the query result into a format suitable for the client and transmits a query response to the client.

Abstract: Disclosed herein are systems and methods for intelligent metadata management and data lineage tracing. In exemplary embodiments of the present disclosure, a data element can be traced throughout multiple applications, platforms, and technologies present in an enterprise to determine how and where the specific data element is utilized. The data element is traced via a hierarchical key that defines it using metadata. In this way, metadata is interpreted and used to trace data lineage from one end of an enterprise to another.

Abstract: Embodiments are directed to managing data in a file system over a network. A file system that includes objects may be provided. A capacity value of the file system may be provided based on a storage size of the objects. A threshold value may be determined based on the capacity value and a metric partition count. Traversals of the file system may determine metric partitions such that each metric partition may be associated with some objects Metrics associated with activities that are associated with the file system may be monitored. Metric partition values may be determined based on metric values associated with the metric partitions such that each metric partition value may be based on a portion of the metric values associated with a portion of the activities associated with a corresponding metric partition. Visualizations may be generated based on the metric partition values.

Abstract: Techniques for generating a crash-consistent clone of file(s) stored by a distributed file system are described herein. To generate the crash-consistent clone, a coordinator node may identify multiple nodes (e.g., at least a first node and a second node) of the distributed file system that are storing different portions of data of the file. The coordinator node may then send a first command to the multiple nodes to cause each of the multiple nodes to quiesce the file and clone the different portions of the data to generate the crash-consistent clone of the file. The coordinator node may then receive, from the multiple nodes, a status associated with generating the crash-consistent clone of the file. Based at least in part on the status, the coordinator node may send a second command to the multiple nodes to cause each of the multiple nodes to unquiesce the file.

Abstract: Systems and methods are provided for accessing master data comprising a plurality of representative data records, where each representative data record represents a cluster of similar data records, and each similar data record has a confidence score indicating a confidence level that the similar data record corresponds to the cluster, and comparing a new data record to each representative data record of the plurality of representative data records using a machine learning model to generate a distance score. The systems and methods further provide for analyzing the cluster of similar data records corresponding to each representative data record in a selected set of representative data records to generate candidate values for the requested data field of the new data record, and generating a candidate score for each of the candidate values using the distance score and the confidence score to use in providing a recommended candidate value.

Abstract: A method of providing file data from a file system comprising a data store and a file record database. The method comprises the steps of receiving a request to open a file in the file system and determining the desired properties of the file using the syntax of the request. A file record is then created in the file record database based on the determined properties of the file, and a file handle for the file record is returned. Data constituting the requested file from data in the data store is generated based on the determined properties of the file, and in response to a request to read a portion of data from the file, a corresponding portion of the generated data is returned.

Abstract: Provided are methods and systems comprising receiving a data set comprising a plurality of records having a plurality of fields, displaying a plurality of graphical objects representing the plurality of records, receiving a selection of one or more of the plurality of fields, determining, based on the selected one or more of the plurality of fields, a similarity score between each of the records of the plurality of records, and adjusting, based on the similarity scores, the plurality of graphical objects.

Abstract: Provided are a computer program product, system, and method for merging multiple point-in-time copies into a merged point-in-time copy. A repository maintains a full copy of the source data and point-in-time copies at point-in-times of the source data. Each of the point-in-time copies have change information indicating changed data in the source data that changed between the point-in-time of the point-in-time copy and a subsequent point-in-time and changed point-in-time data comprising data in the source data as of the point-in-time of the point-in-time copy indicated in the change information as changed. At least two selected of the point-in-time copies in the repository are merged into a merged point-in-time copy by: forming merged change information in the merged point-in-time copy indicating changed data indicated in change information for the selected point-in-time copies; and forming merged changed data in the merged point-in-time copy from the changed data in the selected point-in-time copies.

Abstract: An issue tracking system configured to determine similarity between issue content in an issue request issue content in a previously submitted issue record. Based on the similarity, the issue tracking system may determine a likelihood that a given issue is related to at least one additional issue. In response to a determination that likelihood exceeds a threshold, data can be extracted from the related issue to provide the user with a suggested supplemental issue request to be submitted to the issue tracking system.

Abstract: Information related to apparently successful end users is collected, stored, and used to generate at least one evaluation criteria to compare to at least one component of a respective end user desiring to become a successful end user. The information may be generated based on comparative information with other entities who appear to share some components with the respective end user. The components may be based on actual actions, preferences, constraints, attributes, etc. A number of components of the respective end user are compared relative to a set of defined evaluation criteria that specifies defined evaluation criteria for at least some of those components. In response to detecting at least one discrepancy, a prompt is provided to the respective end user indicative of the discrepancy.

Abstract: A file directory tree structure of a selected storage snapshot is dynamically divided into different portions. A plurality of the different file directory tree structure portions are analyzed in parallel to identify any changes of the selected storage snapshot from a previous storage snapshot. To analyze each of the plurality of the different file directory tree structure portions, a processor is further configured to traverse and compare a corresponding file directory tree structure portion of the selected storage snapshot with a corresponding portion of a file directory tree structure of the previous storage snapshot while at least another one of the plurality of the different file directory tree structure portions of the selected storage snapshot is being analyzed in parallel.

Abstract: A system includes processor hardware and memory hardware that stores instructions. The instructions include, in response to receiving a request, determining a request type of the request, retrieving a first set of collected information, and selecting a first set of instructions corresponding to the request type. The instructions include constructing a first result by executing each instruction of the first set of instructions to create the first entry as a nested entry within the first result including data of the first set of collected information identified in the first set of instructions as nested or retrieve first data of the first set of collected information identified by the first instruction and add the first data to the first entry of the first result. The instructions include transforming a display of the operator device to complete a set of fields displayed on the display with corresponding entries of the first result.

Abstract: Provided are systems and methods for paging data into main memory from checkpoint data stored on disk. In one example, the method may include one or more of receiving a request for a database record in main memory, determining whether the database record has been previously stored in the main memory, in response to determining that the database record has been previously stored in the main memory, identifying a slice where the database record was stored from among a plurality of slices included in the main memory, and paging content of the identified slice including a copy of the requested database record into the main memory from a snapshot captured of content included in the identified slice and previously stored on disk. Accordingly, documents can be paged into main memory on-demand from snapshots of slice content rather than paging an entire partition of content.

Abstract: There is provided a method of automatically tagging social network posts with geo-location tags, comprising: for each of a plurality of user generated content items uploaded to a social network from a plurality of different client devices: selecting a specific geographic region mapping dataset of a plurality of geographic region mapping datasets corresponding to a specific geographic region identified by an analysis of the respective user generated content post, mapping by the specific geographic region mapping dataset, a generic term to a specific geographic location within the specific geographic region, and tagging the respective user generated content item with a geo-location tag of the specific geographic location within the specific geographic region.

Abstract: Embodiments of method and system are directed to a best effort method to reconstruct the corrupt files from available nodes or tiers and heal or repair the corrupt files. The method looks for the existence of fingerprints of the lost segments in available tiers on the nodes or across nodes within a cluster. The method discovers the copies of these missing segments, creates an Lp tree referring these discovered segments and creates a virtual file in the namespace. The file is then moved to the destination tier where the corruption exists, thus healing the corruption.