Abstract: An approach to implement an extract, transform and load system, a queue is provided for holding units of data between extraction and transformation. When units of data suffer a rate limit error in the load phase, they are requeued so they can be resubmitted for transformation. The contents of the queue are monitored and, if too many requeued units of data are detected in the queue, then this is taken as an indicator of an unacceptable number of rate limit errors and active pacing management is triggered. A retry schedule is defined for the requeued units of data. Extraction is temporarily halted to allow the requeued units of data to be retransformed without more units of data queuing up. Then, after the suspension is lifted, a pacing delay is inserted between subsequent extract events to avoid the same load phase bottleneck recurring.

Abstract: System and methods for compressing data for use in a distributed ledger are provided. In one form, the system includes: a network interface; a user interface device; and a control circuit coupled to the network interface and the user interface device. In this form, the control circuit executes a data compression module configured to: receive, via the network interface, a data set; transform and store the data set into a file format of rows and columns in which data of the data set are stored in a columnar fashion; apply a first compression step to compress each row and write each compressed row into the file format; apply a second compression step to further compress data after the compressed rows have been written into the file format; and encode the compressed data and transmit the encoded data to a first node of a distributed ledger.

Abstract: A computing system is provided, which is configured to instantiate a testing environment, define an extract, transform, load (ETL) pipeline within the testing environment, generate a test data set to be inputted into the ETL pipeline, generate assert data predicting an output of the ETL pipeline based on the generated test data set, input the test data set into the ETL pipeline to generate output data, compare the assert data to the output data, and validate the output data using the assert data to generate validation data.

Abstract: Various methods, apparatuses/systems, and media for integration of user defined metrics of cloud hosted relational databases with cloud monitoring services are disclosed. A processor receives via a serverless function implemented within the processor a configuration file containing configuration of collection of database connection string and respective metrics query. The serverless function connects to the secrets provider to fetch a password or authorization token to connect with a plurality of different types of cloud hosted relational databases. The processor returns, in response to receiving the respective metrics query, a table with two columns where one column includes a user defined metric name and the other column includes the metric query to fetch a user defined metric value for the given user defined metric name from the databases; and automatically integrates the fetched user defined metric value of the databases with the cloud monitoring service.

Abstract: A method for facilitating rapid genome sequence analysis includes accessing an output stream of an alignment process that includes aligned reads of a biological sequence that are aligned to a reference genome. The method also includes distributing the aligned reads to a plurality of computing nodes based on genomic position. Each of the plurality of computing nodes is assigned to a separate data bin of a plurality of data bins associated with genomic position. The method also includes, for at least one aligned read determined to overlap separate data bins of the plurality of data bins, duplicating the at least one aligned read and distributing the at least one aligned read to separate computing nodes of the plurality of computing nodes that are assigned to the separate data bins.

Abstract: The present disclosure provides a driving data processing method, an apparatus, a device, an automatic driving vehicle, a medium and a product, including: collecting first driving data generated by using a target structure in a driving system of a target vehicle, where the first driving data uses a binary number system; determining a target descriptor corresponding to the target structure, where the target descriptor is used to describe member information of the target structure; parsing the target descriptor to obtain at least one piece of member information of the target structure; parsing the first driving data according to the at least one piece of the member information to obtain target data corresponding to each of the at least one piece of the member information; and performing a data processing operation based on the target data corresponding to each of the at least one piece of the member information.

Abstract: The subject matter disclosed herein provides methods for downloading chargeback records. For example, the method may include selecting a merchant from a first web portal associated with a first financial institution and providing first authentication information associated with the merchant and the first financial institution into the first web portal. The method may then include accessing, via the first web portal, first chargeback records for the merchant. Additionally, the method can include selecting the merchant from a second web portal associated with a second financial institution and providing second authentication information associated with the merchant and with the second financial institution into the second web portal. The second authentication information can be different than the first authentication information. Then, the method can include accessing second chargeback records for the merchant.

Abstract: The apparatus, systems and methods dynamically provide the reliability of multimedia documents by applying a series of intrinsic criteria and extrinsic criteria by pre-calculating a reliability score for at least a set of multimedia documents of at least one pre-selected source of multimedia documents, and by providing, in response to a request, the multimedia documents from the pre-selected sources associated with the score and the multimedia documents from the other sources associated with a score conditionally calculated.

Abstract: A storage manager for offloading graph components to persistent storage for reducing resident memory in a distributed graph processing engine is provided. The storage manager identifies a set of graph components required to execute a graph processing operation on a graph in a graph processing engine of a database system and reserves an amount of memory needed to load the set of graph components into memory. The storage manager loads the set of graph components into memory and initiates execution of the graph processing operation using the set of graph components in memory. The storage manager evicts one or more unused graph components from memory in response to receiving a request to free a requested amount of memory from memory.

Abstract: Example solutions for performing federated graph queries use schemas and include: generating a logical graph using a query input and generating a physical plan using a query planning schema for double-nested loops, the outer loop for each permutation of data stores in a set of data stores, and the inner loop for each data store—ceasing when the logical graph has been fully consumed. A query cost is determined for each of the permutations, and a physical plan is selected based on the query cost, for example minimizing the query cost, and the query is executed based on the selected physical plan. The cost may be determined based on a weighted combination of factors such as freshness of the data, latency, power consumption, environmental impact (e.g., carbon footprint), energy efficiency, and network burden.

Abstract: The example embodiments are directed to systems and methods of a locking mechanism for a database which permits a partial quantity assignment of a data item to a database request. In one example, the method may include receiving a database request comprising a requested quantity value of a data item, setting an exclusive lock on the data item within a database, protecting the requested quantity value by posting a TQA to a table, wherein the TQA comprises an identifier of the requested quantity value, an identifier of the database request, and a timestamp, releasing the exclusive lock for the database request on the data item within the database in response to recording the TQA, and with the exclusive lock released, determining a quantity value of the data item which is available for the database request and confirming the determined quantity via the temporary quantity assignment table.

Abstract: Systems and methods for generating a playlist on a device for a user of a music service are provided. In embodiments, a method includes initiating an online mode of the music service, where the device is connected to a music service server, and storing, in a cache memory of the device, at least one media object accessed by the user using the music service. The method further includes identifying the at least one media object as satisfying a pre-designated condition, and in response, automatically adding the at least one media object to a playlist stored in the cache memory. The method also includes receiving an indication, via a user interface of the device, to initiate an offline mode of the music service, where the device is not connected to the music service server, and upon initiating the offline mode, enabling playback of the playlist from the cache memory.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying a profile from a content feed within a messaging system. The program and method provide for receiving by a messaging application, first user input corresponding to a request for a content feed, the content feed comprising a plurality of media content items; providing, in response to receiving the first user input, for display of the content feed; receiving, by the messaging application, second user input of a predefined gesture performed with respect to a first media content item of the plurality of media content items; determining, in response to receiving the second user input and based on the first media content item, to display a public profile associated with the first media content item; and providing, based on the determining, for display of the public profile.

Abstract: In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products for linking to a search result. A computing system receives a query and provides, to a first computing device, multiple results that are responsive to the query. The computing system provides, to the first computing device, information that identifies a web address of a user selected result of multiple results that are responsive to query. The computing system receives a request from a second computing device for content responsive to the web address of the user selected result. The computing system provides, to the second computing device, information to cause the second computing device to present the multiple results that are responsive to the query in distinction to the user selected result, responsive to the second computing device not being authorized to access the user selected result.

Abstract: A method for processing a system image, which represents an image of a file system. Each file in the file system is assigned an attribute, this attribute representing a last access time to this file, Initially the attributes of all files are set to a predefined value. The system image is executed, and then all attributes being checked. Finally the files are deleted whose attribute has the preset value.

Abstract: The disclosure provides a cloud computing-based adaptive storage layering system and method. The system includes a data node management module, a metadata management module, an adaptive storage layering module, and a pre-aggregated query routing module. According to predefined rules, node capacity is expanded and shrunk, collected queried hit models and scanned file paths are aggregated and sorted, and layering loading and pre-loading are performed on files. Based on the disclosure, an efficient On-Line Analytical Processing (OLAP) query execution engine may be constructed, to deal with complex OLAP queries of various reporting system. Therefore, the execution efficiency of cloud OLAP engines can be significantly enhanced.

Abstract: A device receives a request to display a learning module. The device receives sensor values from a set of one or more sensor devices attached to a patient. The device displays a set of survey questions. The device receives user-specified responses to the set of survey questions. The device selects a first learning module from a plurality of learning modules based on the sensor values and the user-specified responses to the set of survey questions. The device displays the first learning module on the display in response to receiving the request to displaying the learning module.

Abstract: A method is used for managing a flow of data in at least one database, wherein said database is configured with at least two data models of data storage. In said method, during a first period of time, a first data flow portion is received in a computer, and the first data flow portion is then stored in a first data pool of the database according to the first data model. Then, after the first period of time, a transformation is made on the first portion of the data and the transformed first data is assigned to a second data model, and the first data flow portion is then transferred from the first data pool to a second data pool. The process is repeated with at least a second data flow portion and can be extended to more transformations, and, thus, to more data models, and more data pools accordingly.

Abstract: In some examples, a system receives, as part of a service to update a data store, a collection of operations that are to be performed with respect to the data store, wherein the collection of operations comprises a plurality of different types of operations for respective data blocks of the data store. Multiple operations of a respective type of the different types of operations are batched to produce a batch of operations of the respective type, and the batch of operations of the respective type is flushed to the data store. The system progressively updates a tracking data structure that tracks batching and flushing of operations of the collection of operations during the service, where the tracking data structure includes metrics representing flushes of batches of operations of the different types. After an interruption of the service, the system resumes the service using a checkpointed version of a resume portion of the tracking data structure.

Abstract: Systems, methods, and computer readable storage media that may be used to receive, by a publishing platform, one or more first data events associated with first content of a first content provider and one or more second data events associated with second content of a second content provider, generate, by the publishing platform, multiplexed data by multiplexing the one or more first data vents and the one or more second data events, and communicate, by the publishing platform, the multiplexed data to a de-multiplexer of an analytics system, wherein the de-multiplexer de-multiplexes the multiplexed data into the one or more first data events and the one or more second data events, communicates the one or more first data events to a first endpoint associated with the first content provider, and communicates the one or more second data events to a second endpoint associated with the second content provider.