Abstract: Embodiments are described for a data processing tool configured to cease operations of a plurality of database readers when detecting a congestion condition in the data processing tool. In some embodiments, the data processing tool comprises a memory, one or more processors, and a plurality of database readers. The one or more processors, coupled to the memory and the plurality of database readers are configured to determine a congestion condition in at least one data pipeline of a plurality of data pipelines of the data processing tool. Each data pipeline of the plurality of data pipelines connects a database reader and a transformer of the data processing tool, a transformer and a database writer of the data processing tool, or two transformers of the data processing tool. The one or more processors are further configured to refrain from reading data from one or more databases responsive to the congestion condition.

Abstract: Disclosed herein are system, method, and computer program product embodiments for performing deterministic execution of background jobs in a load-balanced system. An embodiment operates by receiving, at a work server in a load-balanced system, job submission code from a client connected to the work server, wherein the job submission code performs a background job for the client. The embodiment then executes, at the work server, the job submission code. The execution of the job submission code obtains a name of the work server executing the job submission code, maps the name of the work server to a logical server name, and submits the background job for background processing using a job processing function that executes the background job on the logical server name.

Abstract: A computer implemented system and method of performing parallel load operations during an extract, transform, load (ETL) operation. The system generates a single transaction having multiple processes, where each process loads into one target table. The processes are executed in parallel, and the system performs verification on the single transaction.

Abstract: Some embodiments include a system, method, and non-transitory medium, with the method including receiving, from a user, values for a set of usage-related parameters for one or more cloud services; determining, based on historical time series data for the one or more services provided by at least one cloud service provider, a predicted trend of future values for the set of usage-related parameters for the one or more cloud services for the at least one cloud service provider; evaluating the predicted trend of future values for the set of usage-related parameters for the one or more cloud services for each of the at least one cloud service providers to generate a recommendation that best corresponds, at least within a specified threshold, to the values for the set of usage-related parameters received from the user; and storing an indication of the generated recommendation in a data storage device.

Abstract: Embodiments leverage local data available from an on-premises Extract, Transfer, and Load (ETL) job, in order to efficiently perform remote processing (e.g., as implemented on the cloud). Connectivity data (e.g., target setup) and ETL logic (e.g., configuring data flattening, pivot transform, and/or data quality transform) is stored locally in a non-transitory computer readable storage medium. In response to receiving data transformed on-premises, the transformed data, connectivity data, and ETL logic are forwarded to a remote location for processing. Some embodiments may also forward the transformed data on to its original target on-premises (e.g., via a local database loader). Particular embodiments may provide hidden, Representational State Transfer (REST)-based loader(s) that duplicate output of the local ETL job. Embodiments conserve developer effort by allowing preparatory local ETL data that is already available on-premises, to be read and used for processing in a remote cloud intelligence system.

Abstract: Disclosed herein are system, method, and computer program product embodiments for managing data storage devices. In some embodiments, a server receives a request to store data in a first storage device. The server determines the storage space remaining in the first data storage device based on historical data associated with the first data storage device. The server further determines that the first data storage device will exceed its storage capacity based on a size of the data and the storage space remaining in the first data storage device. The server transfers a set of data stored in the first data storage device to a second storage device. Transferring the set of data causes the storage space remaining in the first storage device to be greater than or equal to the size of the data. The server stores the data in the first data storage device.

Abstract: A computer implemented system and method of performing parallel load operations during an extract, transform, load (ETL) operation. The system generates a single transaction having multiple processes, where each process loads into one target table. The processes are executed in parallel, and the system performs verification on the single transaction.

Abstract: Embodiments increase the efficiency of uploading data to a remote system. An expected volume of data for upload, is calculated prior to the actual load event. This calculation allows proactive allocation of resources (e.g., processing, memory, network, disk, others) available in the remote system, in order to accommodate the load being transferred. In certain embodiments, the calculation can be based upon historical data for previous load events. According to particular embodiments, the calculation can be based upon an estimated output data volume (such as parsing the output of EXPLAIN statement for PostgreSQL) or other information provided by data source(s) or an Extract-Transform-Load (ETL) tool. Some embodiments may perform the transfer by first staging data for upload in temporary table(s) on the side of the ETL tool, prior to actually loading to the remote system. In such embodiments, record count(s) can provide a precise value of expected data load volume.

Abstract: A computer implemented system and method of performing parallel load operations during an extract, transform, load (ETL) operation. The system generates a single transaction having multiple processes, where each process loads into one target table. The processes are executed in parallel, and the system performs verification on the single transaction.

Abstract: A computer implemented system and method of performing parallel load operations during an extract, transform, load (ETL) operation. The system generates a single transaction having multiple processes, where each process loads into one target table. The processes are executed in parallel, and the system performs verification on the single transaction.