Abstract: A system and methods for queuing processing jobs. The system is configured to receive a processing job associated with a user; score the processing job; apply one or more bounds; add the processing job to a queue; order the queue based on scores of processing jobs in the queue; and sample processing jobs from the queue for dispatch for processing. Sampling the jobs from the queues can comprise updating scores, updating application of bounds, and updating ordering of the queue; reading a batch of processing jobs from a top of the queue; attempting to dispatch the processing jobs of the batch for processing; removing from the queue any dispatched processing jobs; increasing the batch size by a constant factor; determining whether a maximum batch size is reached, or all processing jobs in the queue were in the batch.

Abstract: Systems and methods provide a first group of computation modules configured to provide back-end compute resources for a first type of computation module by back-end compute resources. Systems and methods provide a second group of computation modules that are configured to provide back-end compute resources for a second type of computation module wherein the first and second type of computation modules are of different types. In some examples, methods and systems identify the type of computation module needed by each of a plurality of queries associated with different types of client applications based on computation module type data associated with each query and route each of the plurality of queries to an appropriate computation module within the first or second computation modules based on the computation module type data. Related methods are also disclosed.

Abstract: Systems and methods provide a first group of computation modules configured to provide back-end compute resources for a first type of computation module by back-end compute resources. Systems and methods provide a second group of computation modules that are configured to provide back-end compute resources for a second type of computation module wherein the first and second type of computation modules are of different types. In some examples, methods and systems identify the type of computation module needed by each of a plurality of queries associated with different types of client applications based on computation module type data associated with each query and route each of the plurality of queries to an appropriate computation module within the first or second computation modules based on the computation module type data. Related methods are also disclosed.

Abstract: Systems and methods provide a first group of computation modules configured to provide back-end compute resources for a first type of computation module by back-end compute resources. Systems and methods provide a second group of computation modules that are configured to provide back-end compute resources for a second type of computation module wherein the first and second type of computation modules are of different types. In some examples, methods and systems identify the type of computation module needed by each of a plurality of queries associated with different types of client applications based on computation module type data associated with each query and route each of the plurality of queries to an appropriate computation module within the first or second computation modules based on the computation module type data. Related methods are also disclosed.

Abstract: A method and system for data pipeline monitoring receives an event data object and a current status data object from one or more subsystems of a pipeline. The system analyzes the event data object and the current status data object to determine a first and second validation value. The system, in response to determining that either the first or second validation value is not valid, sends a notification.

Abstract: A system and methods for determining computer resource allocation, the system having a network communication interface configured to receive a query from a device, the query indicating a request to perform a processing operation on a portion of one or more data set. The system may also include data storage for data including historical information related to processing of data sets by back-end computer resources, and hardware processors configured to determine one or more scores associated with a query and based at least in part on the historical information. The system may determine a particular back-end computer resource with a highest score, provide a compute request to the particular back-end computer resource to perform a processing operation on a portion of one or more data sets, and store processing information related to the processing of the compute request by the particular back-end computer resource as historical information.

Abstract: A system and methods for determining computer resource allocation, the system having a network communication interface configured to receive a query from a device, the query indicating a request to perform a processing operation on a portion of one or more data set. The system may also include data storage for data including historical information related to processing of data sets by back-end computer resources, and hardware processors configured to determine one or more scores associated with a query and based at least in part on the historical information. The system may determine a particular back-end computer resource with a highest score, provide a compute request to the particular back-end computer resource to perform a processing operation on a portion of one or more data sets, and store processing information related to the processing of the compute request by the particular back-end computer resource as historical information.

Abstract: A system and methods for determining computer resource allocation, the system having a network communication interface configured to receive a query from a device, the query indicating a request to perform a processing operation on a portion of one or more data set. The system may also include data storage for data including historical information related to processing of data sets by back-end computer resources, and hardware processors configured to determine one or more scores associated with a query and based at least in part on the historical information. The system may determine a particular back-end computer resource with a highest score, provide a compute request to the particular back-end computer resource to perform a processing operation on a portion of one or more data sets, and store processing information related to the processing of the compute request by the particular back-end computer resource as historical information.

Abstract: A method and system for data pipeline monitoring receives an event data object and a current status data object from one or more subsystems of a pipeline. The system analyzes the event data object and the current status data object to determine a first and second validation value. The system, in response to determining that either the first or second validation value is not valid, sends a notification.

Abstract: A method and system for data pipeline monitoring receives an event data object and a current status data object from one or more subsystems of a pipeline. The system analyzes the event data object and the current status data object to determine a first and second validation value. The system, in response to determining that either the first or second validation value is not valid, sends a notification.

Abstract: A method and system for data pipeline monitoring receives an event data object and a current status data object from one or more subsystems of a pipeline. The system analyzes the event data object and the current status data object to determine a first and second validation value. The system, in response to determining that either the first or second validation value is not valid, sends a notification.

Abstract: A method and system for data pipeline monitoring receives an event data object and a current status data object from one or more subsystems of a pipeline. The system analyzes the event data object and the current status data object to determine a first and second validation value. The system, in response to determining that either the first or second validation value is not valid, sends a notification.