Abstract: Systems and methods for optimizing thread allocation in a model serving system include estimating a batch size for inference requests. An optimal configuration is then determined that defines a number of inference instances, a number of threads per inference instance, and a sub-batch size per inference instance for processing a batch of inference requests of the batch size using intra-operator parallelism that minimizes average per-batch latency. The optimal configuration is determined with reference to a plurality of predetermined model profiles that define single-inference average batch latencies for different combinations of thread counts and batch sizes, the predetermined model profiles being used as input to a dynamic programming algorithm that identifies optimal configurations that minimize the average per-batch latency.

Abstract: The present disclosure relates to methods and systems that create a lineage graph that tracks provenance information across machine learning models. The methods and systems use the lineage graph to facilitate machine learning model testing, diagnostics, and updating. The methods and system also use the lineage graph to determine a storage optimization for reducing a storage footprint of the machine learning models.

Abstract: Layers of a deep neural network (DNN) are partitioned into stages using a profile of the DNN. Each of the stages includes one or more of the layers of the DNN. The partitioning of the layers of the DNN into stages is optimized in various ways including optimizing the partitioning to minimize training time, to minimize data communication between worker computing devices used to train the DNN, or to ensure that the worker computing devices perform an approximately equal amount of the processing for training the DNN. The stages are assigned to the worker computing devices. The worker computing devices process batches of training data using a scheduling policy that causes the workers to alternate between forward processing of the batches of the DNN training data and backward processing of the batches of the DNN training data. The stages can be configured for model parallel processing or data parallel processing.

Abstract: The systems may include receiving, by a processor, transaction information for a transaction, wherein the transaction information comprises a transaction amount; matching, by the processor, the transaction information with a transaction type; retrieving, by the processor, a plurality of possible charge types associated with the transaction type; comparing, by the processor, the transaction information with the plurality of possible charge types; separating, by the processor, the transaction amount of the transaction information into at least one individual charge amount; and/or identifying, by the processor, a charge type of the plurality of possible charge types associated with the at least one individual charge amount.

Abstract: A system for generating and delivering custom data sets in a big data environment may receive a preselected schema that identifies a plurality of columns from a plurality of data sources for inclusion in an output data file. The system reads data from the data sources to generate a data file containing a big data table. The system monitors the plurality of data sources to detect that the data sources have been ingested into a data storage system. The data file is read and a column is filtered from the data file to generate the output data file in response to the preselected schema excluding the column. The output data file is transferred to a client device.

Abstract: The systems may include receiving, by a processor, transaction information for a transaction, wherein the transaction information comprises a transaction amount; matching, by the processor, the transaction information with a transaction type; retrieving, by the processor, a plurality of possible charge types associated with the transaction type; comparing, by the processor, the transaction information with the plurality of possible charge types; separating, by the processor, the transaction amount of the transaction information into at least one individual charge amount; and/or identifying, by the processor, a charge type of the plurality of possible charge types associated with the at least one individual charge amount.

Abstract: Layers of a deep neural network (DNN) are partitioned into stages using a profile of the DNN. Each of the stages includes one or more of the layers of the DNN. The partitioning of the layers of the DNN into stages is optimized in various ways including optimizing the partitioning to minimize training time, to minimize data communication between worker computing devices used to train the DNN, or to ensure that the worker computing devices perform an approximately equal amount of the processing for training the DNN. The stages are assigned to the worker computing devices. The worker computing devices process batches of training data using a scheduling policy that causes the workers to alternate between forward processing of the batches of the DNN training data and backward processing of the batches of the DNN training data. The stages can be configured for model parallel processing or data parallel processing.

Abstract: The systems may include receiving, by a processor, transaction information for a transaction, wherein the transaction information comprises a transaction amount; matching, by the processor, the transaction information with a transaction type; retrieving, by the processor, a plurality of possible charge types associated with the transaction type; comparing, by the processor, the transaction information with the plurality of possible charge types; separating, by the processor, the transaction amount of the transaction information into at least one individual charge amount; and/or identifying, by the processor, a charge type of the plurality of possible charge types associated with the at least one individual charge amount.

Abstract: A system for generating and delivering custom data sets in a big data environment may receive a preselected schema that identifies a plurality of columns from a plurality of data sources for inclusion in an output data file. The system reads data from the data sources to generate a data file containing a big data table. The system monitors the plurality of data sources to detect that the data sources have been ingested into a data storage system. The data file is read and a column is filtered from the data file to generate the output data file in response to the preselected schema excluding the column. The output data file is transferred to a client device.