Abstract: A model comprises different processing nodes that perform different functions on input data, such as data prompts. Each node has batching criteria including a threshold value that when exceeded triggers a transmission of a batch of processing requests to the corresponding node. Different processing requests for a node are routed to a batching cache for that node. Processing requests are prevented from being routed to the node until the batching criteria is met. Once the batching criteria is met, a batch of processing requests stored in the batching cache are routed to the corresponding node.

Abstract: A meta-model topology comprises a plurality of functions and conforms to a global label schema. A new function not included in the plurality of functions is integrated into the meta-model topology. A particular label of interest that is associated with the new function is identified and the new function is configured such that an output from the new function conforms to an output form corresponding to the particular label of interest from the global label schema. The new function is then integrated into the meta-model topology and the meta-model topology that includes the new function is used to generate a model graph. The model graph is then deployed to a remote application that is configured to receive data prompts comprising input data processed by nodes of the model graph.

Abstract: A model graph receives a data prompt as input. The data prompt is segmented into multiple segments. An instance of the model graph is generated for each segment of the data prompt. Each instance of the model graph is also pruned according to policy information associated with the model graph instance's corresponding data prompt segment. Each instance of the model graph generates an intermediary output. A final output of the model graph for the entire data prompt is generated based on a combination of the intermediary outputs.

Abstract: A model comprises different processing nodes that perform different functions on input data, such as data prompts. Each node has batching criteria including a threshold value that when exceeded triggers a transmission of a batch of processing requests to the corresponding node. Different processing requests for a node are routed to a batching cache for that node. Processing requests are prevented from being routed to the node until the batching criteria is met. Once the batching criteria is met, a batch of processing requests stored in the batching cache are routed to the corresponding node.

Abstract: A meta-model topology comprises a plurality of functions and conforms to a global label schema. A new function not included in the plurality of functions is integrated into the meta-model topology. A particular label of interest that is associated with the new function is identified and the new function is configured such that an output from the new function conforms to an output form corresponding to the particular label of interest from the global label schema. The new function is then integrated into the meta-model topology and the meta-model topology that includes the new function is used to generate a model graph. The model graph is then deployed to a remote application that is configured to receive data prompts comprising input data processed by nodes of the model graph.

Abstract: A concurrent data structure allows synchronization to be elided for read accesses. Processing resources that remove one or more elements of the concurrent data structure are allowed to delete the elements only after all other processing resources have reached a safe point. Each processing resource maintains an indicator that indicates whether the processing resource has reached as safe point (i.e., will not access the concurrent data structure). When the indicators indicate that all processing resources have reached a safe point, elements of the data structure may be deleted.

Abstract: An interface between a resource manager and schedulers in a process executing on a computer system allows the resource manager to manage the resources of the schedulers. The resource manager communicates with the schedulers using the interface to access statistical information from the schedulers. The statistical information describes the amount of use of the resources by the schedulers. The resource manager also communicates with the schedulers to dynamically allocate and reallocate resources among the schedulers in the same or different processes or computer systems in accordance with the statistical information.

Abstract: Systems, methods, and computer program products for facilitating a serial debugging experience for programs running in a distributed, non-deterministic, out-of-order (i.e., parallel) fashion are disclosed. Such systems, methods, and computer program products provide a modified combination of breakpoints, stepping, and call stacks that facilitate the ability to improve the debugging experience for out-of-order program execution. In an embodiment, a task scheduler provides a distributed task graph. Then, the user may hit breakpoints sequentially within the graph's execution and use deterministic stepping (i.e., step-in, step-out, and step-over functions) to create a logical call stack from a distributed task graph. Such embodiments allow the program to execute in a distributed and parallel manner, while presenting the user a sequential execution experience in the debugger.

Abstract: Each processing resource in a scheduler of a process executing on a computer system maintains counts of the number of tasks that arrive at the processing resource and the number of tasks that complete on the processing resource. The counts are maintained in storage that is only writeable by the corresponding processing resource. The scheduler collects and sums the counts from each processing resource and provides statistics based on the summed counts and previous summed counts to a resource manager in response to a request from the resource manager. The scheduler does not reset the counts when the counts are collected and stores copies of the summed counts for use with the next request from the resource manager. The counts may be maintained without synchronization and with thread safety to minimize the impact of gathering statistics on the application.

Abstract: A resource manager manages processing and other resources of schedulers of one or more processes executing on one or more computer systems. For each scheduler, the resource manager determines an initial allocation of resources based on the policy of the scheduler, the availability of resources, and the policies of other schedulers. The resource manager receives feedback from the schedulers and dynamically changes the allocation of resources of schedulers based on the feedback. The resource manager determines if changes improved the performance of schedulers and commits or rolls back the changes based on the determination.

Abstract: Systems, methods, and computer program products for facilitating a serial debugging experience for programs running in a distributed, non-deterministic, out-of-order (i.e., parallel) fashion are disclosed. Such systems, methods, and computer program products provide a modified combination of breakpoints, stepping, and call stacks that facilitate the ability to improve the debugging experience for out-of-order program execution. In an embodiment, a task scheduler provides a distributed task graph. Then, the user may hit breakpoints sequentially within the graph's execution and use deterministic stepping (i.e., step-in, step-out, and step-over functions) to create a logical call stack from a distributed task graph. Such embodiments allow the program to execute in a distributed and parallel manner, while presenting the user a sequential execution experience in the debugger.

Abstract: A process for propagating an error in a floating-point calculation is disclosed. A floating-point error occurring from the floating-point arithmetic calculation is trapped, and a special value is generated. Information regarding the error is stored as a payload of the special value. Program operations are resumed with the special value applied to further calculations dependent on the floating-point arithmetic calculation.

Abstract: Embodiments are directed to dynamically allocating processing resources among a plurality of resource schedulers. A resource manager dynamically allocates resources to a first resource scheduler. The resource manager is configured to dynamically allocate resources among a plurality of resource schedulers, and each scheduler is configured to manage various processing resources. The resource manager determines that at least one of the processing resources dynamically allocated to the first resource scheduler is idle. The resource manager determines that at least one other resource scheduler needs additional processing resources and, based on the determination, loans the determined idle processing resource of the first resource scheduler to a second resource scheduler.

Abstract: Embodiments are directed to dynamically allocating processing resources among a plurality of resource schedulers. A resource manager dynamically allocates resources to a first resource scheduler. The resource manager is configured to dynamically allocate resources among a plurality of resource schedulers, and each scheduler is configured to manage various processing resources. The resource manager determines that at least one of the processing resources dynamically allocated to the first resource scheduler is idle. The resource manager determines that at least one other resource scheduler needs additional processing resources and, based on the determination, loans the determined idle processing resource of the first resource scheduler to a second resource scheduler.

Abstract: An interface between a resource manager and schedulers in a process executing on a computer system allows the resource manager to manage the resources of the schedulers. The resource manager communicates with the schedulers using the interface to access statistical information from the schedulers. The statistical information describes the amount of use of the resources by the schedulers. The resource manager also communicates with the schedulers to dynamically allocate and reallocate resources among the schedulers in the same or different processes or computer systems in accordance with the statistical information.

Abstract: A concurrent data structure allows synchronization to be elided for read accesses. Processing resources that remove one or more elements of the concurrent data structure are allowed to delete the elements only after all other processing resources have reached a safe point. Each processing resource maintains an indicator that indicates whether the processing resource has reached as safe point (i.e., will not access the concurrent data structure). When the indicators indicate that all processing resources have reached a safe point, elements of the data structure may be deleted.

Abstract: A resource manager manages processing and other resources of schedulers of one or more processes executing on one or more computer systems. For each scheduler, the resource manager determines an initial allocation of resources based on the policy of the scheduler, the availability of resources, and the policies of other schedulers. The resource manager receives feedback from the schedulers and dynamically changes the allocation of resources of schedulers based on the feedback. The resource manager determines if changes improved the performance of schedulers and commits or rolls back the changes based on the determination.

Abstract: Each processing resource in a scheduler of a process executing on a computer system maintains counts of the number of tasks that arrive at the processing resource and the number of tasks that complete on the processing resource. The counts are maintained in storage that is only writeable by the corresponding processing resource. The scheduler collects and sums the counts from each processing resource and provides statistics based on the summed counts and previous summed counts to a resource manager in response to a request from the resource manager. The scheduler does not reset the counts when the counts are collected and stores copies of the summed counts for use with the next request from the resource manager. The counts may be maintained without synchronization and with thread safety to minimize the impact of gathering statistics on the application.