Abstract: A method optimizes a training of a machine learning system. A conflict detection system discovers a conflict between a first training data and a second training data for a machine learning system, where the first training data and the second training data are ground truths that describe a same type of entity, and where the first training data and the second training data have different labels. In response to discovering the conflict between the first training data and the second training data for the machine learning system, an oracle adjusts the different labels of the first training data and the second training data. The machine learning system is then trained using the first training data and the second training data with the adjusted labels.

Abstract: A method for automatically generating documentation for an artificial intelligence model includes receiving, by a computing device, an artificial intelligence model. The computing device accesses a model facts policy that indicates data to be collected for artificial intelligence models. The computing device collects artificial intelligence model facts regarding the artificial intelligence model according to the model facts policy. The computing device accesses a factsheet template. The factsheet template provides a schema for an artificial intelligence model factsheet for the artificial intelligence model. The computing device populates the artificial intelligence model factsheet using the factsheet template with the artificial intelligence model facts related to the artificial intelligence model.

Abstract: A method for automatically generating documentation for an artificial intelligence model includes receiving, by a computing device, an artificial intelligence model. The computing device accesses a model facts policy that indicates data to be collected for artificial intelligence models. The computing device collects artificial intelligence model facts regarding the artificial intelligence model according to the model facts policy. The computing device accesses a factsheet template. The factsheet template provides a schema for an artificial intelligence model factsheet for the artificial intelligence model. The computing device populates the artificial intelligence model factsheet using the factsheet template with the artificial intelligence model facts related to the artificial intelligence model.

Abstract: A method optimizes a training of a machine learning system. A conflict detection system discovers a conflict between a first training data and a second training data for a machine learning system, where the first training data and the second training data are ground truths that describe a same type of entity, and where the first training data and the second training data have different labels. In response to discovering the conflict between the first training data and the second training data for the machine learning system, an oracle adjusts the different labels of the first training data and the second training data. The machine learning system is then trained using the first training data and the second training data with the adjusted labels.

Abstract: Running a global production rule on data distributed over a plurality of machines may comprise receiving a local production rule that can run on each of the plurality of machines to jointly accomplish a global computation specified by the global production rule. The local production rule may be deployed to each of the plurality of machines, each of which stores a portion of the data and runs an instance of a rules engine that can run the local production rule. The plurality of machines are enabled to communicate intermediate data produced by the instance of the rules engine running the local production rule on said each of the machines. Coordinating between the plurality of machines is enabled to synchronize one or more local computations performed locally according to the local production rule on said each machine.

Abstract: Running a global production rule on data distributed over a plurality of machines may comprise receiving a local production rule that can run on each of the plurality of machines to jointly accomplish a global computation specified by the global production rule. The local production rule may be deployed to each of the plurality of machines, each of which stores a portion of the data and runs an instance of a rules engine that can run the local production rule. The plurality of machines are enabled to communicate intermediate data produced by the instance of the rules engine running the local production rule on said each of the machines. Coordinating between the plurality of machines is enabled to synchronize one or more local computations performed locally according to the local production rule on said each machine.

Abstract: Running a global production rule on data distributed over a plurality of machines may comprise receiving a local production rule that can run on each of the plurality of machines to jointly accomplish a global computation specified by the global production rule. The local production rule may be deployed to each of the plurality of machines, each of which stores a portion of the data and runs an instance of a rules engine that can run the local production rule. The plurality of machines are enabled to communicate intermediate data produced by the instance of the rules engine running the local production rule on said each of the machines. Coordinating between the plurality of machines is enabled to synchronize one or more local computations performed locally according to the local production rule on said each machine.

Abstract: Running a global production rule on data distributed over a plurality of machines may comprise receiving a local production rule that can run on each of the plurality of machines to jointly accomplish a global computation specified by the global production rule. The local production rule may be deployed to each of the plurality of machines, each of which stores a portion of the data and runs an instance of a rules engine that can run the local production rule. The plurality of machines are enabled to communicate intermediate data produced by the instance of the rules engine running the local production rule on said each of the machines. Coordinating between the plurality of machines is enabled to synchronize one or more local computations performed locally according to the local production rule on said each machine.

Abstract: A programming language support for debugging heap related errors includes one or more queries for determining one or more global properties associated with use of the area by the program. The one or more queries may be executed in parallel or concurrently and dynamically utilize available number of cores.

Abstract: Execution states of tasks are inferred from collection of information associated with runtime execution of a computer system. Collection of information may include infrequent samples of executing tasks, the samples which may provide inaccurate executing states. One or more tasks may be aggregated by one or more execution states for determining execution time, idle time, or system policy violations, or combinations thereof.

Abstract: Detecting defects in deployed systems, in one aspect, identify one or more monitoring agents used in a computer program. Total execution metric of the computer program and execution metric associated with the one or more monitoring agents are measured and the measure execution metric is compared with a specified overhead criteria. The execution of the one or more monitoring agents is adjusted based on the comparing step while the computer program is executing to meet the specified overhead criteria.

Abstract: Execution states of tasks are inferred from collection of information associated with runtime execution of a computer system. Collection of information may include infrequent samples of executing tasks, the samples which may provide inaccurate executing states. One or more tasks may be aggregated by one or more execution states for determining execution time, idle time, or system policy violations, or combinations thereof.

Abstract: A programming language support for debugging heap related errors includes one or more queries for determining one or more global properties associated with use of the area by the program. The one or more queries may be executed in parallel or concurrently and dynamically utilize available number of cores.

Abstract: Detecting defects in deployed systems, in one aspect, identify one or more monitoring agents used in a computer program. Total execution metric of the computer program and execution metric associated with the one or more monitoring agents are measured and the measure execution metric is compared with a specified overhead criteria. The execution of the one or more monitoring agents is adjusted based on the comparing step while the computer program is executing to meet the specified overhead criteria.

Abstract: A method for evaluating software performance includes steps of: receiving a plurality of versions of code; selecting starting and stopping points for timing execution of the code versions; dispatching at least two of the plurality of code versions for execution; repeatedly executing the at least two code versions; recording execution times for the at least two code versions, according to the selected starting and stopping points; collecting the execution times; and processing the collected execution times. The method further includes steps of: performing a statistical analysis of the collected execution times for determining which code version is fastest; and invoking a confidence metric periodically to determine if a difference between means is statistically meaningful.

Abstract: A system, method, and computer readable medium, for automatically improving performance of, and optimizing, a program based on on-line profile data of the program and profile data (302) collected across multiple runs of the program and stored in a persistent off-line repository (114). The method includes executing a program in an execution environment. Profile data (302) is collected for the program across multiple runs thereof. The performance of the program is improved, such as by optimization of the program, based on on-line profile data of the executing program and the collected profile data in the persistent off-line repository.

Abstract: A system and method for characterizing runtime behavior of a computer program executing in an execution environment, the method comprising: identifying one or more instances of yield points in a program to be executed, each yield point indicating a potential sampling operation during program execution; during program execution, in response to an identified yield point instance, ascertaining a state of the execution environment for indicating whether a sampling operation is to be performed; and, when the state of the execution environment indicates a sampling operation, recording relevant information for characterizing behavior of the execution environment. Relevant information for characterizing program behavior includes frequencies of methods executed in the program, and calling context associated with methods called by the program.

Abstract: A sampling-based system and method for adaptively optimizing a computer program executing in an execution environment that comprises one or more compiler devices for providing various levels of program optimization. The system comprises a runtime measurements sub-system for monitoring execution of the computer program to be optimized, the monitoring including obtaining raw profile data samples and characterizing the raw profile data; a controller device for receiving the characterized raw profile data from the runtime measurements sub-system and analyzing the data for determining whether a level of program optimization for the executing program is to be performed by a compiler device, the controller generating a compilation plan in accordance with a determined level of optimization; and, a recompilation sub-system for receiving a compilation plan from the controller and invoking a compiler device for performing the level of program optimization of the executing program in accordance with the compilation plan.

Abstract: A method for characterizing runtime behavior of a computer program executing in an execution environment comprising: generating a call stack runtime data structure for tracking methods currently active in an executing program thread, an active method on the call stack is represented by a frame; determining condition for sampling an executing program to determine current program behavior; and, upon determination of a sampling condition, the sampling including examining at least one frame in the call stack in response to evaluate context of one or more methods represented in the call stack, the at least one frame in the call stack providing context relating to an executing program's calling structure.