Abstract: Input sequence information may be analyzed and quantified using n-gram analysis of inputs received by an application. The sequences of inputs may be represented by n-grams, and the frequency of the various n-grams may indicate the ‘real world’ uses of the application in production, which may be compared to a test suite whose coverage may be quantified using a similar n-gram analysis. A coverage factor may compare the observed inputs to the application in production to the test suite for the application. The n-grams may be further quantified or prioritized by resource utilization and several visualizations may be generated from the data.

Abstract: Comparisons of different versions of an application may be compared using a behavior model of the application. A behavior model may be derived from n-gram analysis of observations of the application in production. The behavior model may include sequences of inputs received by the application or functions performed by the application, where each sequence is an n-gram observed in tracer data. Each n-gram may be coupled with a resource consumption to give a behavior model with performance data. A regression analysis may apply a behavior model derived from a first version of an application to the performance observations of a new version to create an expected performance metric for the new version. A similarly calculated metric from a previous version may be compared to the metric from a new version to determine an improvement or degradation of performance.

Abstract: A behavior model for a software application may identify a set of execution sequences that begin from a set of origins. The sequences may be further defined by a set of exits. In some cases, the sequences may be decomposed into subsequences or n-grams. The execution sequences and their frequencies may define a usage or behavior model for the application. The sequences may be defined by semantic level operations of an application, which may be defined by functions, call backs, API calls, or other blocks of code execution. The behavior model may be used for determining code coverage, comparing versions of applications, and other uses.

Abstract: N-grams of input streams or functions executed by an application may be analyzed to identify security breaches or other anomalous behavior. A histogram of n-grams representing sequences of executed functions or input streams may be generated through baseline testing or production use. An alerting system may compare real time n-gram observations to the histogram of n-grams to identify security breaches or other changes in application behavior that may be anomalous. An alert may be generated that identifies the anomalous behavior. The alerting system may be trained using known good datasets and may identify deviations as bad behavior. The alerting system may be trained using known bad datasets and may identify matching behavior as bad behavior.

Abstract: Regression testing of an application may gather performance tests for multiple functions within an application and determine when performance changes from one version of the application to another. The analysis may be further broken down by input sequences that may be processed by various functions. A detailed regression analysis may be presented as a heat map or other visualizations. A regression testing system may be launched during a build process by automatically launching a set of performance tests against an application. In many cases, the application may be executed in a system with a known or consistent performance capabilities. The application may be executed and tested in a new version and at least one prior version on the same hardware and software execution environment, so that results may be normalized from one execution run to another. A regression testing system may be deployed as a paid-for service that may integrate into a source code repository.

Abstract: Execution sequence information may be analyzed and quantified using n-gram analysis of functions executed by an application. The sequences of functions may be represented by n-grams, and the frequency of the various n-grams may indicate the behavior of the application in production, which may be compared to a test suite whose coverage may be quantified using a similar n-gram analysis. A coverage factor may compare the observed behavior of the application in production to the test suite for the application. The n-grams may be further quantified or prioritized by resource utilization, and several visualizations may be generated from the data.

Abstract: Optimized memory management settings may be derived from a mathematical model of an execution environment. The settings may be optimized for each application or workload, and the settings may be implemented per application, per process, or with other granularity. The settings may be determined after an initial run of a workload, which may observe and characterize the execution. The workload may be executed a second time using the optimized settings. The settings may be stored as tags for the executable code, which may be in the form of a metadata file or as tags embedded in the source code, intermediate code, or executable code. The settings may change the performance of memory management operations in both interpreted and compiled environments. The memory management operations may include memory allocation, garbage collection, and other related functions.

Abstract: Processes may be assigned to specific processors when memory objects consumed by the processes are located in memory banks closely associated with the processors. When assigning processes to threads operating in a multiple processor NUMA architecture system, an analysis of the memory objects accessed by a process may identify processor or group of processors that may minimize the memory access time of the process. The selection may take into account the connections between memory banks and processors to identify the shortest communication path between the memory objects and the process. The processes may be pre-identified as functional processes that make little or no changes to memory objects other than information passed to or from the processes.

Abstract: An execution environment may have a monitoring, analysis, and feedback loop that may configure and tune the execution environment for currently executing workloads. A monitoring or instrumentation system may collect operational and performance data from hardware and software components within the system. A modeling system may create an operational model of the execution environment, then may determine different sets of parameters for the execution environment. A feedback loop may change various operational characteristics of the execution environment. The monitoring, analysis, and feedback loop may optimize the performance of a computer system for various metrics, including throughput, performance, energy conservation, or other metrics based on the applications that are currently executing. The performance model of the execution environment may be persisted and applied to new applications to optimize the performance of applications that have not been executed on the system.

Abstract: A tracing and debugging system may collect both performance related tracer data and snapshot data. The tracer data may contain aggregated performance and operational data, while the snapshot data may contain call stack, source code, and other information that may be useful for debugging and detailed understanding of an application. The snapshot data may be stored in a separate database from the tracer data, as the snapshot data may contain data that may be private or sensitive, while the tracer data may be aggregated information that may be less sensitive. A debugging user interface may be used to access, display, and browse the stored snapshot data.

Abstract: A tracing system may divide trace objectives across multiple instances of an application, then deploy the objectives to be traced. The results of the various objectives may be aggregated into a detailed tracing representation of the application. The trace objectives may define specific functions, processes, memory objects, events, input parameters, or other subsets of tracing data that may be collected. The objectives may be deployed on separate instances of an application that may be running on different devices. In some cases, the objectives may be deployed at different time intervals. The trace objectives may be lightweight, relatively non-intrusive tracing workloads that, when results are aggregated, may provide a holistic view of an application's performance.

Abstract: A tracing system may use an evaluation mechanism to determine which functions to include or exclude during tracing. The architecture may evaluate functions when functions or groups of functions may be loaded for execution, as well as each time a function may be encountered. The evaluation mechanism may use whitelists, blacklists, and various expressions to identify which functions to trace and which functions to exclude. The evaluation mechanism may evaluate an expression that may identify specific conditions under which a function may be traced or not traced. The tracing mechanism may create wrapping functions for each function, including callback functions.

Abstract: A tracer may obfuscate trace data such that the trace data may be used in an unsecure environment even though raw trace data may contain private, confidential, or other sensitive information. The tracer may obfuscate using irreversible or lossy hash functions, look up tables, or other mechanisms for certain raw trace data, rendering the obfuscated trace data acceptable for transmission, storage, and analysis. In the case of parameters passed to and from a function, trace data may be obfuscated as a group or as individual parameters. The obfuscated trace data may be transmitted to a remote server in some scenarios.

Abstract: An execution environment for functional code may treat application segments as individual programs for memory management. A larger program of application may be segmented into functional blocks that receive an input and return a value, but operate without changing state of other memory objects. The program segments may have memory pages allocated to the segments by the operating system as other full programs, and may deallocate memory pages when the segments finish operating. Functional programming languages and imperative programming languages may define program segments explicitly or implicitly, and the program segments may be identified at compile time or runtime.

Abstract: Periodicity similarity between two different tracer objectives may be used to identify additional input parameters to sample. The tracer objectives may be individual portions of a large tracer operation, and each of the tracer objectives may have separate set of input objects for which data may be collected. After collecting data for a tracer objective, other tracer objectives with similar periodicities may be identified. The input objects from the other tracer objectives may be added to a tracer objective and the tracer objective may be executed to determine a statistical significance of the newly added objective. An iterative process may traverse multiple input objects until exhausting possible input objects and a statistically significant set of input objects are identified.

Abstract: Code elements may be selected from a graph depicting an application. The graph may show code elements as nodes, with edges representing connections between the nodes. The connections may be messages passed between code elements, code flow relationships, or other relationships. When a code element or group of code elements are selected from the graph, the corresponding source code may be displayed. The code may be displayed in a code editor or other mechanism by which the code may be viewed, edited, and manipulated.

Abstract: Error logs, bug reports, and other databases identifying problems with a tracer system may be mined to determine how a tracer may interact with a given function, module, or other group of functions. Based on such reports, a tracer may be configured to avoid certain functions or to trace such functions in a specific manner. In some cases, tracer may be configured to limit tracing to certain parameters or with other limitations to avoid any known conditions under which errors occur.

Abstract: A tracing system may be updated to include, exclude, or modify tracing configurations for functions based on how a user consumes tracing results. The user's interactions with graphical representations, inspections of data, and other interactions may indicate which functions may be interesting and which functions may not be. The user's interactions may be classified by use, such as during debugging, performance testing, and ongoing monitoring, and multiple user's interactions with the same function, library, module, source code file, or other groups of functions may be combined to predict a user's interest in a function.

Abstract: Optimized memory management settings may be derived from a mathematical model of an execution environment. The settings may be optimized for each application or workload, and the settings may be implemented per application, per process, or with other granularity. The settings may be determined after an initial run of a workload, which may observe and characterize the execution. The workload may be executed a second time using the optimized settings. The settings may be stored as tags for the executable code, which may be in the form of a metadata file or as tags embedded in the source code, intermediate code, or executable code. The settings may change the performance of memory management operations in both interpreted and compiled environments. The memory management operations may include memory allocation, garbage collection, and other related functions.

Abstract: Processes in a message passing system may be launched when messages having data patterns match a function on a receiving process. The function may be identified by an execution pointer within the process. When the match occurs, the process may be added to a runnable queue, and in some embodiments, may be raised to the top of a runnable queue. When a match does not occur, the process may remain in a blocked or non-executing state. In some embodiments, a blocked process may be placed in an idle queue and may not be executed until a process scheduler determines that a message has been received that fulfills a function waiting for input. When the message fulfills the function, the process may be moved to a runnable queue.