Abstract: A tracing system may trace applications and their modules, and may make module-specific data available through various interfaces. The tracing system may collect tracer data while an application executes, and may preprocess the data into application-specific and module-specific databases. An analysis engine may further analyze and process these databases to create application-specific views and module-specific views into the data. The application-specific views may be intended for a developer of the application, while the module-specific views may have a public version accessible to everybody and a module developer version that may contain additional details that may be useful to the module developer.

Abstract: A database of module performance may be generated by adding tracing components to applications, as well as by adding tracing components to modules themselves. Modules may be reusable code that may be made available for reuse across multiple applications. When tracing is performed on an application level, the data collected from each module may be summarized in module-specific databases. The module-specific databases may be public databases that may assist application developers in selecting modules for various tasks. The module-specific databases may include usage and performance data, as well as stability and robustness metrics, error logs, and analyses of similar modules. The database may be accessed through links in module description pages and repositories, as well as through a website or other repository.

Abstract: A module-specific tracing mechanism may trace the usage of a module on behalf of the module developer. The module may be used by multiple application developers, and the tracing system may collect and summarize data for the module in each of the different applications. The data may include usage data as well as performance data. Usage data may include anonymized data for each time the module may be invoked and called, and performance data may include the processing time, memory consumption, and other metrics. The module-specific tracing may be enabled or disabled by an application developer.

Abstract: A load balanced system may incorporate instrumented systems within a group of managed devices and distribute workload among the devices to meet both load balancing and data collection. A workload distributor may communicate with and configure several managed devices, some of which may have instrumentation that may collect trace data for workload run on those devices. Authentication may be performed between the managed devices and the workload distributor to verify that the managed devices are able to receive the workloads and to verify the workloads prior to execution. The workload distributor may increase or decrease the amount of instrumentation in relation to the workload experienced at any given time.

Abstract: A timeline chart may represent multiple data sets gathered from multiple sequences of a process by placing sub-graphs within timeline bars. The sub-graphs may represent summarized data related to each event represented by a timeline bar. The timeline chart may present an overall view of a sequence of process steps with insights to the shape or distribution of the underlying observations. The timeline chart may be an instance of an event chain diagram, where the elements within the event chains are displayed with respect to time. The timeline chart may be presented as representing the aggregated dataset of multiple runs, as well as a representation of a single observed sequence. In both cases, sub-graphs may be included in a timeline bar to represent different views of the aggregated dataset.

Abstract: An event chain visualization of performance data may show the execution of monitored elements as bars on a timeline, with connections or other relationships connecting the various bars into a sequential view of an application. The visualization may include color, shading, or other highlighting to show resource utilization or performance metrics. The visualization may be generated by monitoring many events processed by an application, where each bar on a timeline may reflect multiple instances of a monitored element and, in some case, the aggregated performance.

Abstract: Tracer objectives in a distributed tracing system may be compared to identify input parameters that may have a high statistical relevancy. An iterative process may traverse multiple input objects by comparing results of multiple tracer objectives and scoring possible input objects as being possibly statistically relevant. With each iteration, statistically irrelevant input objects may be discarded from a tracer objective and other potentially relevant objects may be added. The iterative process may converge on a set of statistically relevant input objects for a given measured value without a priori knowledge of an application being traced.

Abstract: An application services marketplace may match various solution providers to a solution request, then creating a bid or proposal for services. Upon acceptance of the bid, the services may be configured and deployed. A market maker may combine multiple solution providers to address a specific request, and may use a schema expressly defined or implied in a request to select and configure a combination of services. The market maker may combine monitoring services with analysis services, monitoring services with load generators, or other combination of services that may be used during development or deployment of an application.

Abstract: A tracing system may perform cost analysis to identify burdensome or costly trace objectives. For a burdensome objective, two or more objectives may be created that can be executed independently. The cost analysis may include processing, storage, and network performance factors, which may be budgeted to collect data without undue performance or financial drains on the application under test. A larger objective may be recursively analyzed to break the larger objective into smaller objectives which may be independently deployed.

Abstract: An instrumented execution environment may connect to an execution environment to provide detailed tracing and logging of an application as it runs. The instrumented execution environment may be configured as a standalone service that can be configured and purchased. The instrumented execution environment may be deployed with various authentication systems, administrative user interfaces, and other components. The instrumented execution environment may engage a customer's system through a distributor that may manage a workload and distribute work to the instrumented execution environment as well as other worker systems. A marketplace may provide multiple preconfigured execution environments that may be selected, further configured, and deployed to address specific data collection objectives.

Abstract: A message passing compute environment may be visualized by illustrating messages passed within the environment. The messages may contain data consumed by a function or other computational element, and may be used to launch or spawn various computational elements. One visualization may be a force directed graph that has each function as a node, with messages passed as edges of the graph. In some embodiments, the edges may display the number of messages, quantity of data, or other metric by showing the edges as wider or thinner, or by changing the color of the displayed edge. The nodes may be illustrated with different colors, size, or shape to show different aspects. Some embodiments may have a mechanism for storing and playing back changes to the graph over time.

Abstract: A parallel tracer may perform detailed or heavily instrumented analysis of an application in parallel with a performance or lightly instrumented version of the application. Both versions of the application may operate on the same input stream, but with the heavily instrumented version having different performance results than the lightly instrumented version. The tracing results may be used for various analyses, including optimization and debugging.

Abstract: A load balanced system may incorporate instrumented systems within a group of managed devices and distribute workload among the devices to meet both load balancing and data collection. A workload distributor may communicate with and configure several managed devices, some of which may have instrumentation that may collect trace data for workload run on those devices. Authentication may be performed between the managed devices and the workload distributor to verify that the managed devices are able to receive the workloads and to verify the workloads prior to execution. The workload distributor may increase or decrease the amount of instrumentation in relation to the workload experienced at any given time.

Abstract: A tracer may obfuscate trace data such that the trace data may be used in an unsecure environment even though raw trace data may contain private, confidential, or other sensitive information. The tracer may obfuscate using irreversible or lossy hash functions, look up tables, or other mechanisms for certain raw trace data, rendering the obfuscated trace data acceptable for transmission, storage, and analysis. In the case of parameters passed to and from a function, trace data may be obfuscated as a group or as individual parameters. The obfuscated trace data may be transmitted to a remote server in some scenarios.

Abstract: An application development environment may have a user interface to a marketplace for development related services, such as monitoring, debugging, load generating, analysis, and other services. Service providers may make their products available through the marketplace, and in some cases, the providers may bid for placement in the user interface. The services may be paid or free, and a clearinghouse may handle financial transactions that may occur. The application development environment may include an editor, debugger, compiler, and other tools by which a developer may write, edit, test, and debug an application. The marketplace may detect characteristics about the application under development, and make those characteristics available to various service providers.

Abstract: A load generator services marketplace may configure and deploy load generators in conjunction with executing an application. The load generators may be selected based on a solution definition, which may include the types of loads and conditions under which loads may be generated. One or more load generators may be configured to operate with a monitoring service, and a connection manager may cause the load generators, application, and monitoring service to execute simultaneously so that the monitoring service may capture performance metrics while the application experiences the load. The marketplace may have load generators from multiple providers and with multiple configurations, as well as a clearinghouse for clearing a financial transaction as the load generators are used.

Abstract: A tracing management system may use cost analyses and performance budgets to dispatch tracing objectives to instrumented systems that may collect trace data while running an application. The tracing management system may analyze individual tracing workloads for processing, storage, and network performance costs, and select workloads to deploy based on a resource budget that may be set for a particular device. In some cases, complementary tracing objectives may be selected that maximize consumption of resources within an allocated budget. The budgets may allocate certain resources for tracing, which may be a mechanism to limit any adverse effects from tracing when running an application.

Abstract: A tracing system may perform cost analysis to identify burdensome or costly trace objectives. For a burdensome objective, two or more objectives may be created that can be executed independently. The cost analysis may include processing, storage, and network performance factors, which may be budgeted to collect data without undue performance or financial drains on the application under test. A larger objective may be recursively analyzed to break the larger objective into smaller objectives which may be independently deployed.

Abstract: A tracing system may divide trace objectives across multiple instances of an application, then deploy the objectives to be traced. The results of the various objectives may be aggregated into a detailed tracing representation of the application. The trace objectives may define specific functions, processes, memory objects, events, input parameters, or other subsets of tracing data that may be collected. The objectives may be deployed on separate instances of an application that may be running on different devices. In some cases, the objectives may be deployed at different time intervals. The trace objectives may be lightweight, relatively non-intrusive tracing workloads that, when results are aggregated, may provide a holistic view of an application's performance.

Abstract: A distributed tracing system may use independent trace objectives for which a profile model may be created. The profile model may be deployed as a monitoring agent on non-instrumented devices to evaluate the profile models. As the profile models operate with statistically significant results, the sampling frequencies may be adjusted. The profile models may be deployed as a verification mechanism for testing models created in a more highly instrumented environment, and may gather performance related results that may not have been as accurate using the instrumented environment. In some cases, the profile models may be distributed over large numbers of devices to verify models based on data collected from a single or small number of instrumented devices.