Abstract: The present disclosure relates to systems and methods for providing a network-based service (e.g., network service) for training service operators of the network services. More specifically, the system can generate various training scenarios that the service operators can perform a service functionality computation. The system also modifies one or more attributes of the training scenarios to render an error when the service operator performs the service functionality computation. Upon detecting the error, the service operator can perform one or more corrective actions to resolve the error. The system can analyze the result of corrective actions performed by the service operator and continuously modifies the scenario that the service operator can continuously perform the service functionality computation and determine the corrective actions.

Abstract: This disclosure describes a test prediction service for predicting needed resources and associated costs for testing projects related to testing of new programs, services, etc., within a service provider network. The test prediction service uses a first machine learning model for predicting “hard costs” using data from a first data source that includes end-to-end test details such as associated resource and infrastructure use (costs) of the service provider network during previous end-to-end testing projects. A second machine learning model is used for predicting “soft costs” using data from a second data source that includes data, e.g., engineering headcount/hours spent developing the end-to-end testing and/or during previous end-to-end testing. The data is used to train the machine learning models. The test prediction service uses the trained machine learning models to generate estimated testing costs of new testing projects when a user enters project attributes for a new testing project.

Abstract: Systems and methods for resource management are disclosed. A search request may be received at a resource management service of a provider network. The search request may be received from a client device that does not have permission to access resources in a protected region of a provider network. The search request may specify a query associated with at least one operational health indicator in the protected region. It may be determined, using a secure query service, that the at least one operational health indicator does not exist in the protected region. The secure query service enables the client device to obtain information about the resources in the protected region without gaining access to the resources in the protected region. Sending of a notification indicating that the at least one operational health indicator does not exist in the protected region to the client device may be caused.

Abstract: A domain registry (DR) service executing within a service provider network protects data, such as account data, that is associated with different accounts for testing and/or performing other operations/actions by registering an account with one or more domains. The DR service may register an account in one or more domains based on a request by a user. The operations performed that uses/changes data associated with accounts may be restricted based on the domains for which an account is registered. For example, an account that is registered in a “testing” domain may have different workflows/operations performed using the account data associated with the account registered in the testing domain as compared to an account that is not part of the testing domain.

Abstract: A system can be configured to receive an onboarding request for a service and an API associated with the service. An API configuration defining access permissions, API call structures, and available operations can be determined for the service by an onboarding service. The API configuration can be utilized by the onboarding service to register the API with a proxy server such that incoming API calls are redirected to an access endpoint associated with the service. The proxy server enables the service and the API to be updated via the onboarding requests as service modification and API modification can be submitted, validated, and integrated into a service environment due to the proxy server decoupling the API from the service.

Abstract: Techniques for auto-intrusive data pattern and test case generation for negative service testing are described. A test engine obtains negative test information specifying negative test input examples or schemas associated with tests that are expected to fail. A test generator generates multiple test cases based on the negative test information. A test execution orchestrator splits each test case up into actions that are inserted into queues, where workflow execution agents perform the tests by reading from the queues and interacting with services. The tests may also include adjusting a rate of transactions allowed between top-level services and/or downstream services. Results from the testing are analyzed by a test analysis engine and used to inform the services or the test originator of test cases where the expected failures did not arise.

Abstract: Techniques for testing and validating content generated by applications provided by a provider network are described. A test execution service is disclosed that provides users with a framework for testing the functionality of an application provided by a provider network. A content validation service is disclosed that provides users with a framework for validating content generated by the application by orchestrating the execution of objects to be validated as part of validating content generated by the application. The content validation service generates a validation result for a user by comparing expected data values corresponding to an object to be validated with corresponding actual data values. The disclosed content validation service provides users with the ability to define objects and object definitions for objects to be validated as part of validating content generated by applications provided by the provider network.

Abstract: Devices, systems, and methods are provided for detecting and mitigating application programming interface (API) anomalies. A method may include identifying, based on a first data log, first API data associated with a first API call and a first computer service, and identifying an agreement associated with the first computer service, the agreement including criteria associated with the first API data. The method may include determining that the first API data fails to satisfy the criteria. The method may include determining one or more service dependencies, the one or more service dependencies indicating that operation of the first service depends on operation of a second computer service. The method may include determining, based on the one or more service dependencies, one or more services to which to send notifications indicative of the criteria. The method may include sending the notifications to the one or more services.