Abstract: A set of test methods for performing a live test of service in a production environment, where the service is provided by a set of configured instances in the production environment and each configured instance is composed of a set of components. The methods including a method that includes instantiating components under test from the set of components to setup paths that are to be taken by test traffic, instantiating test configuration components for the service, executing a test case on a created path, generating a snapshot of a first serving component from the set of components, replacing a tested component under test with a second serving component cloned from the snapshot, relocating the service to the second serving component, replacing the serving component of the snapshot with a new component under test to create a new path to be tested, and determining whether all paths have been tested.

Abstract: There is provided a method of test plan generation for live testing and corresponding system and non-transitory computer readable media. The method comprises generating test configurations under which a plurality of test suite items (TSIs) are to be run, merging call paths, in a plurality of groups, according to intersections of call paths on which each of the plurality of TSIs are to be applied and environment coverage associated with the TSIs. The method comprises selecting a test method for each configured instance in each call path associated with one group of call paths. The method comprises creating an initial UML Testing Profile (UTP) model by mapping the TSIs to UTP test cases, thereby generating test runs, and deleting any duplicate test runs. The method comprises ordering test runs and selecting a test runtime framework for each TSI for which the test plan is generated.

Abstract: The disclosure relates to a method, system and computer readable media for validating that a configuration of a cloud system meets requirements, using a reduced test suite selected from a test suite. The method includes classifying the requirements and service instances realizing the requirements composing the cloud system. The method includes applying test suite reduction rules to the test suite, using the classified requirements and service instances, to obtain the reduced test suite. The method includes applying the reduced test suite to the cloud system configured with the configuration, thereby validating that the configuration of the cloud system meets the requirements.

Abstract: A set of test methods for performing a live test of service in a production environment, where the service is provided by a set of configured instances in the production environment and each configured instance is composed of a set of components. The methods including a method that includes instantiating components under test from the set of components to setup paths that are to be taken by test traffic, instantiating test configuration components for the service, executing a test case on a created path, generating a snapshot of a first serving component from the set of components, replacing a tested component under test with a second serving component cloned from the snapshot, relocating the service to the second serving component, replacing the serving component of the snapshot with a new component under test to create a new path to be tested, and determining whether all paths have been tested.

Abstract: There is provided an architecture, methods and a system for live testing in a production environment. The architecture comprises a platform independent Test Planner for generating a test package in response to receiving an event. Generating a test package comprises selecting test goals, generating a test suite and generating a test plan. The architecture comprises a platform dependent Test Execution Framework (TEF) for executing the test package in an environment serving live traffic. Executing the test package comprises initializing the test plan, starting the test plan and reporting the successful completion of the test plan, reporting the suspension of the test plan and waiting for further instructions, or reporting a failure of the test plan and executing a corresponding contingency plan.

Abstract: The disclosure relates to a method, system and computer readable media for validating that a configuration of a cloud system meets requirements, using a reduced test suite selected from a test suite. The method includes classifying the requirements and service instances realizing the requirements composing the cloud system. The method includes applying test suite reduction rules to the test suite, using the classified requirements and service instances, to obtain the reduced test suite. The method includes applying the reduced test suite to the cloud system configured with the configuration, thereby validating that the configuration of the cloud system meets the requirements.

Abstract: A system configuration is validated in a production environment using test cases selected from a test suite, which has validated a first configuration of the system in a development environment. In response to a change in environments, a reduced set of requirements is formed by removing one or more requirements from a set of requirements against which the first configuration is validated. The removed requirements include environment agnostic requirements. The removing is based on at least a classification of configuration parameters, and a first relation between the requirements and the configuration parameters. A reduced test suite is formed by selecting, from the test suite, the test cases that test the system against the reduced set of requirements. The reduced test suite is applied to a second configuration of the system to validate that the system operates in compliance with the set of requirements in the production environment.

Abstract: An upgrade campaign specification upgrades a system from a current configuration to a target configuration. A method is provided for selecting one or more upgrade campaign specifications from a given set of upgrade campaign specifications. For each upgrade campaign specification, the method (1) identifies software and management operations specified in the upgrade campaign specification for upgrading the system; and (2) for each of a set of one or more scenarios defined in terms of operation execution time selection and execution mode, simulates upgrade of the system according to a corresponding arrangement of the software and management operations, thereby obtaining a corresponding execution time and a corresponding outage. A subset of the upgrade campaign specifications is rejected, for which the corresponding execution times do not meet an execution time threshold or for which the corresponding outages do not meet an outage threshold.

Abstract: An upgrade campaign is generated for entities of a system described by a source configuration. According to the upgrade campaign, the system is upgraded to a target configuration using available software. A change model describes changes from the source configuration to the target configuration. According to the change model, an upgrade campaign specification model is generated, which contains at least one upgrade campaign element for each change in the change model. A subset of upgrade campaign elements are matched based on a first set of rules, and the matched upgrade campaign elements are merged into an upgrade procedure in the upgrade campaign specification model. The upgrade procedures are ordered based on a second set of rules and dependencies among the entities of the system, where the dependencies are extracted from the source configuration, the target configuration and the description of available software.

Abstract: An upgrade campaign specification upgrades a system from a current configuration to a target configuration. A method is provided for selecting one or more upgrade campaign specifications from a given set of upgrade campaign specifications. For each upgrade campaign specification, the method (1) identifies software and management operations specified in the upgrade campaign specification for upgrading the system; and (2) for each of a set of one or more scenarios defined in terms of operation execution time selection and execution mode, simulates upgrade of the system according to a corresponding arrangement of the software and management operations, thereby obtaining a corresponding execution time and a corresponding outage. A subset of the upgrade campaign specifications is rejected, for which the corresponding execution times do not meet an execution time threshold or for which the corresponding outages do not meet an outage threshold.

Abstract: An upgrade campaign is generated for entities of a system described by a source configuration. According to the upgrade campaign, the system is upgraded to a target configuration using available software. A change model describes changes from the source configuration to the target configuration. According to the change model, an upgrade campaign specification model is generated, which contains at least one upgrade campaign element for each change in the change model. A subset of upgrade campaign elements are matched based on a first set of rules, and the matched upgrade campaign elements are merged into an upgrade procedure in the upgrade campaign specification model. The upgrade procedures are ordered based on a second set of rules and dependencies among the entities of the system, where the dependencies are extracted from the source configuration, the target configuration and the description of available software.