Abstract: A system provides a framework for testing template code processed by a templating engine. A multitenant system may use template code for implementing Infrastructure as Code (IAC), for example, to generate pipelines for deploying software or provisioning resources for a datacenter configured in a cloud platform. The system sets the search path in a template engine environment object dynamically for each template file. The system allows testing of macros used by the template engine. The system converts the macros to callable entities that can be invoked by test cases. The system allows developers to finding code defects earlier and increase application availability. The system provides flexibility in testing and automation in running the tests. Furthermore, the system allows a separation of production code with unit tests.

Abstract: A method and device for automating analysis of executed performance testing is disclosed. The device includes a processor, and a communications module and memory coupled to the processor. The memory stores computer executable instructions that when executed by the processor cause the processor to receive an input associated with executing a performance test of an application. The processor identifies a test analysis template from a plurality of test analysis templates based on the performance test or the application, each test analysis template defining analysis parameters for interpreting results of executed performance tests. The processor creates a session for analyzing a result of the performance test being executed. Within the analysis session, one or more models are applied to the result, where the one or more models are responsive to the analysis parameters. The processor generates a performance analysis report based on the applied one or more models.

Abstract: Techniques and solutions are provided for database testing. In particular, aspects of the present disclosure facilitate “automatic exploration” of testing of database features, such as database commands. In particular, test definitions can be created for particular database features, where a test definition can have variants. Different test definitions can be combined into a composite test, which can include different variants for different tests. An executable test can be created that allows a database to execute a composite test. The present disclose also provides techniques and solutions for analyzing test results, particularly when voluminous test results are returned. In one example, test results for multiple iterations of a test can be compared. A change in test results, for example, can cause a test result to be flagged for further review. The composite tests, or result analysis techniques, can help provide better testing of database software to help identify software errors/bugs.

Abstract: A system for reestablishing network connectivity for particular network equipment in a failed network zone comprises a processor associated with a server. The processor receives a flag that identifies the failed network zone including particular network equipment that have lost network connectivity in a network. The processor generates a first non-fungible token using the configuration information associated with each of the particular network equipment. The processor determines routing information based at least in part upon the configuration information and a network topology. The processor generates a second non-fungible token based at least in part upon the routing information. The processor aggregates the first non-fungible token and the second non-fungible token to generate a primary non-fungible token. The processor reestablishes network connectivity for at least a portion of the particular network equipment based at least in part upon information associated with the primary non-fungible token.

Abstract: A system and method for determining a relative importance of a selected test in a plurality of tests, comprising a computational device for receiving one or more characteristics relating to an importance of the code, an importance of each of the plurality of tests, or both; and for determining the relative importance of the selected test according to said characteristics.

Abstract: A test orchestration engine can create a unitary test configured to perform multiple tests of an application program with a single journey. The engine can identify actions performable by the application program and determine, based on a machine learning (ML) model, those actions that are performable sequentially to validate a set of features of the application program. The system creates the unitary test protocol including a directed sequence of actions to validate the set of features for the test protocols. The system then causes execution of the unitary test protocol to generate a result indicative of performance of the application program.

Abstract: Aspects of the disclosure relate to conducting automated web service testing in a continuous integration and delivery test deployment environment using artificial intelligence (AI) generated test data. In some embodiments, a computing platform may receive, from a developer computing platform, a test code request, receive, from a web service computing platform, a training data set, configure a test data set based on the training data set and the test code request, use AI engine to apply one or more corrections to the test data set based on the test code request and to produce a corrected test data set, execute the test code using the corrected test data set to produce test code output results, and send, to the developer computing platform, the test code output results.

Abstract: This information processing device includes: an analysis unit that parses a program assigned to a task which is periodically executed; and an output unit. The analysis unit classifies each of at least one variable used to execute the task into a first group in which a process of reading a value from a memory is performed first and a second group in which a process of writing a value in the memory is performed first. The analysis unit determines that the necessity for a soft error measure for a variable belonging to the first group is greater than the necessity for a soft error measure for a variable belonging to the second group. The output unit outputs a determination result. Thus, a variable that requires a soft error measure can be identified.

Abstract: A computer implemented method, a computer program product and a data processing system for executing an application written in a dynamic language are provided. An execution point of the application is loaded. A list of classes associated with the execution point is generated. The loading of each class in the list of classes is simulated. New execution points and new classes accessible from each execution point within each class in the list of classes are identified by recursively parsing instructions associated with each execution point. The list is modified to include the identified new execution points and new classes. Responsive to a determination that new execution points and new classes have been identified, the steps of identifying new execution points and new classes and modifying the list is repeated. The list is saved.

Abstract: A method and an apparatus for generating software test reports are provided. The method includes following steps: providing a testing platform that supports retrieving one or more documents such as screenshots or DOM-like documents related to screen content of an application under test (AuT) and analyzing the documents to obtain description data of the screen content; selecting to execute a test report generator, and querying the description data, multiple test scripts ever executed on a system under test (SuT) and multiple test actionables from the testing platform by the test report generator, so as to evaluate a test trace of the AuT, calculate at least one test actionable and test data adapted for the AuT, and return the calculated test actionable to the testing platform; and executing the test actionable on the AuT in the SuT by the testing platform, so as to generate test reports of the AuT.

Abstract: A method and computer program for generating code coverage information during testing of a code sequence are described, in which the code sequence comprises decisions, each having one or more conditions as inputs. The method includes executing the code sequence on target processing circuitry under the control of test stimuli and maintaining, in memory, a code coverage table for at least one decision. When a decision is evaluated, a bitstring is created within a storage element, each position in the bitstring being associated with one of the conditions and the value in that position representing the value of that condition used in evaluating the decision. The bitstring is used to identify the entry, in the code coverage table associated with the evaluated decision, for that combination of values of the conditions, and a confirmation value is recorded in that entry, indicating that the decision has been evaluated for that entry.

Abstract: Methods, systems, and computer-readable media are disclosed herein for a system and framework that tests end-user applications for failures, data validation, and performance indicators. In aspects, multiple use-modeling programs that mimic user interactions are used to concurrently run unique instances of the application in real-time to simulate a real-world scenario, at scale and with a full load. Whether data, operations, and/or functions of the end-user application fail or are successful is automatically documented in real-time, while performance is concurrently measured.

Abstract: The system determines data which indicate memory usage by a process and information related to a number of instances of the process. The system determines for the process: a first average amount of memory; a first standard deviation; a first value indicating a difference between a current amount of memory used by the process and the first average amount; a second average number of process instances; a second standard deviation; and a second value indicating a difference between a current number of process instances and the second average number. The system performs a corrective action to address a memory spike associated with the process responsive to comparing at least one of: the current amount of memory used by the process with a watermark; the first value with the first standard deviation scaled; the current number of instances with a watermark; and the second value with the second standard deviation scaled.

Abstract: A system and method for automated chatbot testing to provide training and quality assurance of conversational artificial intelligence systems, comprising a chatbot testing administrator interface which allows chatbot makers to define what a chatbot is supposed to do, create test scripts to test the performance of the chatbot, and review the results of the chatbot tests, a chatbot testing server which provides and interface between chatbot testing agents and the administrator interface, instantiates chatbot agents and distributes them across available hardware and runs testing programs which activate, configure, and deactivate chatbot testing agents as needed. A plurality of chatbot agents may be running in parallel to provide automated testing based upon test script configuration.

Abstract: Systems and methods for automated test data microservices are provided. Test versions of software (such as an Application Programming Interface (API)) may be configured to automatically generate test data and to call a microservice to manage the test data. The microservice may automatically add and remove the test data from an operational data store to facilitate the testing process and to automatically perform setup and teardown stages of the testing process.

Abstract: A system including a data repository storing metrics describing operational behavior of software programs executing in an enterprise system. The system also includes an application programming interface (API) gateway configured to receive the metrics. The system also includes an ingestion layer configured to ingest the metrics to form the ingested metrics. The system also includes a tumbling window processor configured to process the ingested metrics and the events into heat maps, sort the heat maps into the time slices, and populate the time slices with the ingested metrics.

Abstract: In some implementations, a device may receive action information, for an action from multiple actions, that indicates at least one of code associated with the action or one or more inputs required to complete the action. The device may receive workflow information, for a workflow included in a set of workflows, that indicates one or more actions from the multiple actions and an order in which the one or more actions are to be completed for the workflow. The device may receive, via an application programming interface (API) call, a request to complete the workflow The device may identify the workflow information associated with the workflow in a data structure. The device may perform the workflow based on the workflow information and the one or more inputs by executing code for the one or more actions indicated by the workflow information and in the order indicated by the workflow information.

Abstract: In implementations of the present disclosure, a solution for optimization of a learning network in an equivalent class space is provided. In this solution, base paths running through layers of a learning network are determined. Each node utilizes an activation function with a scaling invariant property to process an input from a node of a previous layer, each base path comprises a single node in each layer, and processing in the base paths is linearly independent from each other. A combined value of parameters associated with nodes in each base path is updated. A parameter associated with a node is used to adjust an input obtained from a node of a previous layer. Values of parameters associated with nodes in the base paths are updated based on updated combined values of parameters. Through this solution, optimization efficiency can be improved and more accurate optimized values of parameters are achieved.

Abstract: A distributed system for software verification includes a plurality of processors, each of which actively executes a verified software to perform a verifying operation of a corresponding target object under no external instruction, wherein the plurality of processors independently perform verifying operations of different target objects; a switch including a plurality of first connection ports electrically coupled to the plurality of processors, respectively, and a second connection port electrically coupled to each of the plurality of first connection ports; and a data collector electrically coupled to the second connection port. An information report is generated and sent to the data collector via the corresponding first connection port and the second connection port in response to the verifying operation.

Abstract: Techniques for testing may include: receiving a first command at a framework, wherein the first command includes first information identifying a first test to be executed and a first collector that collects data during execution of the first test; and responsive to receiving the first command, perform processing to execute the first command, wherein the processing is coordinated by the framework and wherein the processing includes: executing the first test; collecting test data, wherein the test data is collected by the first collector during execution of the first test; and generating a report regarding the test data collected while executing the first test.

Abstract: A computer-implemented method of mapping a process model of a software application executed by a hosting platform. A series of actions taken by a user with respect to user interfaces generated by the software application are simulated through one or more application programming interface (API) calls. The user interfaces include a form and the series of actions includes opening the form. A plurality of user interface fields of the form are then identified through one or more other API calls executed while impersonating a session of a user under test. At least some of the user interface fields are set to known values and the form is submitted to the software application after the fields have been set. Changes to field-related and record-related information resulting from the submission are then gathered and a process model of the software application is determined based upon the gathered information.

Abstract: Features of each ordered step of a recorded test script for an application under test (AUT) are generated. The ordered steps are segmentally organized over actions based on feature relatedness of the ordered steps. Each action includes a contiguous subset of the ordered steps. The actions are hierarchically organized within a tree based on an ordered feature relatedness of the actions. The tree has nodes respectively corresponding to the actions. The actions are abstractedly related to one another based on an unordered feature relatedness of the actions to generate abstracted actions that each generalize a subset of the actions. Abstracted action groups are generated based on the abstracted actions and the tree. Each abstracted action group includes abstracted actions mapping to corresponding nodes of the tree within a same branch. The ordered steps of the recorded test script are collapsed into a series of instantiations of the abstracted action groups.

Abstract: The present invention is a process by which an engineer can provide as input the design, functional verification goals, and other abstract design details, and receive as output an agent which can be integrated into traditional test benches and will generate stimuli to automatically hit the functional coverage goals for the design. The present invention may employ a system which includes a learning configurator, a pre-trained learning test generator, and a test bench. The pre-trained learning test generator is communicatively coupled to the generator and notably comprises a learning algorithm.

Abstract: A GUI testing device may be configured to execute a testing state machine for interacting with a software application to generate an initial screen of a GUI. The GUI testing device may be configured to determine a current state in the testing state machine based upon a matching trigger target in the initial screen to a given state. The current state may include an operation, and the operation may associate with a trigger target to operate on. The trigger may include a source state, a destination state, and a trigger target. The operation may include a user input operation, and an operation trigger target. The GUI testing device may be configured to perform the operation on the matching trigger target in the initial screen to generate a next screen of the GUI, and advance from the current state to a next state based upon the trigger.

Abstract: Systems, computer program products, and methods are described herein for testing an application with dynamically linked security tests. The present invention may be configured to perform, using a request engine, based on the first data, and based on test protocols stored in a first data structure, a first security test on an application. The present invention may be further configured to determine, based on determining that the application failed the first security test and based on a second data structure, whether the first security test is linked to one or more other security tests, where the second data structure includes security test sequences linking security tests and/or data to transmit from the first security test to the one or more other security tests, and provide, to the first queue, one or more other security tests to which the first security test is linked.

Abstract: This disclosure relates generally to a human machine interface (HMI) application, and more particularly to modular testing kits and testing framework for testing the HMI application. In one embodiment, a method for developing a modular testing kit for testing a modular component of the HMI application is disclosed. The method may include generating a plurality of test cases for testing the modular component based on a definition of each of a set of interfaces configured to implement the modular component on a target platform, developing the modular testing kit to execute the plurality of test cases by simulating each of the set of interfaces, and creating a test script for each of the plurality of test cases using the modular testing kit.

Abstract: A method includes generating a behavioral state for an endpoint device based on actor identities and corresponding subject identities for a plurality of operations wherein for each operation, a respective actor represented by a respective actor identity performs the operation upon a respective subject represented by a respective subject identity. Performance of a later operation by an actor with an actor identity upon a subject with a subject identity is recorded and the actor identity and the subject identity are used to determine that the performance of the later operation does not match the behavioral state and indicates a security risk.

Abstract: A method of performing auto-naming of content includes: receiving an auto-naming command for the content; performing auto-naming of the content by using different parameters according to different content types to obtain at least one auto-naming result for the content; and displaying the auto-naming result.

Abstract: A script directing functional testing of an application specifies values for properties of a specific graphical user interface (GUI) and specifies an image of the specific GUI element. A computing device detects whether the application has displayed the specific GUI element by evaluating the values for the properties of the specific GUI element against GUI elements that the application has displayed. If unsuccessful, the computing device detects whether the application has displayed the specific GUI element by evaluating the image of the specific GUI element against a screen image including the GUI elements that the application has displayed.

Abstract: Systems, methods, and computer-readable media are described for selecting, at build time, a respective compiler and/or a respective set of compiler options for each section of code to be compiled such that the compiler/compiler options selected for each code section are optimized for that code section with respect to one or more metrics. Abstract syntax tree (AST) analysis and semantic analysis may be performed at build time for each section of code to identify the compiler/compiler options that produce compiled object code for that code section that maximizes or minimizes a desired metric. The metric according to which compiler/compiler option optimization is performed may be any suitable metric including, without limitation, performance, binary size, security, reliability, scalability, and so forth.

Abstract: Aspects of the present disclosure relate to systems and methods that help automate the generation of a test vector. Every functional product, be it a service, device, or combination thereof, has one or more documents associated with it. These documents may include such documentation as: (1) Release Notes; (2) Configuration Guides; (3) command line interfaces (CLIs)/application program interfaces (APIs); (4) Data Sheets; (5) Installation Guides; (6) User Manuals; (7) Errata notices; and (8) other documentation. In embodiments, command data-models and ranges and sequence of valid inputs to the data-model are extracted from natural language expressions in documentation related to a specific product. In embodiments, this extracted information is used to generate one or more table of attributes and properties or constraints about which a test vector may be generated for testing a command.

Abstract: In one example, a method for testing a multi-threaded application includes running an initial test of the multi-threaded application and collecting thread generation data and determining the thread hierarchy. The thread execution is then modified to produce a modified configuration and a second test is run with the modified configuration. A device for testing of multi-threaded applications is also provided.

Abstract: The present invention is a process by which an engineer can provide as input the design, functional verification goals, and other abstract design details, and receive as output an agent which can be integrated into traditional test benches and will generate stimuli to automatically hit the functional coverage goals for the design. The present invention may employ a system which includes a learning configurator, a learning-based test generator, and a test bench. The learning test generator is communicatively coupled to the generator and notably comprises a learning algorithm.

Abstract: A rollback refinement process for software testing using software models in a model-based testing environment. The method includes: identifying a model having a set of model elements assigned relevance scores based on correspondence between each model element in the set of model elements and a pre-defined suspect element, the model producing an expected behavior when running a test case; generating a reduced model on condition that the reduced model include only model elements assigned a threshold reduction score corresponding to the relevance scores; determining that the reduced model does not reproduce the expected behavior when running the test case; rolling back a model element into the reduced model to produce a revised model; and responsive to the revised model not reproducing the expected behavior, performing a rollback refinement of the revised model based on a number of model elements rolled back into the reduced model.

Abstract: Certain aspects of the present disclosure provide techniques for managing the deployment of software applications based on application performance data. Embodiments include transmitting, to a performance testing system, information identifying a test build of a software application component in a first deployment environment in a software development pipeline to test. Performance information for the test build of the software application component is received from the performance testing system. The received performance information for the test build of the software application component is compared to one or more of predetermined performance parameters or historical performance data for the software application component. Based on the comparing, the system takes one or more actions to manage deployment of the test build of the software application component to a second deployment environment in the software development pipeline.

Abstract: A context-sensitive process guidance system provides process guidance based on context data from a system used in performing the process. The system may be a work item tracking system or similar project management tool. Process guidance may provide information to a user performing actions related to a process. Context data may be derived from various sources, including the state of an application at the time of the request for process guidance, the user making the request and the role assigned to that user in the process, or the data displayed for the user who made the request. The context-sensitive process guidance system may display context-sensitive process guidance content and provide links to related process guidance content based on the context data.

Abstract: Example implementations relate to separating verifications from test executions. Some implementations may include a data capture engine that captures data points during test executions of the application under test. The data points may include, for example, application data, test data, and environment data. Additionally, some implementations may include a data correlation engine that correlates each of the data points with a particular test execution state of the application under test based on a sequence of events that occurred during the particular test execution state. Furthermore, some implementations may also include a test verification engine that, based on the correlation of the data points, verifies an actual behavior of the application under test separately from the particular test execution state.

Abstract: A system and method for remediating and redeploying out of compliance applications and cloud services uses integrated vulnerability information. The system and method may use Conda packages and compliance reports to perform the monitoring, remediation and redeployment.

Abstract: This invention generally relates to a process, system and computer code for enabling users to create adapters that enable application automation by collecting automation information; locate application controls and tracking changes between an older and a newer version of the application, such changes to include addition of one or more new data fields, removal of one or more data fields, change in data field type (i.e. type of data held in the field); change field layout; and change the underlying technology framework of the application; to present the changes using an exception management model to the user, so user can by way of example provide feedback in a visual instead of programmatic manner; store the changes, so as to make the adapters resilient to application changes and upgrades; and incorporating the changes to upgrade the application.

Abstract: A system and an operating method thereof include a system under test (SUT) having a hardware array of flash storages, partitions including logical volumes, a kernel subsystem including an operating system, and an application layer including services, applications, and/or systems. The application layer receives test drivers and corresponding test fixtures from an external source. The SUT is segmented into subsystems of interest, and a different test using a different test fixture is applied to each subsystem. The tests are monitored and the results of the tests are collected and verified against suitable benchmarks. Results and associated data are archived.

Abstract: In an embodiment, a modeling component generated in a first modeling environment can be implemented in a second modeling environment. The modeling component is executed in the first modeling environment. A first behavior of the modeling component in the first modeling environment is obtained based on the executing. The modeling component is then executed in the second modeling environment. A second behavior of the modeling component in the second modeling environment is obtained based on the executing the modeling component in the second modeling environment. The first behavior is compared to the second behavior. Based on the comparing, it is verified that the second behavior of the modeling component complies with one or more modeling requirements of the second modeling environment. The second behavior of the modeling component may be implemented in the second modeling environment.

Abstract: The computing device receives information representing a design of an experiment. The design of the experiment comprises a plurality of test cases. Each element of a test case of the design is a test condition for testing one of factors for the experiment. The computing device generates a graphical representation based on the information. The graphical representation comprises a plurality of axes for respective ones of factors used in a fractal sequence. Each of the plurality of axes has two endpoints comprising a first endpoint that corresponds to a first level of a respective factor, and a second endpoint that corresponds to a second level of the respective factor. The computing device plots, on the graphical representation, data corresponding to one or more test cases of the plurality of test cases. The computing device displays an interactive graphical user interface comprising the graphical representation with the plotted data.

Abstract: Systems, methods, and apparatus for automatically parallelizing code segments are provided. For example, an environment includes a profiling agent, a parallelization agent, and a verification agent. The profiling agent executes a code segment and generates a profile of the executed code segment. The parallelization agent analyzes the code segment to determine whether a parallelizable portion is present in the code segment. When a parallelizable portion is present, the parallelization agent determines, based on the profile of the executed code segment, whether to parallelize the parallelizable portion of the code segment. If it is determined to parallelize the parallelizable portion of the code segment, the parallelization agent automatically parallelizes the parallelizable portion of the code segment. The verification agent verifies the functionality and/or correctness of the parallelized code segment.

Abstract: A method and device for design driven development based automation testing is disclosed. The method includes receiving a design document comprising an overview of a proposed product. The method further includes extracting functionality of the proposed product based on an analysis of at least one design object in the design document. The method includes converting the functionality of the proposed product into at least one test script based on the at least one design object. The method includes replacing one or more of the at least one design object with a corresponding real object in a developed product associated with the proposed product. The method further includes executing the at least one test script to test the developed product, in response to the replacing.

Abstract: A user interface (“UI”) code coverage system and process determines the coverage by test code of UI Components in a computer program. In other words, the UI code coverage system and process determine whether the test code tests a UI Component. In at least one embodiment, the UI code coverage system and process identify the particular UI Components tested by the test code, provide coverage (i.e. tested) statistics, provide a visualization of the UI Component tested, provide a visualization of tested and not-tested (also referred to respectively as “covered” and “not covered”) UI Components of a UI Screen.

Abstract: A method includes receiving, by a computing system that includes an application repository, an updated version of an application. The method also includes determining whether the particular computing device contributes additional test scope for the updated version of the application beyond existing test scope for the updated version of the application that is contributed by the group of computing devices. The method further includes, sending the updated version of the application to the particular computing device for installation in response to determining that the particular computing device contributes additional test scope.

Abstract: A device management apparatus includes circuitry configured to: acquire an updating program for updating a device; transmit the updating program to the device; receive introduction success/failure information indicating whether the updating program is successfully introduced, from the device; acquire, if the introduction success/failure information indicates success in introduction, a verification program for verifying an operation of the device updated with the updating program; and verify whether the operation of the updated device can be guaranteed based on the verification program.

Abstract: A system, method and computer-readable medium for testing a service-based application identifies a set of use cases for the service-based application. Each use case in the set is mapped to a test case to generate a first set of test cases. Each of the test cases is refined to generate an updated set of test cases and the updated set of test cases is executed.

Abstract: A method includes receiving, via a processor, a model, a test case, and one or more suspect elements. The method includes calculating, via the processor, a relevance score for each model element of the model based on the one or more suspect elements. The method includes setting, via the processor, a threshold reduction score. The method includes generating, via the processor, a reduced model by reducing the model based on the relevance scores and the threshold reduction score. The method includes evaluating, via the processor, the reduced model based on the test case. The method includes detecting, via the processor, the reduced model reproduces an expected behavior. The method includes outputting, via the processor, a result model in response to detecting the reduced model reproduces the expected behavior. The method includes modifying, via the processor, an application associated with the model based on the reduced model.

Abstract: Techniques for configuring a test system that enable simple specification of a degree of concurrency in testing separate functional portions of a semiconductor device. For a test flow with multiple sub-flows, the pins accessed in connection with each sub-flow may define a flow domain. Site regions, each associated with a flow domain, may be defined. Tester sites may be associated with each of these flow domain specific site regions and independently operating resources may be assigned to these tester sites. A second portion of the defined site regions may be associated with tester sites, but resources assigned to these site regions may be accessed from multiple flow domains. Test blocks, even if not developed for concurrent execution, may be executed concurrently using resources in the flow domain specific site regions. Flexibility is provided to share resources through the use of the second portion of the site regions.