Abstract: Testing computer software applications by performing a first black-box test on a computer software application, identifying any instructions of the computer software application that were reached by a payload of the first black-box test, determining a degree of success of the first black-box test in accordance with predefined success criteria, determining whether any of the instructions that were reached by the payload changed after performing the first black-box test, deciding whether to perform a second black-box test on the computer software application, where the deciding whether to perform the second black-box test is based on whether any of the instructions that were reached by the payload of the first black-box test changed after performing the first black-box test, and the degree of success of the first black-box test.

Abstract: A test using a virtual date and time is to be performed without restriction imposed, depending on the implementation status of another test using a virtual date and time. A control section 220 outputs a specified layer identifier and a date and time conversion instruction. Then, when the date and time conversion instruction has been outputted from the control section 220, a date and time conversion section 230 acquires the present date and time from a clock 210, then reads, from a layer storage section 100, conversion definition information corresponding to the layer identifier outputted from the control section 220, then by using the read-out conversion definition information, converts the present date and time acquired from the clock 210, and then outputs, to the control section 220, virtual date and time information indicating the date and time obtained by conversion.

Abstract: Testing computer software applications by performing a first black-box test on a computer software application, identifying any instructions of the computer software application that were reached by a payload of the first black-box test, determining a degree of success of the first black-box test in accordance with predefined success criteria, determining whether any of the instructions that were reached by the payload changed after performing the first black-box test, deciding whether to perform a second black-box test on the computer software application, where the deciding whether to perform the second black-box test is based on whether any of the instructions that were reached by the payload of the first black-box test changed after performing the first black-box test, and the degree of success of the first black-box test.

Abstract: An exemplary system for electronically managing requirements for software development includes a projects module, a requirements module, a mapping module and a verification module. The projects module is configured to establish a software development project. The requirements module is configured to define requirements for the project based on requirements information captured from a requirements source. For each requirement, the projects module is configured to associate source code developed for the requirement with the project or assign the requirement for development of source code. The mapping module is configured to map procedures identified in the source code to the defined requirements. The verification module is configured to verify the defined requirements based on results of one or more of analyses, code coverage measurements and unit testing performed on the mapped procedures.

Abstract: This disclosure relates generally to application development, and more particularly to systems and methods for fixing software defects in a binary or executable file. In one embodiment, a software defect management system is disclosed, comprising: a processor; and a memory disposed in communication with the processor and storing processor-executable instructions comprising instructions for: obtaining an application programming interface call for a black-box software application; determining whether the black-box software application is configured in a defective manner to process the application programming interface call; identifying a call processing application to process the application programming interface call, based on determining whether the black-box software application is configured in a defective manner to process the application programming interface call; and providing the application programming interface call for the identified call processing application.

Abstract: Provided are techniques for receiving an error inject script that describes one or more error inject scenarios that define under which conditions at least one error inject is to be executed and compiling the error inject script to output an error inject data structure. While executing code that includes the error inject, an indication that an event has been triggered is received, conditions defined in the one or more error inject scenarios are evaluated using the error inject data structure, and, for each of the conditions that evaluates to true, one or more actions defined in the error inject script for the condition are performed.

Abstract: A program stored on a computer-readable medium for self-decommissioning at least a part of a program product is disclosed. The program may include a program code that defines a plurality of functions, a program code that includes a metric for each of the plurality of functions, wherein the metric includes at least a usage pattern and transactions relating to the metric, a program code configured to measure at least one metric, a program code defining a logic relating to the self-decommissioning of a component and/or program, a program code configured to determine decommissioning of the at least a part of the program product based on the logic, and a program code configured to self-decommission the at least a part of the program.

Abstract: Embodiments provided a formalized set of intermediate analysis contexts that are relevant for analysis checks of target code. Such intermediate analysis context may include, but are not limited to, the development phase of targeted code, the type or state of the targeted code, a source manipulating the targeted code, a purpose for the targeted code, or other development or runtime requirements. Accordingly, embodiments dynamically identify a current analysis context under which targeted code is being developed and can then execute rules based on knowledge of what contexts the rule(s) may apply. More specifically, analysis rules can describe (e.g., via metadata) those context conditions under which a rule can run. Based on such description and the current context, those rules that have been configured to apply to such context conditions can be executed.

Abstract: A device creates a graph based on source code, and analyzes the source code to identify private variables and functions of the source code and public variables and functions of the source code. The device determines, based on the graph, a size threshold and semantics-related characteristics of functions and variables for each module, of multiple modules, and assigns, based on the graph, the private variables and functions to a corresponding module of the multiple modules. The device reduces, based on the graph, a number of the public variables and functions assigned to each module, and generates the multiple modules based on one or more of the graph, the size threshold, the assigned private variables and functions, and the number of the public variables and functions assigned to each module.

Abstract: In one embodiment, the invention comprises partial fault tolerant stream processing applications. One embodiment of a method for implementing partial fault tolerance in a stream processing application comprising a plurality of stream operators includes: defining a quality score function that expresses how well the application is performing quantitatively, injecting a fault into at least one of the plurality of operators, assessing an impact of the fault on the quality score function, and selecting at least one partial fault-tolerant technique for implementation in the application based on the quantitative metric-driven assessment.

Abstract: Systems and methods are described that schedule tests based on historical test information, such as tests that validate developed or changed code. For example, the system may review the results of previously used tests, order the tests based on the results, such as based on the performance of the code during the test, and apply the tests during a testing cycle based on the order.

Abstract: In one embodiment, a method determines a plurality of test cases to test an application and a set of attributes assigned to each test case in the plurality of test cases. The method then calculates a test case risk score for each test case in the plurality of test cases based on the set of attributes associated with each respective test case. The test case risk score quantifies a risk in not executing each respective test case. A subset of the plurality of test cases is selected based on at least a portion of the calculated risk scores. The subset of plurality of test cases is output along with a test plan risk score that quantifies the risk in not executing test cases not included in the plurality of test cases.

Abstract: One embodiment of the present invention provides a system for profiling a virtual machine on an embedded device. During operation, the system receives an operation-request at a system-level profiler from a virtual machine on an embedded device. The system then records the operation-request in a system-level log. Next, the system sends the operation-request to a native layer on the embedded device. Then, the system receives an operation-response associated with the operation-request from the native layer. The system then records the operation-response in the system-level log. Next, the system sends the operation-response to the virtual machine, thereby facilitating logging of interactions between the virtual machine and the native layer. Note that logging interactions between the virtual machine and the native layer facilitates emulation of the platform independent virtual machine on a second device.

Abstract: Methods for identifying and analyzing performance traces are provided. Temporal logic formulas are patterns that can be compared with traces and individual events to identify the existence of certain behavior. Traces, sequences of time-stamped events in time order, are compared with one or more temporal logic formulas to identify the event sequences that match the formulas. The temporal logic formulas can be written in the simple temporal logic language that is presented. When a formula matches an event sequence, attributes from the event sequence are extracted and metric expressions are evaluated based on these attributes. The extracted attributes and the results of the metric expression are returned. This temporal logic pattern matching process can efficiently identify and analyze performance traces and significantly reduce manual effort for identifying performance problems.

Abstract: A system, method, and computer program product is disclosed that for debugging errors in software code. According to some approaches, techniques are provided for performing on-the-fly switching from compiled to interpretive debugging for a software program. The test starts with compiled code, and when it needs to stop for debugging, the debugging occurs in interpretive mode. Once debugging has concluded, the execution can switch back to compiled mode. In this way, the debugging activities can achieve the speed and efficiency of using compiled optimized executables, while still being able to allow debugging without performing any recompilations.

Abstract: TSQL debug breakpoint interoperability across heterogeneous project systems within an IDE can be provided. A user can set a breakpoint in the user's SQL source code and the breakpoint can be triggered by the code within another project. Breakpoints can be set and triggered within SQL source code tiles that reside in one or more SQL Database projects. Binding of the application project being debugged with one or more SQL Database projects that reside within the IDE session can be automated, SQL artifacts not found in any of the SQL Database projects can be debugged.

Abstract: A device for generating a performance evaluation program includes: a memory; and a processor coupled to the memory. The processor is configured to: analyze a source code of a target program that is subject to performance evaluation, translate the source code into a binary code based on an analysis result of the source code while generating execution binary that has an evaluation area to be used in the performance evaluation at a target location corresponding to a candidate location of the target program, and write an evaluation code in the evaluation area of the execution binary to evaluate performance of the target program based on an evaluation item and the target location of the target program.

Abstract: The present invention provides a method and system for quantifying a bug preventability measure of a bug prediction technique. In accordance with a disclosed embodiment, the method may include applying a weighted recursive function, on an actual bug count of each version of an application, for computing a golden bug count of the each version. Further, the method shall include deriving a set of source code parameters of the application and applying a linear regression model, on the set of source code parameters of the each version of the application in order to calculate a predicted bug count for the each version. A bug deviation ratio, which shall be indicative the bug preventability measure, can be defined as a ratio of the weighted aggregated deviation and the weighted quadratic aggregation.

Abstract: A method for pre-testing software compatibility is illustrated. A system and software information of an operating system is extracted. While that a patch or new software will be installed in the operating system is detected, whether a compatibility test should be executed is determined. While that the compatibility test should be executed is detected, a virtual machine with a first clone system is generated, wherein the first clone system is generated according to the system and software information. The patch or the new software is installed in the first clone system to generate a second clone system. The compatibility test for the second clone system is executed. According to a result of the compatibility test, whether the patch or new software can be installed in the operating system or the second clone system can be switched to be used is determined.

Abstract: A mechanism for utilizing a virtual machine cloud for automated test system deployment is disclosed. A method of embodiments of the invention includes selecting a master image used to initialize one or more virtual machines (VMs), providing a list of repository definitions and test packages to the one or more VMs, and receiving test results from executing the test packages on a computer system of the VM defined by the master image, wherein the computer system includes an operating system and one or more software applications.

Abstract: In a method for debugging device drivers of a computing device, a command line interface (CLI) is created in a user space of an operating system of the computing device. A command input by a user is received through the CLI, and transmitted to a kernel space of the operating system. The command is parsed to determine a function of a device driver and parameters of the function. An address of the function is inquired in the kernel space. The function is debugged according to the address of the function in the kernel space.

Abstract: A system and method for producing a massive number of diverse program instances so as to deter differential attacks, collusion, and similar hostile actions. Code portions are shown to be defined in various manners, instantiated, and aggregated. The system and method establishes a very large number of program instances that may be deployed. Furthermore, testing is accomplished over a minimal set of instances to provide for high test coverage and high confidence over the fully deployed in stance set without incurring a high penalty.

Abstract: Methods and apparatus in a computer for automatically testing computer programs involve opening a predefined graphical user interface (GUI) on a screen of the computer; loading a set of program script instructions from a script database in communication with the computer that is associated with the predefined GUI; reading a loaded set of program script instructions; retrieving, based on the loaded set, data and at least one image object corresponding to the predefined GUI from a data and image object database in communication with the computer; taking a screenshot of the predefined GUI that includes at least one image object of the predefined GUI; determining whether an image object in the screen shot matches an image object retrieved from the data and object image database; and if a target position on the screen of the matching image object based on data retrieved from the data and image object database, and activating a control function adapted to control the predefined GUI based on the loaded set of progra

Abstract: A computer implemented system and method for testing code for implementation in web browsers, implements test class code defining test cases for testing operations on web applications implementable by different web browser types, and implements handler class code comprising code specific to each web browser and defining strategies to be used by test cases. The handler class code implements a handler interface and the test class code uses the handler interface to identify the methods to be used for test cases. A non-transient storage medium stores code for a handler template supporting different web browser types for use in a test environment, the code comprising code to identify browser specific strategies for implementation in the test environment; code to implement a strategy support interface for use by a handler factory to select a handler instance; and code to implement a handler interface for use by a test case to identify methods for testing web browser functions.

Abstract: Systems and method are provided for determining risk of product release failure in an environment where multiple independent releases come together into a shared environment. A risk factor for success or failure of a product release is based on release information input by a user and information input by other users regarding releases, the users typically on distinct development systems. The information from each distinct system is mapped (e.g., normalized and weighted) into a common schema. A risk index for the product release is determined. The risk index, risk factors, and/or associated information therewith, is presented to a user.

Abstract: A method, apparatus and software related product (e.g., a computer readable memory) are presented for automatic testing of software products such as front-ends of computing devices. Various embodiments described herein can simplify the delivery and verification of software products and services across computing devices and screens by comparing user interface images using image analysis. For example, a screen buffer or visual representation of a new version of a software application can be compared with those representations generated using one or more previously released (positively tested) software versions of that application. This can broaden the reach of the solution and facilitate automation of workflows, which previously required human interaction.

Abstract: A method, system, and computer-program product are provided for automatically performing stability testing on device firmware. The method includes the steps of copying a binary file corresponding to a version of a firmware to one or more nodes that each include a testbench, causing the one or more nodes to perform tests utilizing the version of the firmware, and determining whether a new build of the firmware is available. If the new build is available, then the steps include copying a second binary file corresponding to the new build to the one or more nodes and causing the one or more nodes to perform the tests utilizing the new build. However, if the new build is not available, then the steps include then causing the one or more nodes to perform one or more further iterations of the tests utilizing the version of the firmware.

Abstract: A method for selecting a command file is described. One or more command files coupled with a first device are accessed. Each command file of the one or more command files comprises one or more command file attributes. One or more device attributes associated with a second device are accessed. The one or more command file attributes and the one or more device attributes are compared to determine a correlation between the one or more command files and the second device. Based on the comparing, generating a ranked portion of the one or more command files according to the correlation.

Abstract: Use of an Abstract Syntax Tree (AST) to select portions of source code when analyzing the affect of changes in that source code from one version to another. In this way, it is possible to better focus on how changes in the source code, especially changes which only impact limited portion(s) of the source code, affect code quality with respect to measures like code complexity, performance and so on.

Abstract: A method includes continuously capturing, through an application executing on a data processing device, images of a desktop of the data processing device as a background process as part of a testing session on the data processing device in an active mode thereof. The method also includes encoding, through a processor of the data processing device, the captured images of the desktop as a video sequence, and providing a capability to a user of the data processing device and/or another data processing device to detect a fault event related to the testing session based on access to the encoded video sequence.

Abstract: Arrangements described herein relate to adopting an existing automation script to a new framework. A first version of an automation script configured to execute in a first framework, and expected unit test results for the automation script, are received. A second version of the automation script is created from the first version of the automation script. The second version of the automation script is configured to execute in at least a second framework. The first version of the automation script is executed in the first framework to run a first version of a unit test and the second version of the automation script is executed in the second framework to run a second version of the unit test. Results generated by execution of the first and second versions of the automation script are analyzed to validate the second version of the automation script.

Abstract: There is provided a method of testing an application. The method is performed by a client terminal device and includes: receiving, from a master terminal device, a test case including a code for testing the application, the code being written in a script including Javascript; in response to receiving, from the master terminal device, a request for executing the code in a unit of function or line, performing the execution of the code to test the application; and transmitting, to the master terminal device, a result of testing the application in accordance with the execution of the code.

Abstract: Software defect verification is disclosed. An example method includes accessing, with a processor, a script representative of a set of actions to be performed when executing a software application to be tested, the set of actions being associated with a reported defect, executing the software application to be tested on the computer, and performing, with the processor, the set of actions in the script via the application to be tested to attempt to reproduce the reported defect.

Abstract: Systems and methods for testing an integrated software system. A mock object (18) includes an aspect joined to a corresponding object (16) within the integrated software system. The mock object (18) intercepts a method call associated with the corresponding object (16) and routes the method call to one of the corresponding object and programmed behavior associated with the mock object. A testing agent (20) instructs the aspect to route the method call to the one of the method associated with corresponding object (16) and the programmed behavior associated with the mock object (18).

Abstract: Arrangements described herein relate to adopting an existing automation script to a new framework. A first version of an automation script configured to execute in a first framework, and expected unit test results for the automation script, are received. A second version of the automation script is created from the first version of the automation script. The second version of the automation script is configured to execute in at least a second framework. The first version of the automation script is executed in the first framework to run a first version of a unit test and the second version of the automation script is executed in the second framework to run a second version of the unit test. Results generated by execution of the first and second versions of the automation script are analyzed to validate the second version of the automation script.

Abstract: A system, apparatus and method for reporting a failure in execution of a software transaction (e.g., scripts and executables to install software on a computer) to a network resource that can then store the error-related information and automatically provide a previously determined solution to the error or links to resources to which the customer can refer for help in resolving the problem is provided. Automatically searching a database of reported errors and their solutions for a variety of products and providing a found solution to a customer experiencing an error (e.g., an installation error) is further provided. In the event that a solution to a customer-experienced error is not found by the network resource, alternate sites for information such as a customer support website can also be provided.

Abstract: A method for generating a replayable testing script for iterative use by an automated testing utility may include recording a plurality of scripts, each script relating to a separate iteration of a transaction between a user and a tested application performed by an operator. The method may also include comparing the recorded scripts to identify a location of a data item by finding different values in a pair of corresponding locations in the recorded scripts, indicative of a dynamic data item. The method may further include generating the replayable testing script comprising one of the recorded scripts and having a variable parameter at the identified location of the dynamic data item.

Abstract: There is provided in an embodiment of the present invention a method wherein a processor may receive a test configuration deployment plan (TCDP) template. A user may select the test configuration deployment plan template and provide time schedule, and specific software application names and version numbers as settings for variables within the test configuration deployment plan template, in effect making a subscription request, wherein the processor receives or otherwise prompts to receive such information. Provided such resources are available at the requested time, the processor may wait until that time and provision the test configuration deployment plan to a data center resource. Test cases are run and results are collected until time or test cases are exhausted. The processor may then release resources that had been allocated to the subscription, which may include a step of undeploying or unprovisioning.

Abstract: A computing device obtains a delta between a first version of a target program and a second version of the target program that has been tested by one or more test routines. The computing device identifies a first portion of the second version of the target program that has not been tested in the first version of the target program and a second portion of the second version of the target program that has been tested in the first version of the target program based on the delta. The computing device executes the one or more test routines on at least the first portion of the second version of the target program without executing the one or more test routines on the second portion of the second version of the target program.

Abstract: Autonomous media version testing is described. A method may include testing, by a processing device of a server and without human interaction, a plurality of versions of a game, each having a different set of test conditions, using information received from play of the plurality of versions of the game after a first game move has been made in the game. The method may also include determining, by the processing device and without human interaction, which of the plurality of versions of the game to publicly release based on the testing.

Abstract: A system, method and computer program product to predict elapsed response time of a query during an application stage is disclosed. The query is executed over a database and is classified based on a mode of access of the database by the query. The database is emulated to obtain an emulated database. After classification, an access pattern of the database by the query along with one or more parameters is determined. By applying a classification specific technique over the access pattern and the one or more parameters, an input-access time of the query on the emulated database is then calculated to determine the elapsed response time of the query.

Abstract: A method, system, and/or computer program product tests combined code changesets in a software product. One or more combinations of two or more changesets are selected, wherein a changeset is a set of changes to a software product submitted by a single developer source. For each combination of two or more changesets, an interaction between changesets is calculated, wherein the interaction is an overlapping of code found in two or more changesets. A combination of two or more changesets that has a predetermined minimum interaction between changesets is selected for building and testing.

Abstract: A test script generation system may generate end-to-end test scripts to test a software application. A workflow generator may generate recommendations of suggested modules corresponding to domain-specific reference application maps to include in a workflow. A visual representation of the workflow may be generated and provide an indication of suggested modules that have not been included in the workflow. A script generator may generate the end-to-end test scripts based on the workflow.

Abstract: Embodiments herein provide a method and system to create an optimized test suite for software testing. This system fetches required input parameters such as risk parameters, release type of the application, requirement details, test case details, requirement to test case relation and so on automatically using any suitable tool. Then, first level optimized test suite is formed by removing redundant and obsolete test cases from test case set. Further, probability of failure is calculated for each test case either manually or through automation and risk index value for each test case is defined. Further, test cases are classified based on value of risk index obtained. Further, second level optimized test suite is formed by using orthogonal array methodology. Furthermore, final optimized test suite with greater precision is prepared by considering execution time of iteration of all test cases along with their risk index values.

Abstract: A method is implemented in a computer infrastructure having computer executable code tangibly embodied on a computer readable storage medium having programming instructions. The programming instructions are operable to receive a tool error output determined by a code inspection tool and select at least one defect classification mapping profile based on the code inspection tool. Additionally, the programming instructions are operable to map the tool error output to one or more output classifications using the selected at least one defect classification mapping profile and generate at least one report based on the one or more output classifications.

Abstract: Mechanisms for the publication of web content are disclosed. Various stages can be used in a general publishing framework in moving content to publication. For instance, a development stage for the content can be used, where the development stage allows entities, such as developers, to code any materials associated with the content. Also, a testing stage can be used in order to test content developed in the development stage. Once these two stages are complete, a staging stage can be used for storing the content, and a production stage can be used for the actual publication of the content. Moreover, various interfaces can be used to allow entities to make requests to submit content and to log the state of such content in the framework. Lastly, various entities can collaborate on the publication of web content using the framework.

Abstract: A simulation environment, in one embodiment, includes a debugger server, one or more debuggers, and one or more debugger adapters. Each debugger adapter couples a corresponding debugger to the debugger server. The debugger server coordinates the run mode of the debugger adapters. Each debugger adapter controls the run mode of its corresponding debugger.

Abstract: A system for organizing messages related to tasks in a distributed application is disclosed. The system includes a work-list creator to create a work list of the top-level work items to be accomplished in performing a task. Work-item processors are distributed in the system. The work-item processors process the top-level work item included in a task and also append additional work items to the work list. A work-list scheduler invokes the work-item processors so local work-item processors are invoked prior to remote work-item processors.

Abstract: A method for debugging an application includes obtaining first and second fusible operation requests; if there is a break point between the first and the second operation request, generating a first set of compute kernels including programs corresponding to the first operation request, but not to the second operation request; and generating a second set of compute kernels including programs corresponding the second operation request, but not to the first operation request; if there is no break point between the first and the second operation request, generating a third set of compute kernels which include programs corresponding to a merge of the first and second operation requests; and arranging for execution of either the first and second, or the third set of compute kernels, further including debugging the first or second set of compute kernels when there is a break point set between the first and second operation requests.

Abstract: A plurality of new snoop transaction types are described. Some include address information in the requests, and others include cache entry information in the requests. Some responses include tag address information, and some do not. Some provide tag address content on the data bus lines during the data portion of the transaction. These new snoop transaction types are very helpful during debug of a data processing system.