Patents by Inventor Dale E. Blue

Dale E. Blue has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11436132
    Abstract: A method for testing a system under test (SUT) in an active environment to identify cause of a soft failure includes recording a first difference vector by executing a set of test cases on a baseline system and monitoring performance parameters of the baseline system before and after executing the test cases. Each performance record represents differences in the performance parameters of the baseline system from before and after the execution of a corresponding test case. The method further includes, similarly, recording a second difference vector by executing the test cases on the SUT and monitoring performance parameters of the SUT before and after executing the test cases. The method further includes identifying an outlier performance record from the second difference vector by comparing the difference vectors and further, determining a root cause of the soft failure by analyzing a test case corresponding to the outlier.
    Type: Grant
    Filed: March 16, 2020
    Date of Patent: September 6, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Ryan Thomas Rawlins, Dale E. Blue, Jacob Thomas Snyder
  • Patent number: 11422924
    Abstract: A method for identifying optimal tests includes defining functional coverage by a test suite based on a functional coverage model of a System Under Test (SUT). The test suite includes a plurality of tests. The functional coverage model includes a plurality of attributes, each attribute having a set of possible values. The functional coverage model defines possible combinations of values of the attributes as covered by the test suite. A subset of the possible combinations of values is determined. A subset of the plurality of tests is selected that is operative to cover the determined subset of the possible combinations of values. A plurality of trees is generated to represent the selected subset of tests. Complexity of the generated trees is analyzed based on user-specified criteria. An optimal tree is selected from the subset of the plurality of trees to achieve the objective of the test suite.
    Type: Grant
    Filed: June 13, 2019
    Date of Patent: August 23, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew Hicks, Dale E. Blue, Ryan Thomas Rawlins
  • Publication number: 20220261340
    Abstract: A method for testing a system under test (SUT) in an active environment to identify cause of a soft failure includes recording a first difference vector by executing a set of test cases on a baseline system and monitoring performance parameters of the baseline system before and after executing the test cases. Each performance record represents differences in the performance parameters of the baseline system from before and after the execution of a corresponding test case. The method further includes, similarly, recording a second difference vector by executing the test cases on the SUT and monitoring performance parameters of the SUT before and after executing the test cases. The method further includes identifying an outlier performance record from the second difference vector by comparing the difference vectors and further, determining a root cause of the soft failure by analyzing a test case corresponding to the outlier.
    Type: Application
    Filed: May 6, 2022
    Publication date: August 18, 2022
    Inventors: Andrew C. M. Hicks, RYAN THOMAS RAWLINS, Dale E. Blue, Jacob Thomas Snyder
  • Patent number: 11379352
    Abstract: Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (SAV) is computed based on the binary execution results. The method further includes outputting, to a user, in response to the success rate SAV of the attribute-value pair being below a predetermined threshold, an identification of one or more testcases that use the attribute-value pair, wherein the one or more testcases are to be used for diagnosing a soft failure associated with the SUT.
    Type: Grant
    Filed: December 15, 2020
    Date of Patent: July 5, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Kevin Minerley, Dale E. Blue, Ryan Thomas Rawlins, Daniel Nicolas Gisolfi
  • Publication number: 20220188219
    Abstract: Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (SAV) is computed based on the binary execution results. The method further includes outputting, to a user, in response to the success rate SAV of the attribute-value pair being below a predetermined threshold, an identification of one or more testcases that use the attribute-value pair, wherein the one or more testcases are to be used for diagnosing a soft failure associated with the SUT.
    Type: Application
    Filed: December 15, 2020
    Publication date: June 16, 2022
    Inventors: Andrew C. M. Hicks, Kevin Minerley, Dale E. Blue, RYAN THOMAS RAWLINS, Daniel Nicolas Gisolfi
  • Publication number: 20220188205
    Abstract: Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (SAV) is computed based on the binary execution results. in response to a success rate of an attribute-value pair being below a predetermined threshold, a subset of testcases that use the attribute-value pair is identified. Further, sets of code paths for the subset of testcases are identified, each set of code path respectively corresponding to a testcase from the subset of testcases. Further, an intersection of the sets of code paths is determined. Code paths of the SUT that are in the intersection, are highlighted to represent a soft failure with the SUT.
    Type: Application
    Filed: October 25, 2021
    Publication date: June 16, 2022
    Inventors: Andrew C. M. Hicks, Kevin Minerley, Dale E. Blue, Ryan Thomas Rawlins, Daniel Nicolas Gisolfi
  • Patent number: 11307975
    Abstract: According to one or more embodiments of the present invention, a computer-implemented method for machine code analysis includes executing a set of test cases associated with a software product. The method further includes determining a failing test case, from the set of test cases. The method further includes identifying a portion of a machine code of the software product, the portion of the machine code corresponding to the failing test case. The method further includes analyzing the portion of the machine code to identify a pattern of machine code causing the failing test case to fail. The method further includes searching the machine code of the software product to find the identified pattern of machine code.
    Type: Grant
    Filed: February 20, 2020
    Date of Patent: April 19, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Brian Mo, Andrew C. M. Hicks, Ryan Thomas Rawlins, Dale E. Blue
  • Patent number: 11263116
    Abstract: Systems, methods, and computer-readable media for identifying a champion test case that provides an increased likelihood of exposing a fault and expanding a set of test cases to include the champion test case are described. The fault may occur in a System Under Test (SUT), which may be a hardware system or a software system. A weight may be assigned to a champion test case that indicates the relative strength of the test case in detecting faults. The weight may be used to influence the selection of other test cases.
    Type: Grant
    Filed: January 24, 2019
    Date of Patent: March 1, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew Hicks, Dale E. Blue, Ryan Rawlins
  • Patent number: 11232020
    Abstract: Systems, methods, and computer-readable media are described for utilizing breakpoint value-based fingerprints of failing regression test cases to determine specific components of a System Under Test (SUT) that are causing a fault such as specific lines of source code. A failing test case from a regression run is selected and fault localization and inverse combinatorics techniques are employed to generate a set of failing test cases around the selected failing test case. A set of test fingerprints corresponding to the set of failing test cases is compared to a set of test fingerprints corresponding to a set of passing test cases to determine breakpoints that are indicated as being encountered during execution of at least one failing test case and that are not encountered during execution of any of the passing test cases. Specific lines of source code that correspond to these breakpoints are then identified as causing the fault.
    Type: Grant
    Filed: June 13, 2019
    Date of Patent: January 25, 2022
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew Hicks, Dale E. Blue, Ryan Thomas Rawlins, Steven M. Partlow
  • Patent number: 11204848
    Abstract: Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (SAV) is computed based on the binary execution results. in response to a success rate of an attribute-value pair being below a predetermined threshold, a subset of testcases that use the attribute-value pair is identified. Further, sets of code paths for the subset of testcases are identified, each set of code path respectively corresponding to a testcase from the subset of testcases. Further, an intersection of the sets of code paths is determined. Code paths of the SUT that are in the intersection, are highlighted to represent a soft failure with the SUT.
    Type: Grant
    Filed: December 15, 2020
    Date of Patent: December 21, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Kevin Minerley, Dale E. Blue, Ryan Thomas Rawlins, Daniel Nicolas Gisolfi
  • Patent number: 11194703
    Abstract: A method for testing a system under test (SUT) in an active environment includes generating, by a testing system, a set of tests for testing the SUT, the tests generated based on a coverage model of the SUT, wherein the coverage model uses several attributes. The method further includes creating, by the testing system, a minimal set of tests from the generated tests by selecting tests for a disjoint set of attributes from the several attributes of the coverage model. The method further includes executing, by the testing system, the minimal set of tests on the SUT for analyzing a soft failure of the SUT in the active environment, wherein the soft failure occurs in the active environment during execution of the SUT based at least in part on a parameter of the active environment.
    Type: Grant
    Filed: March 16, 2020
    Date of Patent: December 7, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Ryan Thomas Rawlins, Dale E. Blue
  • Patent number: 11194704
    Abstract: A method for testing a system under test (SUT) in an active environment includes generating, by a testing system, several tests for testing the SUT. The tests are generated based on a coverage model of the SUT, which includes multiple attributes. The method further includes creating, by the testing system, a minimal set of tests from the tests by selecting tests that do not exceed a predetermined performance threshold. The method further includes executing, by the testing system, the minimal set of tests on the SUT for analyzing a soft failure of the SUT in the active environment. The soft failure occurs in the active environment during execution of the SUT based at least in part on a performance parameter of the active environment.
    Type: Grant
    Filed: March 16, 2020
    Date of Patent: December 7, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Ryan Thomas Rawlins, Dale E. Blue, Jacob Thomas Snyder
  • Patent number: 11188453
    Abstract: A system and related method comprise using a processor for executing a plurality of tests associated with a covering array of a test framework of the software test system on a first version of a system under test (SUT). For each of the plurality of tests, on a current test, the method comprises determining a current success rate value (SRV) for the current test that represents a success rate of the current test for the first version of the SUT. The method further comprises combining the current SRV of the first version of the SUT and current SRVs of the current test for prior versions of the SUT into a current test eigenvector associated with the current test. The method further comprises converting the current test eigenvector into a first eigenvalue that represents a health, accuracy, and quality of the first version of the SUT.
    Type: Grant
    Filed: December 15, 2020
    Date of Patent: November 30, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Kevin Minerley, Dale E. Blue, Ryan Thomas Rawlins, Daniel Nicolas Gisolfi
  • Patent number: 11176026
    Abstract: A method for assigning test case priority includes analyzing, based on a set of test vectors, one or more test cases from a set of test cases on source code to determine a particular combination of attribute values associated with the one or more analyzed test cases. The method further includes generating a priority value for each attribute in the determined particular combination of attribute values. A priority value for each of the analyzed one or more test cases is generated based on the generated priority values of the particular combination of attribute values associated with the analyzed one or more test cases.
    Type: Grant
    Filed: February 20, 2020
    Date of Patent: November 16, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Ryan Thomas Rawlins, Dale E. Blue, Brian Mo
  • Patent number: 11132273
    Abstract: Inputs to a system under test (SUT) are modeled as a collection of attribute-value pairs. A set of testcases is executed using an initial set of test vectors that provides complete n-wise coverage of the attribute-value pairs. For each execution of the testcases, for each attribute-value pair, a non-binary success rate (SAV) is computed based on the binary execution results. An attribute is selected in response to a set of success rates corresponding to a set of attribute-value pairs that includes said attribute are all below a predetermined threshold. The set of testcases is executed using another set of test vectors using additional values for the selected attribute. For each execution of the set of testcases, for each attribute-value pair, a second non-binary success rate (SAV?) is recorded. If the predetermined threshold is now satisfied, a user is notified of the additional values for the attribute that were detected.
    Type: Grant
    Filed: December 15, 2020
    Date of Patent: September 28, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Andrew C. M. Hicks, Kevin Minerley, Dale E. Blue, Ryan Thomas Rawlins, Daniel Nicolas Gisolfi
  • Publication number: 20210286713
    Abstract: A method for testing a system under test (SUT) in an active environment includes generating, by a testing system, several tests for testing the SUT. The tests are generated based on a coverage model of the SUT, which includes multiple attributes. The method further includes creating, by the testing system, a minimal set of tests from the tests by selecting tests that do not exceed a predetermined performance threshold. The method further includes executing, by the testing system, the minimal set of tests on the SUT for analyzing a soft failure of the SUT in the active environment. The soft failure occurs in the active environment during execution of the SUT based at least in part on a performance parameter of the active environment.
    Type: Application
    Filed: March 16, 2020
    Publication date: September 16, 2021
    Inventors: Andrew C. M. Hicks, RYAN THOMAS RAWLINS, Dale E. Blue, Jacob Thomas Snyder
  • Publication number: 20210286710
    Abstract: A method for testing a system under test (SUT) in an active environment includes generating, by a testing system, a set of tests for testing the SUT, the tests generated based on a coverage model of the SUT, wherein the coverage model uses several attributes. The method further includes creating, by the testing system, a minimal set of tests from the generated tests by selecting tests for a disjoint set of attributes from the several attributes of the coverage model. The method further includes executing, by the testing system, the minimal set of tests on the SUT for analyzing a soft failure of the SUT in the active environment, wherein the soft failure occurs in the active environment during execution of the SUT based at least in part on a parameter of the active environment.
    Type: Application
    Filed: March 16, 2020
    Publication date: September 16, 2021
    Inventors: Andrew C. M. Hicks, RYAN THOMAS RAWLINS, Dale E. Blue
  • Publication number: 20210286714
    Abstract: A method for testing a system under test (SUT) in an active environment to identify cause of a soft failure includes recording a first difference vector by executing a set of test cases on a baseline system and monitoring performance parameters of the baseline system before and after executing the test cases. Each performance record represents differences in the performance parameters of the baseline system from before and after the execution of a corresponding test case. The method further includes, similarly, recording a second difference vector by executing the test cases on the SUT and monitoring performance parameters of the SUT before and after executing the test cases. The method further includes identifying an outlier performance record from the second difference vector by comparing the difference vectors and further, determining a root cause of the soft failure by analyzing a test case corresponding to the outlier.
    Type: Application
    Filed: March 16, 2020
    Publication date: September 16, 2021
    Inventors: Andrew C. M. Hicks, Ryan Thomas Rawlins, Dale E. Blue, Jacob Thomas Snyder
  • Publication number: 20210286711
    Abstract: A method for testing a system under test (SUT) in an active environment includes receiving, by a testing system, a code path of the SUT that causes a soft failure in the active environment. The soft failure occurs in the active environment during execution of the SUT based at least on a parameter of the active environment. The method further includes generating, by the testing system, multiple tests for testing the SUT, the tests generated based on a coverage model of the SUT, wherein the coverage model uses several attributes. The method further includes selecting, by the testing system, from the generated tests, a set of tests that are associated with the code path. The method further includes executing, by the testing system, only the set of tests that are selected on the SUT to analyze a cause of the soft failure.
    Type: Application
    Filed: March 16, 2020
    Publication date: September 16, 2021
    Inventors: Andrew C. M. Hicks, RYAN THOMAS RAWLINS, Dale E. Blue, Kevin Minerley
  • Publication number: 20210286712
    Abstract: A method for testing a system under test (SUT) in an active environment includes executing, by the testing system, on the SUT, a test from a set of tests. The method further includes, monitoring a first execution time to complete the test on the SUT in the active environment. Based on the first execution time being different than a second execution time of the test, marking, by the testing system, a code path associated with the test. The second execution time is a duration to complete execution of the test on the SUT in a clean execution environment. The method further includes communicating, by the testing system, the code path for analyzing a soft failure of the SUT in the active environment, wherein the soft failure occurs in the active environment during execution of the SUT based at least on a parameter of the active environment.
    Type: Application
    Filed: March 16, 2020
    Publication date: September 16, 2021
    Inventors: Andrew C. M. Hicks, RYAN THOMAS RAWLINS, Dale E. Blue, Kevin Minerley