Abstract: A computing device receives a request for a design of an experiment. The device generates a data representation of a matrix Y that defines a supersaturated design for the design of the experiment. The generating the data representation is by: generating a data representation of a matrix X according to an obtained design; computing an indication of correlation between effects of factors of a matrix Y; and generating, based on the indication of correlation, the data representation of the matrix Y that is the transposition of the matrix X or is the transposition of a matrix X*. The matrix X* is a first subset of the matrix X such that the transposition of the matrix X* represents a same number of factors as the transposition of the matrix X. The device outputs a setting for each test condition of the supersaturated design for the experiment.

Abstract: A computing system generates a subset of design cases of candidate design cases. The system indexes, in the subset, data elements. The system generates a design of an experiment by, for each respective data element, determining a status indicating whether the respective data element corresponds to an uncontrolled factor or a controlled factor. When the status indicates the uncontrolled factor, the system determines if substituting a respective set of specified options of a respective candidate design case comprising the respective data element with a different set of specified options of the candidate design cases improves a criterion measure according to a design criterion. When the status indicates the controlled factor, the system determines if changing an assigned option of the respective data element improves the criterion measure. The system updates the criterion measure with an updated criterion measure according to a change of the subset based on generating the design.

Abstract: A computing system receives a request to validate, according to a validation specification, a response of a system of operation. The validation specification relates to determining deviation from a specified result in response to testing one or more candidate inputs. The initial design space defines design points for the system of operation according to the respective candidate inputs for each factor. The computing system, responsive to the request to validate the response of the system of operation, generates data representing a covering array for design points defined by the initial design space or a subset design space. The computing system generates, based on the data, a test suite for testing the system of operation. The computing system obtains the response of the system of operation and generates an output indicating the deviation from the specified result.

Abstract: A computing system determines a design space for designing a design system. The computing system receives a request to identify input(s) that will provide a response of the design system that advances a user-defined goal for the design system. The computing system, responsive to the request to identify input(s), generates a design for the design space that provides design cases for generating the response of the design system and obtains responses of the design system generated for multiple design cases of the design for the design system. The computing system selects at least one design case based on obtained responses of the design system for the multiple design cases, and based on the at least one design case, outputs an indication of suggested input(s) for the design system. The suggested input(s) advances the user-defined goal for the design system and is not disallowed according to disallowed combination(s).

Abstract: A computing system receives a request for a computer-generated design of an experiment. A design space is defined by candidate inputs for each factor of the experiment. The system receives a disallowed combination indication indicating a user-defined disallowed combination for the design space. The disallowed combination constrains a first set of values from a first set of candidate inputs from being assigned to the first factor if the second factor is assigned one of a second set of values from a second set of candidate inputs. The system determines additional constraint(s) on computer generation of the design. The system evaluates if a user-defined model can be generated according to the disallowed combination and the constraint(s). The system generates a computer suggested modification of the user-defined model such that the design can be generated according to the computer suggested modification.

Abstract: A computing system receives a request to validate, according to a validation specification, a response of a system of operation. The validation specification relates to determining deviation from a specified result in response to testing one or more candidate inputs. The initial design space defines design points for the system of operation according to the respective candidate inputs for each factor. The computing system, responsive to the request to validate the response of the system of operation, generates data representing a covering array for design points defined by the initial design space or a subset design space. The computing system generates, based on the data, a test suite for testing the system of operation. The computing system obtains the response of the system of operation and generates an output indicating the deviation from the specified result.

Abstract: An apparatus may include a processor caused to: receive indications of selection of an experiment design for regression analysis, of a type of distribution for a simulation of random data in the regression analysis, and of selection of a number of iterations of the simulation of random data; generate executable instructions in a pre-selected programming language to be executable by the processor to perform the regression analysis with the selected number of iterations of simulation of random data and with the selected type of distribution; generate a human readable form of a portion of the first executable instructions that includes the coefficients and terms in mathematical notation, and that specifies the selected number of iterations and the selected type of distribution for the simulation of random data; and present, on a display communicatively coupled to the processor, the human readable form of the portion of the first executable instructions.

Abstract: A computing device receives factor information indicating multiple factors (e.g., hyperparameters for designing a system comprising a machine learning algorithm). The computing device receives range information indicating initial ranges with a range for each of possible options for the multiple factors. The computing device obtains a space-filling design for the design space. The space-filing design indicates selected design points in the design space. Each of the selected design points represents assigned options assigned to the multiple factors. The assigned options are assigned from the initial ranges. The computing device generates, based on the space-filling design, an initial design suite that provides initial design cases corresponding to one or more of the selected design points. The computing device generates evaluations of the initial design cases. The computing device outputs, based on the evaluations of the initial design cases, an indication of a selected design case.

Abstract: A computing device receives a request for a design of an experiment. The device generates a data representation of a matrix Y that defines a supersaturated design for the design of the experiment. The generating the data representation is by: generating a data representation of a matrix X according to an obtained design; computing an indication of correlation between effects of factors of a matrix Y; and generating, based on the indication of correlation, the data representation of the matrix Y that is the transposition of the matrix X or is the transposition of a matrix X*. The matrix X* is a first subset of the matrix X such that the transposition of the matrix X* represents a same number of factors as the transposition of the matrix X. The device outputs a setting for each test condition of the supersaturated design for the experiment.

Abstract: A computing device receives a request for a design of an experiment. The design comprises test cases with test conditions for testing factors for the experiment. The device receives a value for a parameter of multiple parameters for the design. The multiple parameters indicate a total number of test cases for the design, a total number of factors for the design, and a total number of groups for grouping factors. The device generates a value for each of one or more other parameters of the multiple parameters such that the design is a supersaturated design. The device generates, based on the first value for the first parameter and the value for each of the one or more other parameters, the supersaturated design for the experiment that is a design that distributes each of the factors into one of the groups. The device outputs an indication of the supersaturated design.

Abstract: A computing device receives data comprising inputs representing a respective option for each of factors in each of test cases. The data comprises a response of the system for each of the test cases. The computing device receives a request requesting an evaluation of the data for generating a model (e.g. a machine learning algorithm) to predict responses based on the factors. The computing device obtains different group identifiers for each of groups for distributing the test cases for the system (e.g., groups of a K-fold cross-validation). The computing device for each of validation(s): generates a data set comprising a respective data element for each of the test cases of the plurality of test cases; and controls assignment of a group identifier of the different group identifiers to each of the respective data elements. The computing device outputs an indication of one or more generated data sets for the validation(s).

Abstract: A computing device receives data comprising inputs representing a respective option for each of factors in each of test cases. The data comprises a response of the system for each of the test cases. The computing device receives a request requesting an evaluation of the data for generating a model (e.g. a machine learning algorithm) to predict responses based on the factors. The computing device obtains different group identifiers for each of groups for distributing the test cases for the system (e.g., groups of a K-fold cross-validation). The computing device for each of validation(s): generates a data set comprising a respective data element for each of the test cases of the plurality of test cases; and controls assignment of a group identifier of the different group identifiers to each of the respective data elements. The computing device outputs an indication of one or more generated data sets for the validation(s).

Abstract: A computing device receives factor information indicating multiple factors (e.g., hyperparameters for designing a system comprising a machine learning algorithm). The computing device receives range information indicating initial ranges with a range for each of possible options for the multiple factors. The computing device obtains a space-filling design for the design space. The space-filing design indicates selected design points in the design space. Each of the selected design points represents assigned options assigned to the multiple factors. The assigned options are assigned from the initial ranges. The computing device generates, based on the space-filling design, an initial design suite that provides initial design cases corresponding to one or more of the selected design points. The computing device generates evaluations of the initial design cases. The computing device outputs, based on the evaluations of the initial design cases, an indication of a selected design case.

Abstract: A computing device obtains a metric N indicating a quantity of a plurality of test cases for an output design of an experiment Each element of a test case of the output design is a test condition for testing one of factors for the experiment. The computing device obtains input indicating a quantity p of an indicated plurality of factors for the output design. The computing device determines whether there are stored instructions for generating an initial screening design for the experiment. The computing device responsive to determining that there are stored instructions, selects, using the stored instructions, the initial screening design for the experiment. The computing device determines whether to modify the initial screening design based on modification criteria comprising a secondary criterion, the metric N, and/or the quantity p. The computing device outputs an indication of the updated screening design for the output design of the experiment.

Abstract: A computing device generates representative points, each representing a potential design point for a design space. The computing device determines for the design space primary clusters, a categorical factor, and at least two levels for the categorical factor. The computing device, for each of the primary clusters, selects a design point from each sub-cluster of the respective primary cluster. The computing device, for each of the primary clusters, allocates the at least two levels of the categorical factor, such that a level of the at least two levels is allocated to each selected design point in the respective primary cluster. The computing device modifies an initial sub-design that represents the selected design points allocated a given level of the categorical factor by increasing separation between design points allocated a same level of the categorical factor. The computing device outputs to an output device a modified design for the design space.

Abstract: A computing device obtains a metric N indicating a quantity of a plurality of test cases for an output design of an experiment. Each element of a test case of the output design is a test condition for testing one of factors for the experiment. The computing device obtains input indicating a quantity p of an indicated plurality of factors for the output design. The computing device determines whether there are stored instructions for generating an initial screening design for the experiment. The computing device responsive to determining that there are stored instructions, selects, using the stored instructions, the initial screening design for the experiment. The computing device determines whether to modify the initial screening design based on modification criteria comprising a secondary criterion, the metric N, and/or the quantity p. The computing device outputs an indication of the updated screening design for the output design of the experiment.

Abstract: A computing device generates representative points, each representing a potential design point for a design space. The computing device determines for the design space primary clusters, a categorical factor, and at least two levels for the categorical factor. The computing device, for each of the primary clusters, selects a design point from each sub-cluster of the respective primary cluster. The computing device, for each of the primary clusters, allocates the at least two levels of the categorical factor, such that a level of the at least two levels is allocated to each selected design point in the respective primary cluster. The computing device modifies an initial sub-design that represents the selected design points allocated a given level of the categorical factor by increasing separation between design points allocated a same level of the categorical factor. The computing device outputs to an output device a modified design for the design space.

Abstract: The computing device receives information representing a design of an experiment. The design of the experiment comprises a plurality of test cases. Each element of a test case of the design is a test condition for testing one of factors for the experiment. The computing device generates a graphical representation based on the information. The graphical representation comprises a plurality of axes for respective ones of factors used in a fractal sequence. Each of the plurality of axes has two endpoints comprising a first endpoint that corresponds to a first level of a respective factor, and a second endpoint that corresponds to a second level of the respective factor. The computing device plots, on the graphical representation, data corresponding to one or more test cases of the plurality of test cases. The computing device displays an interactive graphical user interface comprising the graphical representation with the plotted data.

Abstract: The computing device generates a test suite that provides test cases for testing a system. A test condition in the test suite comprises one of different levels representing different options assigned to a categorical factor for the system. The computing device receives input weights for one or more levels of the test suite. The computing device receives a failure indication indicating a test conducted according to the test cases failed. The computing device determines a plurality of cause indicators based on the input weights and any commonalities between test conditions of any failed test cases of the test suite that resulted in a respective failed test outcome. The computing device identifies, based on comparing the plurality of cause indicators, a most likely potential cause for a potential failure of the system. The computing device outputs an indication of the most likely potential cause for the potential failure of the system.

Abstract: An apparatus may include a processor caused to: receive indications of first and second experiment designs to be compared; for each factor of the model of the first experiment design, identify a matching factor of the model of the second experiment design based on factor type, wherein the factor type is selected from the group consisting of a categorical factor and a continuous factor; for each categorical factor of the model of the first experiment design, identify a matching factor of the model of the second experiment design additionally based on quantity of levels of each factor; for each term of the model of the first experiment design, identify a matching term of the model of the second experiment design based on an order of each term; and present, on a display, the identified matches between the terms and between the responses of the first and second experiment designs.