Abstract: Systems and processes for facilitating the sharing of models trained on a data set confined within a given firewall, i.e., a hidden data set, along with the model's performance metrics are described. The trained models may be used in further processes to improve the trained models to solve a predetermined problem or make a prediction.

Abstract: Roughly described, in an evolutionary technique for finding optimal solutions to a provided problem, a computer system uses a grouping algorithm that is better able to find diverse and optimum solutions in data mining environment with multiple solution landscapes and a plurality of candidate individuals. Each candidate individual identifies with a potential solution, and is associated with a testing experience level and one or more partition tags. Each candidate individual is assigned into one of a plurality of competition groups in dependence upon the individual's testing experience level and partition tag. During competition among candidate individuals, a candidate individual can only replace another candidate individual if both the candidate individuals have a common partition tag and are in the same competition group. A candidate individual cannot replace another candidate individual if they have different partition tags or are in different competition groups.

Abstract: Roughly described, a computer-implemented evolutionary system evolves candidate solutions to provided problems. It includes a memory storing a candidate gene database containing active and epigenetic individuals; a gene pool processor which tests only active individuals on training data and updates their fitness estimates; a competition module which selects active individuals for discarding in dependence upon both their updated fitness estimate and their testing experience level; and a gene harvesting module providing for deployment selected ones of the individuals from the gene pool. The gene database has an experience layered elitist pool, and individuals compete only with other individuals in their same layer. Certain individuals are made epigenetic in the procreation module, after which they are not subjected to testing and competition. Epigenetic individuals are retained in the candidate gene pool regardless of their fitness.

Abstract: In many environments, rules are trained on historical data to predict an outcome likely to be associated with new data. Described is a ruleset which predicts the probability of a particular outcome. Roughly described, an individual identifies a ruleset, where each of the rules has a plurality of conditions and also indicates a rule-level probability of a predetermined classification. The conditions indicate a relationship (e.g., ‘<’ or ‘!<’) between an input feature and a corresponding value. The rules are evaluated against input data to derive a certainty for each condition, and aggregated to a rule-level certainty. The rule probabilities are combined using the rule-level certainty values to derive a probability output for the ruleset, which can be used to provide a basis for decisions. In an embodiment, the per-condition certainty values are fuzzy values aggregated by fuzzy logic. A novel genetic algorithm can be used to derive the ruleset.

Abstract: The embodiments describe a technique for customizing activation functions automatically, resulting in reliable improvements in performance of deep learning networks. Evolutionary search is used to discover the general form of the function, and gradient descent to optimize its parameters for different parts of the network and over the learning process. The new approach discovers new parametric activation functions which improve performance over previous activation functions by utilizing a flexible search space that can represent activation functions in an arbitrary computation graph. In this manner, the activation functions are customized to both time and space for a given neural network architecture.

Abstract: The technology disclosed relates to evolving deep neural network structures. A deep neural network structure includes a plurality of modules with submodules and interconnections among the modules and the submodules. In particular, the technology disclosed relates to storing candidate genomes that identify respective values for a plurality of hyperparameters of a candidate genome. The hyperparameters include global topology hyperparameters, global operational hyperparameters, local topology hyperparameters, and local operational hyperparameters. It further includes evolving the hyperparameters by training, evaluating, and procreating the candidate genomes and corresponding modules and submodules.

Abstract: The technology disclosed identifies parallel ordering of shared layers as a common assumption underlying existing deep multitask learning (MTL) approaches. This assumption restricts the kinds of shared structure that can be learned between tasks. The technology disclosed demonstrates how direct approaches to removing this assumption can ease the integration of information across plentiful and diverse tasks. The technology disclosed introduces soft ordering as a method for learning how to apply layers in different ways at different depths for different tasks, while simultaneously learning the layers themselves. Soft ordering outperforms parallel ordering methods as well as single-task learning across a suite of domains. Results show that deep MTL can be improved while generating a compact set of multipurpose functional primitives, thus aligning more closely with our understanding of complex real-world processes.

Abstract: Roughly described, a computer system uses a behavior-driven algorithm that is better able to find optimum solutions to a problem by balancing the use of fitness and novelty measures in evolutionary optimization. In competition among candidate individuals, a domination estimate between a pair of individuals is determined by both their fitness estimate difference and their behavior difference relative to one another. An increase in the fitness estimate difference of one individual of the pair over the other increases the domination estimate of the first individual. An increase in the behavior distance between the pair of individuals decreases the domination estimate of both of the individuals. Individuals with a higher domination estimate are more likely to survive competitions among the candidate individuals.

Abstract: The technology disclosed relates to evolving a deep neural network based solution to a provided problem. In particular, it relates to providing an improved cooperative evolution technique for deep neural network structures. It includes creating blueprint structures that include a plurality of supermodule structures. The supermodule structures include a plurality of modules. The modules are neural networks. A first loop of evolution executes at the blueprint level. A second loop of evolution executes at the supermodule level. Further, multiple mini-loops of evolution execute at each of the subpopulations of the supermodules. The first loop, the second loop, and the mini-loops execute in parallel.

Abstract: The present invention relates to an ESP decision optimization system for epidemiological modeling. ESP based modeling approach is used to predict how non-pharmaceutical interventions (NPIs) affect a given pandemic, and then automatically discover effective NPI strategies as control measures. The ESP decision optimization system comprises of a data-driven predictor, a supervised machine learning model, trained with historical data on how given actions in given contexts led to specific outcomes. The Predictor is then used as a surrogate in order to evolve prescriptor, i.e. neural networks that implement decision policies (i.e. NPIs) resulting in best possible outcomes. Using the data-driven LSTM model as the Predictor, a Prescriptor is evolved in a multi-objective setting to minimize the pandemic impact.

Abstract: An explainable surrogate-assisted evolutionary optimization method, E-ESP, discovers rule-based decision strategies for which actions to take to achieve certain outcomes when historical training data is limited or unavailable. The resulting rules are human readable and thus facilitate explainability and trustworthiness unlike the black box solutions resulting from neural network solutions.

Abstract: A system and method for evolving a recurrent neural network (RNN) that solves a provided problem includes: a memory storing a candidate RNN genome database having a pool of candidate RNN nodes, each of the candidate RNN nodes representing a neural network as a unique tree structure; an assembly module that assembles N RNN layers; an evolution module that evolves the H candidate RNN nodes of each respective RNN layer; a training module that trains the candidate RNN nodes of each of the N RNN layers using training data; an evaluation module that evaluates a performance of each candidate RNN node of each RNN layer using validation data and assigns a fitness value to each candidate RNN node; a competition module that forms an elitist pool of candidate RNN nodes in dependence on their assigned fitness values; and a solution harvesting module providing for deployment of RNN layers instantiated with candidate RNN nodes from the elitist pool.

Abstract: A system and process for generalizing an evolutionary process applied to a particular domain involving different problems includes a researcher module for generating a configuration specification applicable to a particular problem. An evolution module parses the configuration specification into a representative tree structure, assembles policies for each node in the tree structure, and generates candidate genomes using the policies for each node in the tree structure. The policies may be applied to new data or data from prior runs to generate candidate genomes. The evolution module translates internal representations of the generated candidate genomes into known representations of the candidate genome for evaluation in accordance with the particular domain parameters by a candidate evaluation module.

Abstract: A system for outputting an action signal to a controlled system is provided. The system includes a memory storing individuals to be deployed to a production environment as an actor, wherein each of the individuals has a rule associated therewith for asserting an action, and the actor includes one or more individuals, is associated with the controlled system and is configured to transmit an intermediate action signal for asserting the action. The system includes a management server configured to receive the intermediate action signal, select, from a set of available operations, a selected operation to perform with respect to the intermediate action signal, and the set of available operations including allowance and a blocking of the intermediate action signal. Further, in response to the selected operation being the allowance, transmitting the intermediate action signal, and in response to the selected operation being the blocking, blocking the intermediate action signal.

Abstract: User-driven exploration functionality, referred to herein as a Scratchpad, is a post-learning extension for machine learning systems. For example, in ESP, consisting of the Predictor (a surrogate model of the domain) and Prescriptor (a solution generator model), the Scratchpad allows the user to modify the suggestions of the Prescriptor, and evaluate each such modification interactively with the Predictor. Thus, the Scratchpad makes it possible for the human expert and the AI to work together in designing better solutions. This interactive exploration also allows the user to conclude that the solutions derived in this process are the best found, making the process trustworthy and transparent to the user.

Abstract: A system and process for generalizing an evolutionary process applied to a particular domain involving different problems includes a researcher module for generating a configuration specification applicable to a particular problem. An evolution module parses the configuration specification into a representative tree structure, assembles policies for each node in the tree structure, and generates candidate genomes using the policies for each node in the tree structure. The policies may be applied to new data or data from prior runs to generate candidate genomes. The evolution module translates internal representations of the generated candidate genomes into known representations of the candidate genome for evaluation in accordance with the particular domain parameters by a candidate evaluation module.

Abstract: Evolution and coevolution of neural networks via multitask learning is described. The foundation is (1) the original soft ordering, which uses a fixed architecture for the modules and a fixed routing (i.e. network topology) that is shared among all tasks. This architecture is then extended in two ways with CoDeepNEAT: (2) by coevolving the module architectures (CM), and (3) by coevolving both the module architectures and a single shared routing for all tasks using (CMSR). An alternative evolutionary process (4) keeps the module architecture fixed, but evolves a separate routing for each task during training (CTR). Finally, approaches (2) and (4) are combined into (5), where both modules and task routing are coevolved (CMTR).

Abstract: A system and method for evolving a deep neural network structure that solves a provided problem includes: a memory storing a candidate supermodule genome database having a pool of candidate supermodules having values for hyperparameters for identifying a plurality of neural network modules in the candidate supermodule and further storing fixed multitask neural networks; a training module that assembles and trains N enhanced fixed multitask neural networks and trains each enhanced fixed multitask neural network using training data; an evaluation module that evaluates a performance of each enhanced fixed multitask neural network using validation data; a competition module that discards supermodules in accordance with assigned fitness values and saves others in an elitist pool; an evolution module that evolves the supermodules in the elitist pool; and a solution harvesting module providing for deployment of a selected one of the enhanced fixed multitask neural networks, instantiated with supermodules selected fr

Abstract: A genetic algorithm is described to determine a near-optimal schedule for assigning heterogeneous computations to resources in a heterogeneous computational environment. The genetic algorithm evolves a computation-to-compute resource mapping optimized with respect to a set of inter-dependent, and possibly conflicting objectives including cost of computation, data transfer cost, time to complete computation, profitability, etc. A set of scheduling plans are randomly created and then evaluated and assigned a fitness value based on the ability to meet a set of weighted objectives. Scheduling plans having desirable fitness values are selected as parents to procreate one or more new scheduling plans, each new plan inheriting resource mappings from at least two parents. This evolutionary process is repeated until the fitness values across scheduling plans converge or a time threshold is exceeded. At the end of evolution, a scheduling plan with the best assigned value is chosen for scheduling.

Abstract: A process for optimizing loss functions includes progressively building better sets of parameters for loss functions represented as multivariate Taylor expansions in accordance with an iterative process. The optimization process is built upon CMA-ES. At each generation (i.e., each CMA-ES iteration), a new set of candidate parameter vectors is sampled. These candidate parameter vectors are sampled from a multivariate Gaussian distribution representation that is modeled by the CMA-ES covariance matrix and the current mean vector. The candidates are then each evaluated by training a model (neural network) using the candidates and determining a fitness value for each candidate against a validation data set.