Abstract: A system and method for optimized generation, evaluation and selection of solutions associated with problems of different domain types using distributed evolutionary computing is provided. A request corresponding to problem associated with domain type is sent. A seed population corresponding to the request is generated. The generated seed population corresponding to the request is evaluated based on privately hosted datasets. The evaluated seed population is associated with one or more metrics to generate metric dataset. A best candidate solution associated with the metrics dataset is selected by recursively processing the metrics dataset associated with the evaluated seed population until a termination condition is reached. In the event the termination condition is not reached a next population is generated based on the best candidate solution and evaluated based on the privately hosted datasets. The best candidate solution is selected based on the next population until the termination condition is reached.

Abstract: A residual estimation with an I/O kernel (“RIO”) framework provides estimates of predictive uncertainty of neural networks, and reduces their point-prediction errors. The process captures neural network (“NN”) behavior by estimating their residuals with an I/O kernel using a modified Gaussian process (“GP”). RIO is applicable to real-world problems, and, by using a sparse GP approximation, scales well to large datasets. RIO can be applied directly to any pretrained NNs without modifications to model architecture or training pipeline.

Abstract: Roughly described, the technology disclosed provides a so-called machine-learned conversion optimization (MLCO) system that uses artificial neural networks and evolutionary computations to efficiently identify most successful webpage designs in a search space without testing all possible webpage designs in the search space. The search space is defined based on webpage designs provided by marketers. Neural networks are represented as genomes. Neural networks map user attributes from live user traffic to different dimensions and dimension values of output funnels that are presented to the users in real time. The genomes are subjected to evolutionary operations like initialization, testing, competition, and procreation to identify parent genomes that perform well and offspring genomes that are likely to perform well.

Abstract: The present invention provides a system and a method for augmenting population of candidate solutions with respect to segments of one or more best solution(s) of the population to improve evolutionary computing. In operation, a segment is randomly selected from one of the one or more best solutions of a population. Further, a first population of solutions is generated with respect to randomly selected segment. Furthermore, a second population of candidate solutions is generated with respect to complement of the randomly selected segment. Yet further, the steps of randomly selecting a segment, and generating a first and a second population of solutions is repeated for other of the one or more best solutions of the population. Yet further, the first and the second population of candidate solutions generated with respect to respective one or more best solutions of the population are merged with the population to generate an augmented population.

Abstract: A surrogate-assisted evolutionary optimization method, ESP, discovers decision strategies in real-world applications. Based on historical data, a surrogate is learned and used to evaluate candidate policies with minimal exploration cost. Extended into sequential decision making, ESP is highly sample efficient, has low variance, and low regret, making the policies reliable and safe. As an unexpected result, the surrogate also regularizes decision making, making it sometimes possible to discover good policies even when direct evolution fails.

Abstract: A residual estimation with an I/O kernel (“RIO”) framework provides estimates of predictive uncertainty of neural networks, and reduces their point-prediction errors. The process captures neural network (“NN”) behavior by estimating their residuals with an I/O kernel using a modified Gaussian process (“GP”). RIO is applicable to real-world problems, and, by using a sparse GP approximation, scales well to large datasets. RIO can be applied directly to any pretrained NNs without modifications to model architecture or training pipeline.

Abstract: A process for training and sharing generic functional modules across multiple diverse (architecture, task) pairs for solving multiple diverse problems is described. The process is based on decomposing the general multi-task learning problem into several fine-grained and equally-sized subproblems, or pseudo-tasks. Training a set of (architecture, task) pairs then corresponds to solving a set of related pseudo-tasks, whose relationships can be exploited by shared functional modules. An efficient search algorithm is introduced for optimizing the mapping between pseudo-tasks and the modules that solve them, while simultaneously training the modules themselves.

Abstract: A genetic loss function optimization (hereafter “GLO”) process uses a genetic algorithm to construct candidate loss functions as trees. The process takes the best candidate loss functions from this set and optimizes the coefficients thereof using covariance-matrix adaptation evolutionary strategy (hereafter “CMA-ES”), resulting in new loss functions showing substantial improvements in accuracy, convergence speed, and data requirements.

Abstract: The present invention provides a system and a method for evaluating weight initialization techniques for individual layers of neural network model by preserving mean and variance of output signals propagated through respective layers of model. In operation, the present invention provides for deriving a mean-variance mapping function (g-layer) for each layer of received neural network model. Further, the present invention provides for determining if weight parameter is associated with respective layers of model. Furthermore, weight initialization technique is evaluated for setting initial value of weight parameter of layers determined to have weight parameter by using derived mean-variance mapping functions, such that mean of output signal of respective layers is zero and variance is one.

Abstract: A process for discovering optimal Generative Adversarial Networks (GAN) includes jointly optimizing the three functions of a GANs process including (i) a real component of a discriminator network's loss that is a function of D(x), wherein D(x) is the discriminator network's output for a real sample from an input dataset; (ii) a synthetic component of the discriminator network's loss that is a function of D(G(z)), wherein D(G(z)) is the discriminator network's output for a generator network's synthetic samples z from a latent distribution; and (iii) a generator network's loss which is a function of D(G(z)), with the discriminator network's total loss being the sum of components (i) and (ii).

Abstract: The present invention relates to metalearning of deep neural network (DNN) architectures and hyperparameters. Precisely, the present system and method utilizes Evolutionary Population-Based Based Training (EPBT) that interleaves the training of a DNN's weights with the metalearning of loss functions. They are parameterized using multivariate Taylor expansions that EPBT can directly optimize. Further, EPBT based system and method uses a quality-diversity heuristic called Novelty Pulsation as well as knowledge distillation to prevent overfitting during training. The discovered hyperparameters adapt to the training process and serve to regularize the learning task by discouraging overfitting to the labels. EPBT thus demonstrates a practical instantiation of regularization metalearning based on simultaneous training.

Abstract: Roughly described, an evolutionary data mining system includes at least two processing units, each having a pool of candidate individuals in which each candidate individual has a fitness estimate and experience level. A first processing unit tests candidate individuals against training data, updates an individual's experience level, and assigns each candidate to one of multiple layers of the candidate pool based on the individual's experience level. Individuals within the same layer of the same pool compete with each other to remain candidates. The first processing unit selects a set of candidates to retain based on the relative novelty of their responses to the training data. The first processing unit reports successful individuals to the second processing unit, and receives individuals for further testing from the second processing unit. The second processing unit selects individuals to retain based on their fitness estimate.

Abstract: A computer-implemented method optimizing genetic algorithms for finding solutions to a provided problem is described. The method implements a multi-arm bandit algorithm to determine performance scores for candidate individuals from a candidate pool in dependence on successes and failures of the one or more candidates. The method evolves the candidate individuals in the candidate pool by performing evolution steps including: determining a fitness score for each of the candidate individuals in the candidate pool in dependence on the performance scores for the candidate individuals, discarding candidate individuals from the candidate pool in dependence upon their assigned performance measure, and adding, to the candidate pool, a new candidate individual procreated from candidate individuals remaining in the candidate pool after the discarding of the candidate individuals.

Abstract: A system and method of combining and improving sets of diverse prescriptors for Evolutionary Surrogate-assisted Prescription (ESP) model is described. The prescriptors are distilled into neural networks and evolved further using ESP. The system and method can handle diverse sets of prescriptors in that it makes no assumptions about the form of the input (i.e., contexts) of the initial prescriptors; it relies only on the prescriptions made in order to distill each prescriptor to a neural network with a fixed form. The resulting set of high performing prescriptors provides a practical way for ESP to incorporate external human and machine knowledge and generate more accurate and fitting set of solutions.

Abstract: A composite novelty method approach to deceptive problems where a secondary objective is available to diversify the search is described. In such cases, composite objectives focus the search on the most useful tradeoffs and allow escaping deceptive areas. Novelty-based selection increases exploration in the focus area, leading to better solutions, faster and more consistently and it can be combined with other fitness-based methods.

Abstract: The technology disclosed proposes a novel asynchronous evaluation strategy (AES) that increases throughput of evolutionary algorithms by continuously maintaining a queue of K individuals ready to be sent to the worker nodes for evaluation and evolving the next generation once a fraction Mi of the K individuals have been evaluated by the worker nodes, where Mi<<K. A suitable value for Mi is determined experimentally, balancing diversity and efficiency. The technology disclosed is extended to coevolution of deep neural network supermodules and blueprints in the form of AES for cooperative evolution of deep neural networks (CoDeepNEAT-AES). Applied to image captioning domain, a threefold speedup is observed on 200 graphics processing unit (GPU) worker nodes, demonstrating that the disclosed AES and CoDeepNEAT-AES are promising techniques for evolving complex systems with long and variable evaluation times.

Abstract: The technology disclosed relates to a webinterface production and deployment system. In particular, it relates to a presentation module that applies a selected candidate individual to a presentation database to determine frontend element values corresponding to dimension values identified by the selected candidate individual, and which presents toward a user a funnel having the determined frontend element values.

Abstract: The computer system and method herein uses a multi-objective driven evolutionary algorithm that is better able to find optimum solutions to a problem because it balances the use of objectives as composite functions, and relative novelty and diversity in evolutionary optimization. In particular, the system and method herein described herein presents an improved process which introduces novelty pulsation, i.e., a systematic method to alternate between novelty selection and local optimization.

Abstract: A method for finding a solution to a problem is provided. The method includes storing candidate individuals in a candidate pool and evolving the candidate individuals by performing steps including (i) testing each of the candidate individuals to obtain test results, (ii) assigning a performance measure to the tested candidate individuals, (iii) discarding candidate individuals from the candidate pool in dependence upon their assigned performance measure, and (iv) adding, to the candidate pool, a new candidate individual procreated from candidate individuals remaining in the candidate pool.

Abstract: Roughly described, the technology disclosed provides a so-called machine learned conversion optimization (MLCO) system that uses evolutionary computations to efficiently identify most successful webpage designs in a search space without testing all possible webpage designs in the search space. The search space is defined based on webpage designs provided by marketers. Website funnels with a single webpage or multiple webpages are represented as genomes. Genomes identify different dimensions and dimension values of the funnels. The genomes are subjected to evolutionary operations like initialization, testing, competition, and procreation to identify parent genomes that perform well and offspring genomes that are likely to perform well. Each webpage is tested only to the extent that it is possible to decide whether it is promising, i.e., whether it should serve as a parent for the next generation, or should be discarded.