Abstract: A process for optimizing loss functions includes progressively building better sets of parameters for loss functions represented as multivariate Taylor expansions in accordance with an iterative process. The optimization process is built upon CMA-ES. At each generation (i.e., each CMA-ES iteration), a new set of candidate parameter vectors is sampled. These candidate parameter vectors are sampled from a multivariate Gaussian distribution representation that is modeled by the CMA-ES covariance matrix and the current mean vector. The candidates are then each evaluated by training a model (neural network) using the candidates and determining a fitness value for each candidate against a validation data set.

Abstract: A system and method for optimized evolutionary neural architecture search is provided. The present invention provides for generating at least two neural network architectures as two parents based on one or more received inputs. The neural network architectures are in the form of organized structures represented as computation graphs. A similarity is computed between the generated two neural network architectures by computing Graph Edit Distance (GED) between neural network architectures corresponding to computation graphs. Graph edit operations are executed for computing the GED. A Shortest Edit Path (SEP) crossover operation is carried out by analyzing computed edit paths between computation graphs. A shuffling operation is carried out randomly for shuffling edit paths in SEP between two parents by selecting half of shuffled edit paths randomly and applying selected edit paths to one of parents to generate an offspring graph for optimizing Neural Architecture Search (NAS) to solve real-world problems.

Abstract: An evolutionary AutoML framework called LEAF optimizes hyperparameters, network architectures and the size of the network. LEAF makes use of both evolutionary algorithms (EAs) and distributed computing frameworks. A multiobjective evolutionary algorithm is used to maximize the performance and minimize the complexity of the evolved networks simultaneously by calculating the Pareto front given a group of individuals that have been evaluated for multiple objectives.

Abstract: The technology disclosed relates to a webinterface production and deployment system. In particular, it relates to a presentation module that applies a selected candidate individual to a presentation database to determine frontend element values corresponding to dimension values identified by the selected candidate individual, and which presents toward a user a funnel having the determined frontend element values.

Abstract: A system and method for adding explainability to deep learning models based on rule-set evolution is provided. A set of inputs from an input unit is received which comprises pre-generated deep learning models. Set of inputs is evaluated using pre-defined querying datasets. An output comprising outcomes of the evaluation is generated and mapped with each of the pre-defined querying datasets used for querying deep learning model. A new dataset is generated based on the mapping. Population of initial rule-set models is randomly generated based on a set of hyper parameters. The hyper parameters relate to configuration parameters used for generating population of initial rule-set models. An evolution process is carried out on generated rule-set models for evolving the rule-set models by using the generated new datasets. Lastly, the evolved rule-set model is executed to solve one or more real-world problems.

Abstract: User-driven exploration functionality, referred to herein as a Scratchpad, is a post-learning extension for machine learning systems. For example, in ESP, consisting of the Predictor (a surrogate model of the domain) and Prescriptor (a solution generator model), the Scratchpad allows the user to modify the suggestions of the Prescriptor, and evaluate each such modification interactively with the Predictor. Thus, the Scratchpad makes it possible for the human expert and the AI to work together in designing better solutions. This interactive exploration also allows the user to conclude that the solutions derived in this process are the best found, making the process trustworthy and transparent to the user.

Abstract: A domain-independent problem-solving system and process addresses domain-specific problems with varying dimensionality and complexity, solving different problems with little or no hyperparameter tuning, and adapting to changes in the domain, thus implementing lifelong learning.

Abstract: The technology disclosed relates to webinterface generation and testing to promote a predetermined target user behavior. In particular, the technology disclosed stores a candidate database having a population of candidate individuals. Each of the candidate individuals identify respective values for a plurality of hyperparameters of the candidate individual. The hyperparameters describe topology of a respective neural network and coefficients for interconnects of the respective neural network. The technology disclosed writes a preliminary pool of candidate individuals into the candidate individual population. The technology disclosed tests each of the candidate individuals in the candidate individual population. The technology disclosed adds to the candidate individual population new individuals based on the testing. The technology disclosed repeats the candidate testing and the addition of the new individuals.

Abstract: A multi-task (MTL) process is adapted to the single-task learning (STL) case, i.e., when only a single task is available for training. The process is formalized as pseudo-task augmentation (PTA), in which a single task has multiple distinct decoders projecting the output of the shared structure to task predictions. By training the shared structure to solve the same problem in multiple ways, PTA simulates the effect of training towards distinct but closely-related tasks drawn from the same universe. Training dynamics with multiple pseudo-tasks strictly subsumes training with just one, and a class of algorithms is introduced for controlling pseudo-tasks in practice.

Abstract: A genetic loss function optimization (hereafter “GLO”) process uses a genetic algorithm to construct candidate loss functions as trees. The process takes the best candidate loss functions from this set and optimizes the coefficients thereof using covariance-matrix adaptation evolutionary strategy (hereafter “CMA-ES”), resulting in new loss functions showing substantial improvements in accuracy, convergence speed, and data requirements.

Abstract: A computer-implemented method optimizing genetic algorithms for finding solutions to a provided problem is described. The method implements a multi-arm bandit algorithm to determine performance scores for candidate individuals from a candidate pool in dependence on successes and failures of the one or more candidates. The method evolves the candidate individuals in the candidate pool by performing evolution steps including: determining a fitness score for each of the candidate individuals in the candidate pool in dependence on the performance scores for the candidate individuals, discarding candidate individuals from the candidate pool in dependence upon their assigned performance measure, and adding, to the candidate pool, a new candidate individual procreated from candidate individuals remaining in the candidate pool after the discarding of the candidate individuals.

Abstract: A system and method for optimized generation, evaluation and selection of solutions associated with problems of different domain types using distributed evolutionary computing is provided. A request corresponding to problem associated with domain type is sent. A seed population corresponding to the request is generated. The generated seed population corresponding to the request is evaluated based on privately hosted datasets. The evaluated seed population is associated with one or more metrics to generate metric dataset. A best candidate solution associated with the metrics dataset is selected by recursively processing the metrics dataset associated with the evaluated seed population until a termination condition is reached. In the event the termination condition is not reached a next population is generated based on the best candidate solution and evaluated based on the privately hosted datasets. The best candidate solution is selected based on the next population until the termination condition is reached.

Abstract: A residual estimation with an I/O kernel (“RIO”) framework provides estimates of predictive uncertainty of neural networks, and reduces their point-prediction errors. The process captures neural network (“NN”) behavior by estimating their residuals with an I/O kernel using a modified Gaussian process (“GP”). RIO is applicable to real-world problems, and, by using a sparse GP approximation, scales well to large datasets. RIO can be applied directly to any pretrained NNs without modifications to model architecture or training pipeline.

Abstract: Roughly described, the technology disclosed provides a so-called machine-learned conversion optimization (MLCO) system that uses artificial neural networks and evolutionary computations to efficiently identify most successful webpage designs in a search space without testing all possible webpage designs in the search space. The search space is defined based on webpage designs provided by marketers. Neural networks are represented as genomes. Neural networks map user attributes from live user traffic to different dimensions and dimension values of output funnels that are presented to the users in real time. The genomes are subjected to evolutionary operations like initialization, testing, competition, and procreation to identify parent genomes that perform well and offspring genomes that are likely to perform well.

Abstract: The present invention provides a system and a method for augmenting population of candidate solutions with respect to segments of one or more best solution(s) of the population to improve evolutionary computing. In operation, a segment is randomly selected from one of the one or more best solutions of a population. Further, a first population of solutions is generated with respect to randomly selected segment. Furthermore, a second population of candidate solutions is generated with respect to complement of the randomly selected segment. Yet further, the steps of randomly selecting a segment, and generating a first and a second population of solutions is repeated for other of the one or more best solutions of the population. Yet further, the first and the second population of candidate solutions generated with respect to respective one or more best solutions of the population are merged with the population to generate an augmented population.

Abstract: A surrogate-assisted evolutionary optimization method, ESP, discovers decision strategies in real-world applications. Based on historical data, a surrogate is learned and used to evaluate candidate policies with minimal exploration cost. Extended into sequential decision making, ESP is highly sample efficient, has low variance, and low regret, making the policies reliable and safe. As an unexpected result, the surrogate also regularizes decision making, making it sometimes possible to discover good policies even when direct evolution fails.

Abstract: A residual estimation with an I/O kernel (“RIO”) framework provides estimates of predictive uncertainty of neural networks, and reduces their point-prediction errors. The process captures neural network (“NN”) behavior by estimating their residuals with an I/O kernel using a modified Gaussian process (“GP”). RIO is applicable to real-world problems, and, by using a sparse GP approximation, scales well to large datasets. RIO can be applied directly to any pretrained NNs without modifications to model architecture or training pipeline.

Abstract: A process for training and sharing generic functional modules across multiple diverse (architecture, task) pairs for solving multiple diverse problems is described. The process is based on decomposing the general multi-task learning problem into several fine-grained and equally-sized subproblems, or pseudo-tasks. Training a set of (architecture, task) pairs then corresponds to solving a set of related pseudo-tasks, whose relationships can be exploited by shared functional modules. An efficient search algorithm is introduced for optimizing the mapping between pseudo-tasks and the modules that solve them, while simultaneously training the modules themselves.

Abstract: A genetic loss function optimization (hereafter “GLO”) process uses a genetic algorithm to construct candidate loss functions as trees. The process takes the best candidate loss functions from this set and optimizes the coefficients thereof using covariance-matrix adaptation evolutionary strategy (hereafter “CMA-ES”), resulting in new loss functions showing substantial improvements in accuracy, convergence speed, and data requirements.

Abstract: The present invention provides a system and a method for evaluating weight initialization techniques for individual layers of neural network model by preserving mean and variance of output signals propagated through respective layers of model. In operation, the present invention provides for deriving a mean-variance mapping function (g-layer) for each layer of received neural network model. Further, the present invention provides for determining if weight parameter is associated with respective layers of model. Furthermore, weight initialization technique is evaluated for setting initial value of weight parameter of layers determined to have weight parameter by using derived mean-variance mapping functions, such that mean of output signal of respective layers is zero and variance is one.