Abstract: An online system generates predicted outcomes for a content distribution program that distributes content to users of the online system, the predicted outcome indicating a likelihood for the occurrence of an outcome of a content presentation. The online system transmits the one or more predicted outcomes to the third-party system, and receives prediction improvement data from the third-party system, the prediction improvement data indicating an adjustment to errors in the predicted outcomes based on a prediction by the third-party system. The online system updates the properties of a content distribution program based on the prediction improvement data, the updated content distribution program causing the online system to generate new predicted outcomes based on the prediction improvement data in content presentation opportunities. The online system also transmits content to users of the online system based on the updated content distribution program.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network used to select actions performed by a reinforcement learning agent interacting with an environment. In one aspect, a method includes maintaining a replay memory, where the replay memory stores pieces of experience data generated as a result of the reinforcement learning agent interacting with the environment. Each piece of experience data is associated with a respective expected learning progress measure that is a measure of an expected amount of progress made in the training of the neural network if the neural network is trained on the piece of experience data. The method further includes selecting a piece of experience data from the replay memory by prioritizing for selection pieces of experience data having relatively higher expected learning progress measures and training the neural network on the selected piece of experience data.

Abstract: Some embodiments provide a method for training a machine-trained (MT) network. The method propagates multiple inputs through the MT network to generate an output for each of the inputs. each of the inputs is associated with an expected output, the MT network uses multiple network parameters to process the inputs, and each network parameter of a set of the network parameters is defined during training as a probability distribution across a discrete set of possible values for the network parameter. The method calculates a value of a loss function for the MT network that includes (i) a first term that measures network error based on the expected outputs compared to the generated outputs and (ii) a second term that penalizes divergence of the probability distribution for each network parameter in the set of network parameters from a predefined probability distribution for the network parameter.

Abstract: A method identifies and removes bias from a machine learning model. A user/computer inputs a plurality of input training data into a machine learning system to generate an output of labeled output data. The user/computer evaluates the labeled output data according to a consistency metric to associate the labeled output data with a corresponding consistency assessment. The user/computer selects each labeled output data having a consistency assessment indicating a consistency assessment that is greater than a predetermined threshold to form a labeled output data subset, and then creates additional labeling for the labeled output data subset. The user/computer utilizes the additional labeling to distinguish each labeled training data from labeled output data subset as being mislabeled and biased, and then adjusts the learning machine based on the labeled output data subset being mislabeled and biased.

Abstract: A policy violation detection computer-implemented method, system, and computer program product, includes extracting a policy activity from a policy, the policy activity including an actor in the policy, an object of the policy, an action of the policy, and policy scope metadata, capturing a transaction by a user including metadata of the transaction, translating the transaction by the user into an actor in the transaction, an action of the transaction, and an object of the transaction, and alerting the user of a policy violation by navigating a knowledge graph is-a hierarchy to relate the actor in the transaction to the actor in the policy, the object of the transaction to an object of the policy, and the action of the transaction to an action of the policy activity.

Abstract: Handshake protocol layer features are extracted from training data associated with encrypted network traffic of a plurality of classified devices. Record protocol layer features are extracted from the training data. One or more models are trained based on the extracted handshake protocol layer features and the extracted record protocol layer features. The one or more models are applied to an observed encrypted network traffic stream associated with a device to determine a predicted device classification of the device.

Abstract: The present invention discloses a neural network training method for a memristor memory for memristor errors, which is mainly used for solving the problem of decrease in inference accuracy of a neural network based on the memristor memory due to a process error and a dynamic error. The method comprises the following steps: performing modeling on a conductance value of a memristor under the influence of the process error and the dynamic error, and performing conversion to obtain a distribution of corresponding neural network weights; constructing a prior distribution of the weights by using the weight distribution obtained after modeling, and performing Bayesian neural network training based on variational inference to obtain a variational posterior distribution of the weights; and converting a mean value of the variational posterior of the weights into a target conductance value of the memristor memory.

Abstract: In automated assistant systems, a deep-learning model in form of a long short-term memory (LSTM) classifier is used for mapping questions to classes, with each class having a manually curated answer. A team of experts manually create the training data used to train this classifier. Relying on human curation often results in such linguistic training biases creeping into training data, since every individual has a specific style of writing natural language and uses some words in specific context only. Deep models end up learning these biases, instead of the core concept words of the target classes. In order to correct these biases, meaningful sentences are automatically generated using a generative model, and then used for training a classification model. For example, a variational autoencoder (VAE) is used as the generative model for generating novel sentences and a language model (LM) is utilized for selecting sentences based on likelihood.

Abstract: The present disclosure relates to the electronic document review field and, more particularly, to various apparatuses and methods of implementing batch-mode active learning for technology-assisted review (TAR) of documents (e.g., legal documents).

Abstract: Graphical interactive model selection is provided. A response variable vector for each value of a group variable and an explanatory variable vector are defined. A wavelet function is fit to the explanatory variable vector paired with the response variable vector defined for each value of the group variable. Each fit wavelet function defines coefficients for each value of the group variable. A curve is presented for each value of the group variable and is defined by the plurality of coefficients of an associated fit wavelet function. An indicator is received of a request to perform functional analysis using the coefficients for each value of the of the group variable based on a predefined factor variable. A model is trained using the coefficients for each value of the group variable and a factor variable value associated with each observation vector of each plurality of observation vectors as a model effect.

Abstract: A method, a system, and a system of systems, that couples one or more non-interpretable systems to one or more interpretable systems by using the output results of the non-interpretable systems as the training targets for the interpretable systems. The method, the system, and the system of systems provide for non-interpretable complex nonlinear interactions among inputs by augmenting the set of in-sample and out-sample inputs. The result of the coupling is one or more resulting interpretable systems that allow for the development of explanations, justifications, and rationalizations for systems heretofore non-explainable or non-interpretable. The method, system and system of systems solve transparency and bias problems for non-interpretable systems and provides a basis for ethical systems, such as ethical artificial intelligent (AI) systems.

Abstract: The present disclosure relates to the electronic document review field and, more particularly, to various apparatuses and methods of implementing batch-mode active learning for technology-assisted review (TAR) of documents (e.g., legal documents).

Abstract: Certain aspects involve optimizing neural networks or other models for assessing risks and generating explanatory data regarding predictor variables used in the model. In one example, a system identifies predictor variables. The system generates a neural network for determining a relationship between each predictor variable and a risk indicator. The system performs a factor analysis on the predictor variables to determine common factors. The system iteratively adjusts the neural network so that (i) a monotonic relationship exists between each common factor and the risk indicator and (ii) a respective variance inflation factor for each common factor is sufficiently low. Each variance inflation factor indicates multicollinearity among the common factors. The adjusted neural network can be used to generate explanatory indicating relationships between (i) changes in the risk indicator and (ii) changes in at least some common factors.

Abstract: Systems, methods, and non-transitory computer readable media are configured to receive a uniform resource locator. A time and one or more features associated with the uniform resource locator can be provided to a first machine learning model. A prediction relating to a quantity of views the uniform resource locator achieves by the time can be received from the first machine learning model.

Abstract: This disclosure describes methods, apparatuses, and systems for network training and testing for evaluating hardware characteristics and for hardware selection. For example, a sensor can capture a dataset, which may be transformed into a plurality of modified datasets to simulate changes to hardware. Each of the plurality of modified datasets may be used to individually train an untrained neural network, thereby producing a plurality of trained neural networks. In order to evaluate the trained neural networks, each neural network can be used to ingest an evaluation dataset to perform a variety of tasks, such as identifying various objects within the dataset. A performance of each neural network can be determined and compared. A performance curve can be determined for each characteristic under review, facilitating a selection of one or more hardware components and/or configurations.

Abstract: The disclosed technology relates identifying causes of an observed outcome. A system is configured to receive an indication of a user experience problem, wherein the user experience problem is associated with observed operations data including an observed outcome. The system generates, based on the observed operations data, a predicted outcome according to a model, determines that the observed outcome is within range of the predicted outcome, and identifies a set of candidate causes of the user experience problem when the observed outcome is within range of the predicted outcome.

Abstract: A response generation apparatus ensures accurate output. A computer stores graph knowledge including a response generation module generating a response to an input document including a plurality of sentences, the graph knowledge database includes graph data that manages a structure of each type of graph knowledge, and the response generation module generates a first graph knowledge from each of the sentences; searches a second graph knowledge similar to each of the plurality of first graph knowledge while referring to the graph data on the basis of the plurality of first graph knowledge; identifies the plurality of second graph knowledge included in a dense location where a density of the second graph knowledge is high in a graph space; searches third graph knowledge for generating the response while referring to the graph data on the basis of the identified second graph knowledge; and generates the response using the third graph knowledge.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for balanced-weight sparse convolution processing.

Abstract: An analytic system provides direct functional principal component analysis. (A) A next group variable value is selected from values of a group variable. (B) Explanatory variable values of observations having the selected next group variable value are sorted in ascending order. (C) The response variable value associated with each sorted explanatory variable value is stored in a next row of a data matrix. (D) (A) through (C) are repeated. (E) An eigenfunction index is incremented. (F) An FPCA is performed using the data matrix to define an eigenfunction for the eigenfunction index. (G) (E) and (F) are repeated. (H) FPCA results from the performed FPCA are presented within a window of a display. The FPCA results include an eigenvalue and an eigenfunction associated with the eigenvalue for each functional principal component identified from the performed FPCA in (F).

Abstract: Techniques that facilitate semantic and time series analysis using machine learning are provided. In one example, a system includes a data analysis component, a prediction component and a learning component. The data analysis component that establishes one or more relationships between one or more elements of semantic data, including one or more time series identifiers, and one or more elements of time series data in a relationship database. The prediction component generates one or more advisory outputs, wherein generation of the one or more advisory outputs is performed in response to a trigger event. A learning component that determines the one or more relationships in the relationship database, wherein determination of the one or more relationships is based on information indicative of whether the advisory outputs satisfy a defined criterion.