Abstract: Methods, devices, and computer-readable media for multi-task based lifelong learning. A method for lifelong learning includes identifying a new task for a machine learning model to perform. The machine learning model trained to perform an existing task. The method includes adaptively training a network architecture of the machine learning model to generate an adapted machine learning model based on incorporating inherent correlations between the new task and the existing task. The method further includes using the adapted machine learning model to perform both the existing task and the new task.

Abstract: In general, the disclosure describes techniques for facilitating trust in neural networks using a trusted neural network system. For example, described herein are multi-headed, trusted neural network systems that can be trained to satisfy one or more constraints as part of the training process, where such constraints may take the form of one or more logical rules and cause the objective function of at least one the heads of the trusted neural network system to steer, during machine learning model training, the overall objective function for the system toward an optimal solution that satisfies the constraints. The constraints may be non-temporal, temporal, or a combination of non-temporal and temporal. The constraints may be directly compiled to a neural network or otherwise used to train the machine learning model.

Abstract: A method includes accessing video data that includes at least two different modalities. The method also includes using a convolutional neural network layer to incorporate temporal coherence into a machine learning model architecture configured to process the video data. The method further includes learning dependency among the at least two different modalities in an attention space of the machine learning model architecture. In addition, the method includes predicting one or more correlations among the at least two different modalities.

Abstract: A method implemented by one or more computing systems includes accessing a set of content objects, in which each content object of the set of content objects is pre-labeled with concepts of a plurality of concepts organized according to a hierarchical relationship. The method further includes training, by a machine-learning model, a classification model for classifying content objects within the set of content objects. Training the classification model includes determining, for each object, a plurality of classification values corresponding to the plurality of concepts, calculating a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object, utilizing a hierarchical constraint loss function to calculate a maximum loss based on the calculated loss for each of the plurality of classification values, and updating the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

Abstract: Methods and systems of using machine learning to create a trusted model that improves the operation of a computer system controller are provided herein. In some embodiments, a machine learning method includes training a model using input data, extracting the model, and determining whether the model satisfies the trust-related constraints. If the model does not satisfy the trust-related constraint, modifying at least one of: the model using one or more model repair algorithms, the input data using one or more data repair algorithms, or a reward function of the model using one or more reward repair algorithms, and re-training the model using at least one of the modified model, the modified input data, or the modified reward function. If the model satisfies the trust-related constraints, providing the model as a trusted model that enables a computer system controller to perform system actions within predetermined guarantees.

Abstract: Methods, devices, and computer-readable media for multi-task based lifelong learning. A method for lifelong learning includes identifying a new task for a machine learning model to perform. The machine learning model trained to perform an existing task. The method includes adaptively training a network architecture of the machine learning model to generate an adapted machine learning model based on incorporating inherent correlations between the new task and the existing task. The method further includes using the adapted machine learning model to perform both the existing task and the new task.

Abstract: Methods, devices, and computer-readable media for incremental learning in image classification and/or object detection. A method for incremental learning includes identifying, for a model for object detection or classification, a first set of object classes the model is trained to detect or classify and adapting the model for use with a second set of object classes different from the first set of object classes to generate an adapted model. The method further includes retaining detection or classification performance on the first set of object classes in the adapted model by performing a knowledge distillation process for the model; and using the adapted model to detect or classify one or more objects from the first set of object classes and one or more objects from the second set of object classes.

Abstract: In general, the disclosure describes techniques for facilitating trust in neural networks using a trusted neural network system. For example, described herein are multi-headed, trusted neural network systems that can be trained to satisfy one or more constraints as part of the training process, where such constraints may take the form of one or more logical rules and cause the objective function of at least one the heads of the trusted neural network system to steer, during machine learning model training, the overall objective function for the system toward an optimal solution that satisfies the constraints. The constraints may be non-temporal, temporal, or a combination of non-temporal and temporal. The constraints may be directly compiled to a neural network or otherwise used to train the machine learning model.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing word sequences using neural networks. One of the methods includes receiving a first sequence of words arranged according to a first order; and for each word in the first sequence, beginning with a first word in the first order: determining a topic vector that is associated with the word; generating a combined input from the word and the topic vector, and processing the combined input through one or more sequence modeling layers to generate a sequence modeling output for the word; and processing one or more of the sequence modeling outputs through an output layer to generate a neural network output for the first sequence of words.

Abstract: Methods and systems of using machine learning to create a trusted model that improves the operation of a computer system controller are provided herein. In some embodiments, a machine learning method includes training a model using input data, extracting the model, and determining whether the model satisfies the trust-related constraints. If the model does not satisfy the trust-related constraint, modifying at least one of: the model using one or more model repair algorithms, the input data using one or more data repair algorithms, or a reward function of the model using one or more reward repair algorithms, and re-training the model using at least one of the modified model, the modified input data, or the modified reward function. If the model satisfies the trust-related constraints, providing the model as a trusted model that enables a computer system controller to perform system actions within predetermined guarantees.