Abstract: A method for controlling an artificial intelligence (AI) device can include obtaining, via a processor in the AI device, an AI model trained on a dataset that includes a majority class and at least one minority class, generating, via the processor, at least one evaluation metric for the AI model based multiplying a first score for positive samples of a target class within the dataset by a number of negative samples of the target class within the dataset and multiplying a second score for the negative samples within the dataset by a number of the positive samples, and outputting, via an output unit in the AI device, the at least one evaluation metric. Also, the method can further include adding the trained AI model to a pool of trained AI models and selecting a best AI model from the pool for deployment based on the at least one evaluation metric.

Abstract: A neural network system, comprising: instructions for implementing at least a SWBN layer in a neural network, and wherein the instructions perform operations comprising: during training of the neural network system on a plurality of batches of training data and for each of the plurality of batches: obtaining a respective first layer output for each of the plurality of training data; determining a plurality of normalization statistics for the batch from the first layer outputs; generating a respective normalized output for each training data in the batch; updating the whitening matrix by a covariance matrix; performing stochastic whitening on the normalized components of each first layer output; transforming the whitened data for each training data; generating a respective SWBN layer output for each of the training data from the transformed whitened data for each training data in the batch; and providing the SWBN layer output.

Abstract: A method for controlling an artificial intelligence (AI) device can include obtaining a video sample of a user and an audio sample of the user, generating, via a neural network, a visual embedding based on the video sample and an audio embedding based on the audio sample, the visual embedding and the audio embedding being multi-dimensional vectors, generating, via the neural network, an audio-visual embedding based on a combination of the visual and audio embeddings. The method can further include determining a specific pre-enrolled audio-visual embedding from among pre-enrolled audio-visual embeddings corresponding pre-enrolled users based on a distance away from the audio-visual embedding within a joint audio-visual subspace and verifying the user as the specific pre-enrolled user. Also, the neural network can be trained based on a loss function that uses a plurality of audio-visual embeddings, each including an audio component and a visual component.

Abstract: A method for controlling an artificial intelligence (AI) device can include obtaining an input query, an input image, bounding boxes for objects detected in the input image, object labels corresponding to the bounding boxes, and at least one topic label for a word in the input query, generating at least one word embedding for the at least one topic label, and generating a plurality of word embeddings for the object labels corresponding to the bounding boxes. The method can further include generating output attention maps corresponding to scaled dot product attention matrices based on the at least one word embedding for the at least one topic label from the input query and each of the plurality of word embeddings for the object labels, and combining the output attention maps to generate a final attention map corresponding to the at least one topic label from the input query.

Abstract: A method for controlling an artificial intelligence (AI) device can include receiving, via a processor, a base dataset, and receiving, via the processor, an image that is unlabeled. Also, the method can include inputting the image to at least one of a caption-based pipeline and a data programming pipeline to generate a labeled image training data sample including triplet information, the caption-based pipeline including a matching model configured to receive textual scene graph information for the image and bounding box information for the image, and the data programming pipeline including a feature extraction model configured to output three vectors and a label generator configured to receive the three vectors. The method can further include in response to the predicate matching a same predicate in a tail distribution of the base dataset, merging the labeled image training data sample with the base dataset to generate an enhanced dataset.

Abstract: A method for controlling an artificial intelligence (AI) device can include obtaining, via a processor in the AI device, a plurality of universal modules, receiving, via the processor in the AI device, an input image and a query related to the input image, and selecting, via the processor, a group of universal modules from among the plurality of universal modules. Also, the method can further include determining, via the processor, a layout arrangement for the group of universal modules and connecting the group of universal modules together according to the layout arrangement to form a neural module network (NMN), and outputting, via the processor, an answer based on the NMN, the query and the input image.

Abstract: A method for controlling an artificial intelligence (AI) device can include receiving, via a processor in the AI device, an input image and a query related to the input image, generating, via the processor, an answer prompt template based on the query, the answer prompt template including a sentence containing a mask token located at a position corresponding to an answer within the sentence, and combining the query and the answer prompt template to generate a string of text including the mask token. Also, the method can further include inputting the string of text to a pre-trained mask language module (MLM) and generating a plurality of scores respectfully corresponding to a plurality of answers, each of the plurality of answers being a candidate for replacing the mask token, determining a selected answer among the plurality of answers based on the plurality of scores, and outputting the selected answer.

Abstract: A method for controlling an artificial intelligence (AI) device can include obtaining, via a processor in the AI device, a knowledge graph, training, via the processor, a link prediction model on the knowledge graph to generate a trained link prediction model, extracting, via the processor, logic rules from the trained link prediction model, generating, via the processor, at least one evaluation metric based on the logic rules, and generating, via the processor, evaluation results based on comparing the at least one evaluation metric to a predetermined criteria, and outputting, via an output unit in the AI device, the evaluation results. Also, the method can include saving the trained link prediction model in a memory of the AI device for deployment or transmitting the trained link prediction model to an external device for deployment, based on the evaluation results.

Abstract: A method for controlling a device to manage a visual scene graph model can include obtaining, via a processor in the device, a first dataset; obtaining, via the processor, a second data set different from the first dataset, the second dataset including one or more of a causal relation or an intention relation; combining, via the processor, the first dataset and the second dataset to generate a combined dataset. Also, the method can further include applying a knowledge embedding function to the combined dataset to generate learned common sense knowledge embeddings; and training a visual scene graph model based on the learned common sense knowledge embeddings to generate a trained visual scene graph model. Further, the method can include executing a function based on an output of the trained visual scene graph model. The device can include at least one of a smart television, a mobile phone or a robot.

Abstract: A method for controlling a device to manage a visual saliency model can include receiving, via a processor in the device, a center bias map and a saliency density ground-truth map for an image; normalizing values of the saliency density ground-truth map to generate a normalized density ground-truth map; and comparing values of the normalized density ground-truth map to a predefined threshold value to generate an enhanced ground-truth map. The method can further include subtracting the enhanced ground-truth map from the center bias map to generate a negative candidates map; normalizing values of the negative candidates map to generate a normalized candidates map; and performing a sampling process on the normalized candidates map to generate a negative point map. Also, the method includes applying a filter function to the negative point map to generate a negative density map.

Abstract: A neural network system, comprising: instructions for implementing at least a SWBN layer in a neural network, and wherein the instructions perform operations comprising: during training of the neural network system on a plurality of batches of training data and for each of the plurality of batches: obtaining a respective first layer output for each of the plurality of training data; determining a plurality of normalization statistics for the batch from the first layer outputs; generating a respective normalized output for each training data in the batch; updating the whitening matrix by a covariance matrix; performing stochastic whitening on the normalized components of each first layer output; transforming the whitened data for each training data; generating a respective SWBN layer output for each of the training data from the transformed whitened data for each training data in the batch; and providing the SWBN layer output.