Abstract: A method for determining a complexity of a natural language query includes converting a first natural language query into executable program code, the first natural language query being a query for a first video to be answered by one or more video question answering (VideoQA) models. The method also includes generating, via a complexity model, an abstract syntax tree (AST) based on the executable program code. The method further includes determining, via the complexity model, a complexity of the first natural language query based on quantity of subtrees, from a group of subtrees, that are present in the AST.

Abstract: A method for determining a complexity of a natural language query includes converting a first natural language query into executable program code, the first natural language query being a query for a first video to be answered by one or more video question answering (VideoQA) models. The method also includes generating, via a complexity model, an abstract syntax tree (AST) based on the executable program code. The method further includes determining, via the complexity model, a complexity of the first natural language query based on quantity of subtrees, from a group of subtrees, that are present in the AST.

Abstract: A method may include receiving parameters associated with a pre-trained transformer trained on first training data, modifying an architecture of the pre-trained transformer to generate a modified transformer, the modified transformer replacing a dot-product softmax attention layer with a linear kernel squared dot product attention layer utilizing Group Normalization, receiving second training data, and training the modified transformer based on the training data.

Abstract: A method may include receiving parameters associated with a pre-trained transformer trained on first training data, modifying an architecture of the pre-trained transformer to generate a modified transformer, the modified transformer replacing a dot-product softmax attention layer with a linear kernel dot product attention layer utilizing Group Normalization, receiving second training data, and training the modified transformer based on the training data.

Abstract: System, methods, and other embodiments described herein relate to estimating a depth map from an image using a diffusion model that efficiently learns in optimal computational spaces using a learning model and trains with sparse data. In one embodiment, a method includes estimating a local vector for an image by combining random noise, an image embedding about the image, and a geometric embedding using a learning model, the local vector having depth information at a pixel-level for the image. The method also includes predicting a global vector by combining the image embedding and the geometric embedding at a scene-level by the learning model. The method also includes inferring a depth map of the image by combining the local vector and the global vector using a diffusion model.

Abstract: Systems and methods are provided for generating amodal images from occlusions objects in input images. Examples herein include receiving a prompt selecting an object in an input image; applying the input image to a trained conditional generative model that generates an amodal image of the selected object based on the prompt and the input image; and outputting the amodal image.

Abstract: A method for discovering human-interpretable concepts from video-based transformer models is described. The method includes passing a set of videos through a video-based transformer model to select an intermediate video feature of each of the set of videos. The method also includes clustering the intermediate video feature of each of the set of videos to obtain corresponding tubelets to the selected intermediate video features of each of the set of videos. The method further includes clustering an entire dataset of tubelets to form concepts of the set of videos. The method also includes calculating an importance of each of the concepts of the set of videos to an output of the video-based transformer model.