Abstract: According to one embodiment, a method, computer system, and computer program product for identifying one or more intrinsic physical properties of one or more objects is provided. The present invention may include identifying one or more objects in a video set, extracting observable physical properties of the identified one or more objects from the video set, including one or more trajectories, and inferring, by a property-based graph neural network, intrinsic properties of the one or more objects based on the trajectories.

Abstract: The exemplary embodiments disclose a method, a computer program product, and a computer system for determining that one or more model pipelines satisfy one or more constraints. The exemplary embodiments may include detecting a user uploading data, one or more constraints, and one or more model pipelines, collecting the data, the one or more constraints, and the one or more model pipelines, and determining that one or more of the model pipelines satisfies all of the one or more constraints based on applying one or more algorithms to the collected data, constraints, and model pipelines.

Abstract: Use a computerized trained graph neural network model to classify an input instance with a predicted label. With a computerized graph neural network interpretation module, compute a gradient-based saliency matrix based on the input instance and the predicted label, by taking a partial derivative of class prediction with respect to an adjacency matrix of the model. With a computerized user interface, obtain user input responsive to the gradient-based saliency matrix. Optionally, modify the trained graph neural network model based on the user input; and re-classify the input instance with a new predicted label based on the modified trained graph neural network model.

Abstract: Mechanisms are provided for performing artificial intelligence-based video question answering. A video parser parses an input video data sequence to generate situation data structure(s), each situation data structure comprising data elements corresponding to entities, and first relationships between entities, identified by the video parser as present in images of the input video data sequence. First machine learning computer model(s) operate on the situation data structure(s) to predict second relationship(s) between the situation data structure(s). Second machine learning computer model(s) execute on a received input question to predict an executable program to execute to answer the received question. The program is executed on the situation data structure(s) and predicted second relationship(s). An answer to the question is output based on results of executing the program.

Abstract: A vehicle light signal detection and recognition method, system, and computer program product include bounding, using a coarse attention module, one or more regions of an image of an automobile including at least one of a brake light and a signal light generated by automobile signals which include illuminated sections to generate one or more bounded region, removing, using a fine attention module, noise from the one or more bounded regions to generate one or more noise-free bounded regions, and identifying the at least one of the brake light and the signal light from the one or more noise-free bounded regions.

Abstract: A processor may receive a video including a plurality of video frames in sequence and a question regarding the video. For a video frame in the plurality of video frames, a processor may parse the video frame into objects and relationships between the objects, and create a subgraph of nodes representing objects and edges representing the relationships, where parsing and creating are performed for each video frame in the plurality of video frames, where a plurality of subgraphs can be created. A processor may create a hypergraph connecting subgraphs by learning relationships between the nodes of the subgraphs, where a hyper-edge is created to represent a relationship between at least one node of one subgraph and at least one node of another subgraph in the plurality of subgraphs. A processor may generate an answer to the question based on the hypergraph.

Abstract: Techniques regarding transferring learning outcomes across machine learning tasks in automated machine learning systems are provided. For example, one or more embodiments described herein can comprise a system, which can comprise a memory that can store computer executable components. The system can also comprise a processor, operably coupled to the memory, and that can execute the computer executable components stored in the memory. The computer executable components can comprise a transfer learning component that can executes a machine learning task using an existing artificial intelligence model on a sample dataset based on a similarity between the sample dataset and a historical dataset. The existing artificial intelligence model can be generated by automated machine learning and trained on the historical dataset.

Abstract: One or more computer processors improve action recognition by removing inference introduced by visual appearances of objects within a received video segment. The one or more computer processors extract appearance information and structure information from a received video segment. The one or more computer processors calculate a factual inference (TE) for the received video segment utilizing the extracted appearance information and structure information. The one or more computer processors calculate a counterfactual debiasing inference (NDE) for the received video segment. The one or more computer processors calculate a total indirect effect (TIE) by subtracting the calculated counterfactual debiased inference from the calculated factual inference. The one or more computer processors action recognize the received video segment by selecting a classification result associated with a highest calculated TIE.

Abstract: A method, computer system and computer program product is presented for providing a self-supervised speech representation. In one embodiment, audio input is received including speech utterances. A label sequence is generated from these speech utterances by a teacher label generator. A speech representation is generated of a partially masked version of the speech utterance using a speech representation network. The speech utterance is passed into two random transformations that alter only speaker information prior to the partial masking. A predictor will then predict the label sequence. In one embodiment performance-based assessment is made on a cross-entropy loss between the generated label sequence and a predicted label sequence.

Abstract: An artificial intelligence (AI) interaction method, system, and computer program product include selecting an artificial intelligence model to respond to a query to generating a response to the query using the selected artificial intelligence model, and receiving the response to the query from the selected artificial intelligence model.

Abstract: To automate time series forecasting machine learning pipeline generation, a data allocation size of time series data may be determined based on one or more characteristics of a time series data set. The time series data may be allocated for use by candidate machine learning pipelines based on the data allocation size. Features for the time series data may be determined and cached by the candidate machine learning pipelines. Predictions of each of the candidate machine learning pipelines using at least the one or more features may be evaluated. A ranked list of machine learning pipelines may be automatically generated from the candidate machine learning pipelines for time series forecasting based upon evaluating predictions of each of the one or more candidate machine learning pipelines.

Abstract: A computer-implemented method for fine-grained referring expression comprehension is provided. The computer-implemented method includes receiving, at a processor, a textual expression and an image as inputs and executing, at the processor, fine-grained referring expression comprehension. The executing includes decomposing the textual expression into different textual modules, extracting visual regional proposals from the image, using language-guided graph neural networks to mine fine-grained object relations from the visual regional proposals and aggregating different matching similarities between the different textual modules and the fine-grained object relations.

Abstract: Obtain, at a computing device, a segment of computer code. With a classification module of a machine learning system executing on the computing device, determine a required annotation category for the segment of computer code. With an annotation generation module of the machine learning system executing on the computing device, generate a natural language annotation of the segment of computer code based on the segment of computer code and the required annotation category. Provide the natural language annotation to a user interface for display adjacent the segment of computer code.

Abstract: An embodiment of the invention may include a method, computer program product, and system for creating a data analysis tool. The method may include a computing device that generates an AI pipeline based on an input dataset, wherein the AI pipeline is generated using an Automated Machine Learning program. The method may include converting the AI pipeline to a non-native format of the Automated Machine Learning program. This may enable the AI pipeline to be used outside of the Automated Machine Learning program, thereby increasing the usefulness of the created program by not tying it to the Automated Machine Learning program. Additionally, this may increase the efficiency of running the AI pipeline by eliminating unnecessary computations performed by the Automated Machine Learning program.

Abstract: A bi-directional spatial-temporal transformer neural network (BDSTT) is trained to predict original coordinates of a skeletal joint in a specific frame through relative relationships of the skeletal joint to other joints and to the state of the skeletal joint in other frames. Obtain a plurality of frames comprising coordinates of the skeletal joint and coordinates of other joints. Produce a spatially masked frame by masking the original coordinates of the skeletal joint. Provide the specific frame, the spatially masked frame, and at least one more frame to a coordinate prediction head of the BDSTT. Obtain, from the coordinate prediction head, a prediction of coordinates for the skeletal joint. Adjust parameters of the BDSTT until a mean-squared error, between the prediction of coordinates for the skeletal joint and the original coordinates of the skeletal joint, converges.

Abstract: A vehicle light signal detection and recognition method, system, and computer program product include bounding, using a coarse attention module, one or more regions of an image of an automobile including at least one of a brake light and a signal light generated by automobile signals which include illuminated sections to generate one or more bounded region, removing, using a fine attention module, noise from the one or more bounded regions to generate one or more noise-free bounded regions, and identifying the at least one of the brake light and the signal light from the one or more noise-free bounded regions.

Abstract: A method, a computer program product, and a computer system generate an accurate mental model of an automated agent. The method includes receiving an input from a user device associated with a user during a communication session between the user and the automated agent. The method includes determining a response to the input. The method includes determining a confidence score of the response relative to a confidence threshold. The method includes determining an assertiveness feature associated with the response, the assertiveness feature comprising an expression of the automated agent based on the confidence score. The method includes transmitting the response and the assertiveness feature to the user device, the expression configured to update anthropomorphic characteristics of a graphical representation of the automated agent shown on a graphical user interface of the communication session displayed on a display device of the user device.

Abstract: One or more computer processors improve action recognition by removing inference introduced by visual appearances of objects within a received video segment. The one or more computer processors extract appearance information and structure information from a received video segment. The one or more computer processors calculate a factual inference (TE) for the received video segment utilizing the extracted appearance information and structure information. The one or more computer processors calculate a counterfactual debiasing inference (NDE) for the received video segment. The one or more computer processors calculate a total indirect effect (TIE) by subtracting the calculated counterfactual debiased inference from the calculated factual inference. The one or more computer processors action recognize the received video segment by selecting a classification result associated with a highest calculated TIE.

Abstract: Unsupervised learning for video classification. One or more features from one or more video clips are extracted using a spatial-temporal encoder. The one or more extracted features are processed, using a video instance discrimination task, to generate a classification label, the classification label indicating whether two of the video clips are from a same video. The one or more extracted features are processed, using a pair-wise speed discrimination task, to generate a comparison label, the comparison label indicating a relative playback speed between two given video clips. A search is performed in a video database for a video that is similar to a given video based on the comparison label.

Abstract: According to one embodiment, a method, computer system, and computer program product for identifying one or more intrinsic physical properties of one or more objects is provided. The present invention may include identifying one or more objects in a video set, extracting observable physical properties of the identified one or more objects from the video set, including one or more trajectories, and inferring, by a property-based graph neural network, intrinsic properties of the one or more objects based on the trajectories.