Abstract: This disclosure relates generally to visio-linguistic understanding. Conventional methods use contextual visio-linguistic reasoner for visio-linguistic understanding which requires more compute power and large amount of pre-training data. Embodiments of the present disclosure provide a method for visio-linguistic understanding using contextual language model reasoner. The method converts the visual information of an input image into a format that the contextual language model reasoner understands and accepts for a downstream task. The method utilizes the image captions and confidence score associated with the image captions along with a knowledge graph to obtain a combined input in a format compatible with the contextual language model reasoner. Contextual embeddings corresponding to the downstream task is obtained using the combined input. The disclosed method is used to solve several downstream tasks such as scene understanding, visual question answering, visual common-sense reasoning and so on.

Abstract: This disclosure relates generally to visio-linguistic understanding. Conventional methods use contextual visio-linguistic reasoner for visio-linguistic understanding which requires more compute power and large amount of pre-training data. Embodiments of the present disclosure provide a method for visio-linguistic understanding using contextual language model reasoner. The method converts the visual information of an input image into a format that the contextual language model reasoner understands and accepts for a downstream task. The method utilizes the image captions and confidence score associated with the image captions along with a knowledge graph to obtain a combined input in a format compatible with the contextual language model reasoner. Contextual embeddings corresponding to the downstream task is obtained using the combined input. The disclosed method is used to solve several downstream tasks such as scene understanding, visual question answering, visual common-sense reasoning and so on.

Abstract: The online shopping is highly based on human perception on products and the human perception on products depends on semantic features of products. Conventional methods provides product recommendation based on historical data and are supervised. The present disclosure receives a set of multi-modal data. A plurality of features are extracted from the set of data at a plurality of resolution levels and the plurality of features are arranged as parallel corpus based on a category associated with each data from the set of data. Further, an abstract interaction vector is computed for each element of the set of data using the parallel corpus. Further, the set of recommendations are identified by comparing the abstract interaction vector associated with the set of data with an abstract interaction vector associated with each of a plurality of items stored in the database by utilizing a similarity metric.