Abstract: A semi-supervised model incorporates deep feature learning and pseudo label estimation into a unified framework. The deep feature learning can include multiple convolutional neural networks (CNNs). The CNNs can be trained on available training datasets, tuned using a small amount of labeled training samples, and stored as the original models. Features are then extracted for unlabeled training samples by utilizing the original models. Multi-view clustering is used to cluster features to generate pseudo labels. Then the original models are tuned by using an updated training set that includes labeled training samples and unlabeled training samples with pseudo labels. Iterations of multi-view clustering and tuning using an updated training set can continue until the updated training set is stable.

Abstract: A method may include executing a neural network to extract a first plurality of features from a plurality of first training images and a second plurality of features from a second training image; generating a model comprising a first image performance score for each of the plurality of first training images and a feature weight for each feature, the feature weight for each feature of the first plurality of features calculated based on an impact of a variation in the feature on first image performance scores of the plurality of first training images; training the model by adjusting the impact of a variation of each of a first set of features that correspond to the second plurality of features; executing the model using a third set of features from a candidate image to generate a candidate image performance score; and generating a record identifying the candidate image performance score.

Abstract: A multiple scenario-oriented item retrieval method and system. The method includes the steps of extracting, by Hashing learning, image features from an image training set to train a pre-built item retrieval model; when an image is in a scenario of hard samples, introducing an adaptive similarity matrix, optimizing the similarity matrix by an image transfer matrix, constructing an adaptive similarity matrix objective function in combination with an image category label; constructing a loss quantization objective function between the image and a Hash code according to the image transfer matrix; when the image is in a scenario of zero samples, introducing an asymmetric similarity matrix, constructing an objective function by taking the image category label as supervisory information in combination with equilibrium and decorrelation constraints of the Hash code; and training the item retrieval model based on the above objective function to obtain a retrieved result of a target item image.

Abstract: A method for lifelong machine learning using boosting includes receiving a new task and a learning sample for the new task. A distribution of weights is learned over the learning sample using previously learned classifiers from old tasks. A set of task-specific classifiers are learned for the new task using a boosting algorithm and the distribution of weights over the learning sample, whereby the distribution of weights over the learning sample is updated using the task-specific classifiers for the new task.

Abstract: Systems and methods are provided for multi-modal test-time adaptation. The method includes inputting a digital image into a pre-trained Camera Intra-modal Pseudo-label Generator, and inputting a point cloud set into a pre-trained Lidar Intra-modal Pseudo-label Generator. The method further includes applying a fast 2-dimension (2D) model, and a slow 2D model, to the inputted digital image to apply pseudo-labels, and applying a fast 3-dimension (3D) model, and a slow 3D model, to the inputted point cloud set to apply pseudo-labels. The method further includes fusing pseudo-label predictions from the fast models and the slow models through an Inter-modal Pseudo-label Refinement module to obtain robust pseudo labels, and measuring a prediction consistency for the pseudo-labels.

Abstract: Embodiments of the present disclosure relate to a method, device and computer readable storage medium of scene text detection. In the method, a first visual representation of a first image is generated with an image encoding process. A first textual representation of a first text unit in the first image is generated with a text encoding process based on a first plurality of symbols obtained by masking a first symbol of a plurality of symbols in the first text unit. A first prediction of the masked first symbol is determined with a decoding process based on the first visual and textual representations. At least the image encoding process is updating according to at least a first training objective to increase at least similarity of the first prediction and the masked first symbol.

Abstract: The second multi-dimensional feature vectors 92a of sample image data 34a having instruction signals that are converted by a feature converter 27 are read in (Step S10), two-dimensional graph data for model 36a is generated based on the read second multi-dimensional feature vectors 92a to be stored (Step S12), two-dimensional model graphs Og and Ng are generated based on the generated two-dimensional graph data for model 36a, to be displayed on the window 62 (Step S14). The second multi-dimensional feature vectors 92a are indicators appropriate for visualization of the trained state (individuality) of a trained model 35. Thus, it is possible to visually check and evaluate whether the trained model 35 is in an appropriately trained state (individuality) or not.

Abstract: Disclosed herein are systems and methods for using a robotic surgical system comprising a GUI and a robotic arm.

Abstract: The present disclosure describes systems, robots, and methods for organizing and selecting classifiers of a library of classifiers. The classifiers of the library of classifiers can be organized in a relational model, such as a hierarchy or probability model. Instead of storing, activating, or executing the entire library of classifiers at once by a robot system, computational resource demand is reduced by executing subset of classifiers to determine context, and the determined context is used as a basis for selection of another subset of classifiers. This process can be repeated, to iteratively refine context and select more specific subsets of classifiers. A selected subset of classifiers can eventually be specific to a task to be performed by the robot system, such that the robot system can take action based on output from executing such specific classifiers.

Abstract: Provided are computing systems, methods, and platforms for generating training and testing data for machine-learning models. The operations can include receiving signal extraction information that has instructions to query a data store. Additionally, the operations can include accessing, using Structured Query Language (SQL) code generated based on the signal extraction information, raw data from the data store. Moreover, the operations can include processing the raw data using signal configuration information to generate a plurality of signals. The signal configuration information can have instructions on how to generate the plurality of signals from the raw data. Furthermore, the operations can include joining, using SQL code, the plurality of signals with a first label source to generate training data and testing data. Subsequently, the operations can include processing the training data and the testing data to generate the input data. The input data being an ingestible for a machine-learning pipeline.

Abstract: The present disclosure describes approaches for evaluating interest points for localization uses based on a repeatability of the detection of the interest point in images capturing a scene that includes the interest point. The repeatability of interest points is determined by using a trained repeatability model. The repeatability model is trained by analyzing a time series of images of a scene and determining repeatability functions for each interest point in the scene. The repeatability function is determined by identifying which images in the time series of images allowed for the detection of the interest point by an interest point detection model.

Abstract: A computer-implemented method can include a training phase and a hemorrhage detection phase. The training phase can include: receiving a first plurality of frames from at least one original computed tomography (CT) scan of a target subject, wherein each frame may or may not include a visual indication of a hemorrhage, and further wherein each frame including a visual indication of a hemorrhage has at least one label associated therewith; and using a fully convolutional neural network (FCN) to train a model by determining, for each of the first plurality of frames, whether at least one sub-portion of the frame includes a visual indication of a hemorrhage and classifying the sub-portion of the frame based on the determining.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for object detection. In one aspect, a method comprises: obtaining: (i) an image, and (ii) a set of one or more query embeddings, wherein each query embedding represents a respective category of object; processing the image and the set of query embeddings using an object detection neural network to generate object detection data for the image, comprising: processing the image using an image encoding subnetwork of the object detection neural network to generate a set of object embeddings; processing each object embedding using a localization subnetwork to generate localization data defining a corresponding region of the image; and processing: (i) the set of object embeddings, and (ii) the set of query embeddings, using a classification subnetwork to generate, for each object embedding, a respective classification score distribution over the set of query embeddings.

Abstract: A recording medium determination apparatus includes an image data acquisition unit configured to acquire image data obtained by capturing an image of a predetermined area in a recording medium, a first extraction unit configured to extract a first feature amount by processing the image data using a first parameter, a second extraction unit configured to extract a second feature amount by processing the image data using a second parameter different from the first parameter, and a determination unit configured to determine a type of the recording medium based on the first feature amount and the second feature amount.

Abstract: At least one processor generates a crowd state image as an image in which a person image corresponding to a person state is synthesized with previously-prepared image at a predetermined size. The previously-prepared image is an background image that include no person. The at least one processor specifies a training label for the crowd state image. The at least one processor outputs a pair of crowd state image and training label.

Abstract: An information processing apparatus outputs operational information as information regarding a first operation and a second operation performed by an operator on an object as an operation target, the information processing apparatus including a scene estimator, a chunk estimator, a transition destination suggestion unit, and an output unit. The scene estimator obtains first images as images of a scene in a state where the operator performs the first operation and the second operation, and estimates the scene by using a first learned model describing an association between the first image and a scene ID that uniquely indicates the scene. The chunk estimator obtains second images as images of an object of the first operation and the second operation, and estimates a chunk by using one of a plurality of second learned models that store an association between the second image and one or a plurality of meta IDs for chunk.

Abstract: Systems, methods and non-transitory computer readable media for generating content using a generative model without relying on selected training examples are provided. An input indicative of a desire to generate a new content using a generative model may be received. The generative model may be a result of training a machine learning model using a plurality of training examples. Each training example of the plurality of training examples may be associated with a respective content. Further, an indication of a particular subgroup of at least one but not all of the plurality of training examples may be obtained. Based on the indication, the input and the generative model may be used to generate the new content, abstaining from basing the generation of the new content on any training example included in the particular subgroup. The new content may be provided.

Abstract: An image processing method includes steps of acquiring first model output generated based on a captured image by a first machine learning model, acquiring second model output generated based on the captured image by a second machine learning model which is different from the first machine learning model, and generating an estimated image by using the first model output and the second model output, based on a comparison based on the second model output and one of the captured image and first model output.

Abstract: Methods, systems, and apparatus for receiving a request for a damage propensity score for a parcel, receiving imaging data for the parcel, wherein the imaging data comprises street-view imaging data of the parcel, extracting, by a machine-learned model including multiple classifiers, characteristics of vulnerability features for the parcel from the imaging data, determining, by the machine-learned model and from the characteristics of the vulnerability features, a damage propensity score for the parcel, and providing a representation of the damage propensity score for display.

Abstract: A pipeline-based machine learning method includes: determining a plurality of target components from candidate components configured to construct a machine learning model; performing standardization processing on input data and output data of the plurality of target components to obtain a plurality of standardized components respectively corresponding to the plurality of target components; assembling, based on a connection relationship between the plurality of standardized components, the plurality of standardized components into a pipeline corresponding to the machine learning model; performing scheduling processing on the plurality of standardized components included in the pipeline, to obtain a scheduling result; and executing, based on THE scheduling result, a machine learning task corresponding to the machine learning model.

Abstract: A method and system may survey a property using aerial images captured from an unmanned aerial vehicle (UAV), a manned aerial vehicle (MAV) or from a satellite device. The method may include identifying a commercial property for a UAV to perform surveillance, and directing the UAV to hover over the commercial property and capture aerial images at predetermined time intervals. Furthermore, the method may include receiving the aerial images of the commercial property captured at the predetermined time intervals, detecting a surveillance event at the commercial property, generating a surveillance alert, and transmitting the surveillance alert to an electronic device associated with an owner of the commercial property.

Abstract: Embodiments described herein provide for training a machine learning model for automatic organ segmentation. A processor executes a machine learning model using an image to output at least one predicted organ label for a plurality of pixels of the image. Upon transmitting the at least one predicted organ label to a correction computing device, the processor receives one or more image fragments identifying corrections to the at least one predicted organ label. Upon transmitting the one or more image fragments and the image to a plurality of reviewer computing devices, the processor receives a plurality of inputs indicating whether the one or more image fragments are correct. When a number of inputs indicating an image fragment of the image fragments is correct exceeds a threshold, the processor aggregates the image fragment into a training data set. The processor trains the machine learning model with the training data set.

Abstract: An auto-tagging engine receives a training set of data comprising documents including a set of tagged fields with each tagged field corresponding to a portion of the document. The auto-tagging engine trains a machine learned model using the training set of data. The trained machine learned model, when applied to a target document in a document management environment, identifies portions of the target document each corresponding to fields of the target document. For each field of the target document, the auto-tagging engine identifies text of the target document associated with the identified portions of the target document corresponding to fields. Natural language processing is performed on the identified text in order to identify field types for the fields. The target document is automatically modified to include a tag identifying the portion of the target document corresponding to each field and identifying a field type of the field.

Abstract: Provided is an image analysis unit which analyzes a captured image of a camera mounted on a mobile device, executes object identification of an image, and sets a label as an identification result to an image region; a low-confidence region extraction unit which extracts a region with low confidence of object identification from an image analysis result; and a label updating unit which updates a label of the low-confidence region on the basis of information received via a communication unit. The label updating unit updates a label in a case where a matching rate between an object region analyzed from information received via the communication unit and the low-confidence region is equal to or greater than a specified threshold.

Abstract: A method and system for detecting a typical object of a transmission line based on UAV federated learning. The method includes: determining a detection model for a typical object of a transmission line by YOLOv3 object detection algorithm according to a prior database for the typical object; dividing a UAV network into multiple federated learning units; acquiring pictures, taken by the UAV network, of the typical object and tags corresponding to each picture to determine a training database; training, based on Horovod framework and FATE federated learning framework, each federated learning unit according to the training database and the detection model for the typical object, and determining the trained UAV network according to the trained federated learning unit; and determining, by the trained UAV network, the typical object in each picture. A congestion of communication links is avoided, thereby improving detection efficiency.

Abstract: An abnormality detection apparatus (100) includes an acquisition unit (110) that acquires input data, an abnormality degree computation unit (120) that has a discriminative model for computing an abnormality degree of input data, inputs the acquired input data to the discriminative model, and thereby computes an abnormality degree of the input data, a normality degree computation unit (130) that has a normality model for computing a normality degree of input data, inputs the input data to the normality model, and thereby computes a normality degree of the input data, a determination unit (140) that has a determination model for performing determination relating to an abnormality level of input data, inputs the abnormality degree and the normality degree to the determination model, and thereby performs determination relating to an abnormality level of the input data, and an output unit (150) that outputs output information based on a result of the determination.

Abstract: Methods and systems for providing an interactive image scene graph pattern search are provided. A user is provide with an image having a plurality of selectable segmented regions therein. The user selects one or more of the segmented regions to build a query graph. Via a graph neural network, matching target graphs are retrieved that contain the query graph from a target graph database. Each matching target graph has matching target nodes that match with the query nodes of the query graph. Matching target images from an image database are associated with the matching target graphs. Embeddings of each of the query nodes and the matching target nodes are extracted. A comparison of the embeddings of each query node with the embeddings of each matching target node is performed. The user interface displays the matching target images that are associated with the matching target graphs.

Abstract: Techniques for synthetic data generation in computer-based reasoning systems are discussed and include receiving a request for generation of synthetic data based on a set of training data cases. One or more focal training data cases are determined. For undetermined features (either all of them or those that are not subject to conditions), a value for the feature is determined based on the focal cases. In some embodiments, the generated synthetic data may be checked for similarity against the training data, and if similarity conditions are met, it may be modified (e.g., resampled), removed, and/or replaced.

Abstract: Systems and methods for analyzing image data to identify cabinet products are disclosed. A computer-implemented method may include receiving, from an electronic device via a network connection, at least one digital image depicting a cabinet. The method also may include analyzing, by one or more processors, the at least one digital image to determine a first set of characteristics of the cabinet. Additionally, the method may include accessing, by the one or more processors from memory, a second set of characteristics corresponding to a plurality of cabinet products and comparing the first set of characteristics to the second set of characteristics to identify a cabinet product of the plurality of cabinet products that matches the cabinet. Further, the method may include transmitting, to the electronic device via the network connection, an indication of the cabinet product.

Abstract: A method for context based lane prediction, the method may include obtaining sensed information regarding an environment of the vehicle; providing the sensed information to a second trained machine learning process; and locating one or more lane boundaries by the second trained machine learning process. The second trained machine learning process is generated by: performing a self-supervised training process, using a first dataset, of a first machine learning process to provide a first trained machine learning process; wherein the first trained machine learning process comprises a first encoder portion and a first decoder portion; replacing the first decoder portion by a second decoder portion to provide a second machine learning process; and performing an additional training process, using a second dataset that is associated with lane boundary metadata, of the second machine learning process to provide a second trained machine learning process.

Abstract: A simplified handwriting recognition approach includes a first network comprising convolutional neural network comprising one or more convolutional layers and one or more max-pooling layers. The first network receives an input image of handwriting and outputs an embedding based thereon. A second network comprises a network of cascaded convolutional layers including one or more subnetworks configured to receive an embedding of a handwriting image and output one or more character predictions. The subnetworks are configured to downsample and flatten the embedding to a feature map and then a vector before passing the vector to a dense neural network for character prediction. Certain subnetworks are configured to concatenate an input embedding with an upsampled version of the feature map.

Abstract: A system including one or more processors and one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform generating one or more item relational graphs for one or more items based on historical user purchases; transforming, using spectral filtering, the one or more item relational graphs into one or more frequency signals to remove noise from the one or more frequency signals; constructing, using a machine learning model, one or more item pair label classifications for one or more item pairs of the one or more items; generating a respective similarity score for each of the one or more item pairs; outputting a top k results for the one or more item pairs ranked by the respective similarity scores; and re-ranking, using a re-ranking algorithm, the top k results of the one or more item pairs based on a user preference for display on a user interface of an electronic device of a user. Other embodiments are disclosed.

Abstract: An image recognition method and system based on deep learning are provided. The image recognition system includes a first recognizing engine, at least one second recognizing engine and a processing circuit. The second recognizing engine is activated to recognize an image when the first recognizing engine is recognizing the image. The processing circuit determines whether to interrupt the first recognizing engine recognizing the image according to a result outputted by the second recognizing engine after the second recognizing engine completes recognition of the image.

Abstract: A method for detecting road conditions applied in an electronic device obtains images of a scene in front of a vehicle, and inputs the images into a trained semantic segmentation model. The electronic device inputs the images into a backbone network for feature extraction and obtains a plurality of feature maps, inputs the feature maps into the head network, processes the feature maps by a first segmentation network of the head network, and outputs a first recognition result. The electronic device further processes the feature maps by a second segmentation network of the head network, and outputs a second recognition result, and determines whether the vehicle can continue to drive on safely according to the first recognition result and the second recognition result.

Abstract: Embodiments of the present disclosure provide a method, a device, and a storage medium for targeted adversarial discriminative domain adaptation (T-ADDA). The method includes pre-training a source model including a source feature encoder and a source classifier, adapting a target feature encoder, and generating a target model by concatenating the adapted target feature encoder with the pre-trained source classifier.

Abstract: Certain embodiments involve using a machine-learning tool to generate metadata identifying segments and topics for text within a document. For instance, in some embodiments, a text processing system obtains input text and applies a segmentation-and-labeling model to the input text. The segmentation-and-labeling model is trained to generate a predicted segment for the input text using a segmentation network. The segmentation-and-labeling model is also trained to generate a topic for the predicted segment using a pooling network of the model to the predicted segment. The output of the model is usable for generating metadata identifying the predicted segment and the associated topic.

Abstract: A diagnostic assistance system for assisting a person in performing diagnosis of an object includes target data storage means for storing target data indicating a state of the object, diagnostic task means for providing each of a first user and a second user with the target data to enable each of the first user and the second user to perform a diagnostic task of the object, diagnostic result means for receiving a result of diagnosis of the object by each of the first user and the second user, and sharing means for enabling the first user and the second user to share result of diagnosis by each of the first user and the second user. The diagnostic task includes a first sub-task of investigating the target data to output a first diagnostic result and a second sub-task of using the first diagnostic result to output a second diagnostic result.

Abstract: A method for handling contradictory queries on a shared device includes receiving a first query issued by a first user, the first query specifying a first long-standing operation for a digital assistant to perform, and while the digital assistant is performing the first long-standing operation, receiving a second query, the second query specifying a second long-standing operation for the digital assistant to perform. The method also includes determining that the second query was issued by another user different than the first user and determining, using a query resolver, that performing the second long-standing operation would conflict with the first long-standing operation. The method further includes identifying one or more compromise operations for the digital assistant to perform, and instructing the digital assistant to perform a selected compromise operation among the identified one or more compromise operations.

Abstract: Disclosed herein is an object detection system, including apparatuses and methods for object detection. An implementation may include receiving a first image frame from an ROI detection model that generated a first ROI boundary around a first object detected in the first image frame and subsequently receiving a second image frame. The implementation further includes predicting, using an ROI tracking model, that the first ROI boundary will be present in the second image frame and then detecting whether the first ROI boundary is in fact present in the second image frame. The implementation includes determining that the second image frame should be added to a training dataset for the ROI detection model when detecting that the ROI detection model did not generate the first ROI boundary in the second image frame as predicted and re-training the ROI detection model using the training dataset.

Abstract: Disclosed are systems and methods that convert digital video data, such as two-dimensional digital video data, into a natural language text description describing the subject matter represented in the video. For example, the disclosed implementations may process video data in real-time, near real-time, or after the video data is created and generate a text-based video narrative describing the subject matter of the video. In addition, the disclosed implementations may also support a question and answer session in which a user may submit queries about the subject matter of one or more videos and the disclosed implementations will present natural language responses based on the subject matter of the video and any corresponding context.

Abstract: Provided in the present application are a method and apparatus for training a visual language pre-training model, and a device and a medium. The method includes: acquiring pairing groups respectively corresponding to N images, wherein the pairing group of a first image includes: a first pairing group which is composed of the first image and description text of the first image, and a second pairing group which is composed of a local image of the first image and description text of the local image, N is an integer greater than 1, and the first image is any one of the N images; and training a visual language pre-training model according to the pairing groups respectively corresponding to the N images.

Abstract: A method for lifelong machine learning using boosting includes receiving a new task and a learning sample for the new task. A distribution of weights is learned over the learning sample using previously learned classifiers from old tasks. A set of task-specific classifiers are learned for the new task using a boosting algorithm and the distribution of weights over the learning sample, whereby the distribution of weights over the learning sample is updated using the task-specific classifiers for the new task.

Abstract: Disclosed herein is a method to disentangle linear-encoded facial semantics from facial images without external supervision. The method uses linear regression and sparse representation learning concepts to make the disentangled latent representations easily interpreted and manipulated. Generated facial images are decomposed into multiple semantic features and latent representations are extracted to capture interpretable facial semantics. The semantic features may be manipulated to synthesize photorealistic facial images by sampling along vectors representing the semantic features, thereby changing the associate semantics.

Abstract: A learning device includes a class classification learning unit that learns class classification of a classification target by using a loss function in which a loss is calculated to become smaller as a magnitude of a difference between a function value obtained by inputting a log-likelihood ratio to a function having a finite value range and a constant associated with a correct answer to the class classification of the classification target becomes smaller, the log-likelihood ratio being the logarithm of a ratio between the likelihood that the classification target belongs to a first class and the likelihood that the classification target belongs to a second class.

Abstract: A method of performing person re-identification includes: obtaining a person feature vector according to an extracted image containing a person; obtaining state information of the person according to a state of the person in the extracted image; comparing the person feature vector with a plurality of registered person feature vectors in a database; when the person feature vector successfully matches a first registered person feature vector of the plurality of registered person feature vectors, identifying the person as a first identity corresponding to the first registered person feature vector; and selectively utilizing the person feature vector to update one of the first registered person feature vector and at least one second registered person feature vector that correspond to the first identity according to the state information.

Abstract: A computer-implemented neural network system including a first machine learning system, in particular a first neural network, a second machine learning system, in particular a second neural network, and a third machine learning system, in particular a third neural network. The first machine learning system is designed to ascertain a higher-dimensional constructed image from a predefinable low-dimensional latent variable. The second machine learning system is designed to ascertain the latent variable again from the higher-dimensional constructed image, and the third machine learning system is designed to distinguish whether or not an image it receives is a real image.

Abstract: Performing semantic segmentation in an absence of labels for one or more semantic classes is provided. One or more weak predictors are utilized to obtain label proposals of novel classes for an original dataset for which at least a subset of sematic classes are unlabeled classes. The label proposals are merged with ground truth of the original dataset to generate a merged dataset, the ground truth defining labeled classes of portions of the original dataset. A machine learning model is trained using the merged dataset. The machine learning model is utilized for performing semantic segmentation on image data.

Abstract: Methods, system, and computer storage media are provided for multi-modal localization. Input data comprising two modalities, such as image data and corresponding text or audio data, may be received. A phrase may be extracted from the text or audio data, and a neural network system may be utilized to spatially and temporally localize the phrase within the image data. The neural network system may include a plurality of cross-modal attention layers that each compare features across the first and second modalities without comparing features of the same modality. Using the cross-modal attention layers, a region or subset of pixels within one or more frames of the image data may be identified as corresponding to the phrase, and a localization indicator may be presented for display with the image data. Embodiments may also include unsupervised training of the neural network system.

Abstract: Various embodiments relate generally to data science and data analysis, computer software and systems, to provide a platform to facilitate updating compatible distributed data files, among other things, and, more specifically, to a computing and data platform that implements logic to facilitate correlation of event data via analysis of electronic messages, including executable instructions and content, etc., via a cross-stream data processor application configured to, for example, update or modify one or more compatible distributed data files automatically.

Abstract: Embodiments described herein a CROss-Modal Distribution Alignment (CROMDA) model for vision-language pretraining, which can be used for retrieval downstream tasks. In the CROMDA mode, global cross-modal representations are aligned on each unimodality. Specifically, a uni-modal global similarity between an image/text and the image/text feature queue are computed. A softmax-normalized distribution is then generated based on the computed similarity. The distribution thus takes advantage of property of the global structure of the queue. CROMDA then aligns the two distributions and learns a modal invariant global representation. In this way, CROMDA is able to obtain invariant property in each modality, where images with similar text representations should be similar and vice versa.