Abstract: A method extracts a trajectory of an arrow in a binary image. The method extracts a skeleton of an arrow in the image and splits the skeleton into bones. For each of the bones, the method extends a bone to obtain an extended bone having both ends of the bone reaching a border of the arrow, and warps the extended bone into a straight line using a thin plate spline. For each bone, the method warps the binary image into a warped image using a thin plate spline with same parameters as the thin plate spline used to warp the extended bone into the straight line. For each bone, the method selects a component that includes a center point of the warped image, computes a symmetric error of the selected component, and selects, as the trajectory of the arrow, a bone where the selected component has the lowest symmetric error.

Abstract: A computer-implemented method is provided for training a multi-source sound localization model using labeled simulation data and unlabeled real data. The method includes inputting the labeled simulation data and the unlabeled real data respectively into a multi-source sound localization model of a neural network to obtain a localization heatmap from an output layer of the multi-source sound localization model for each of the labeled simulation data and the unlabeled real data. The method further includes inputting the localization heatmap for each of the labeled simulation data and the unlabeled real data into an output discriminator. The method also includes training the output discriminator so that the output discriminator assigns a domain class label to distinguish simulation data from real data. The method additionally includes training, by a hardware process, the multi-source sound localization model by a first adversarial loss for the output discriminator with an original localization model loss.

Abstract: A method for training a neural network to cluster multiple people in an image into groups and estimate an activity of each of the groups is provided. The method extracts a feature of each of the multiple people in the image. The method inputs the feature to the neural network to estimate an affinity matrix A and the activity of each of the groups. The method calculates a first loss between the estimated affinity matrix and a ground truth affinity matrix for the image, and a second loss between the estimated activity of each of the groups and a ground truth activity of each of the groups for the image. The first loss is calculated using Maximum Spanning Trees. The method trains the neural network based on the first and second losses.

Abstract: A method, a computer program product, and a computer system identifies a last person of a queue. The method includes receiving an image of the queue where the image includes a plurality of individuals comprising the queue. The method includes determining positions and facing directions of the individuals comprising the queue. The method includes identifying the last person of the queue based on a vector field analysis according to the positions and the facing directions of the individuals comprising the queue. The method includes generating instructions to join the queue based on the identified last person of the queue. The instructions include a modified image indicating the last person of the queue in the image.

Abstract: A method, a computer program product, and a computer system identifies a last person of a queue. The method includes receiving an image of the queue where the image includes a plurality of individuals comprising the queue. The method includes determining positions and facing directions of the individuals comprising the queue. The method includes identifying the last person of the queue based on a vector field analysis according to the positions and the facing directions of the individuals comprising the queue. The method includes generating instructions to join the queue based on the identified last person of the queue. The instructions include a modified image indicating the last person of the queue in the image.

Abstract: A method, system and computer program product are presented for generating a description in natural language for a color. The method includes acquiring a list of tuples, generating a graph by using each of the tuples as a node and adding edges between the nodes when a difference between colors of the nodes in terms of human perception is outside a predetermined range, filtering the edges based on external color comparative descriptions, incorporating a new node in the graph by finding a closest neighbor node based on the color difference and adding a new edge between the new node and the closest neighbor node, learning a feature vector for each of the nodes by using message passing and the colors of the nodes as initial seeds, and generating a description of the new node by using each of the learned feature vectors as initial states for a neural-network based decoder.

Abstract: Methods and systems for localizing a sound source include determining a spatial transformation between a position of a reference microphone array and a position of a displaced microphone array. A sound is measured at the reference microphone array and at the displaced microphone array. A source of the sound is localized using a neural network that includes respective paths for the reference microphone array and the displaced microphone array. The neural network further includes a transformation layer that represents the spatial transformation.

Abstract: A method, system and computer program product are presented for generating a description in natural language for a color. The method includes acquiring a list of tuples, generating a graph by using each of the tuples as a node and adding edges between the nodes when a difference between colors of the nodes in terms of human perception is outside a predetermined range, filtering the edges based on external color comparative descriptions, incorporating a new node in the graph by finding a closest neighbor node based on the color difference and adding a new edge between the new node and the closest neighbor node, learning a feature vector for each of the nodes by using message passing and the colors of the nodes as initial seeds, and generating a description of the new node by using each of the learned feature vectors as initial states for a neural-network based decoder.

Abstract: Methods and systems for localizing a sound source include determining a spatial transformation between a position of a reference microphone array and a position of a displaced microphone array. A sound is measured at the reference microphone array and at the displaced microphone array. A source of the sound is localized using a neural network that includes respective paths for the reference microphone array and the displaced microphone array. The neural network further includes a transformation layer that represents the spatial transformation.

Abstract: A computer-implemented method is provided for training a multi-source sound localization model using labeled simulation data and unlabeled real data. The method includes inputting the labeled simulation data and the unlabeled real data respectively into a multi-source sound localization model of a neural network to obtain a localization heatmap from an output layer of the multi-source sound localization model for each of the labeled simulation data and the unlabeled real data. The method further includes inputting the localization heatmap for each of the labeled simulation data and the unlabeled real data into an output discriminator. The method also includes training the output discriminator so that the output discriminator assigns a domain class label to distinguish simulation data from real data. The method additionally includes training, by a hardware process, the multi-source sound localization model by a first adversarial loss for the output discriminator with an original localization model loss.

Abstract: A semi-supervised computer-implemented method for group and group activity detection is provided. The method includes detecting, by a hardware processor, entity areas in an image. The method further includes extracting relative position features from pairs of the entity areas. The method also includes extracting pixel features from the pairs of the entity areas. The method additionally includes combining the relative position features and the pixel features to generate edge features. The method further includes identifying, using a display device, groups formed from the entity areas by processing the edge features using an Edge-Labeling Graph Neural Network. The method also includes identifying, using the display device, for each of the groups of the entity areas, a group activity performed by the persons therein based on a result of a voting scheme.

Abstract: A computer-implemented method is presented for constructing a trained model for a plurality of edge classifiers in a network having a federated classifier, a generator, and a discriminator. The method includes obtaining edge trained models from the plurality of edge devices, each edge trained model being trained independently with data from private data of each edge, training the generator model and discriminator model by employing the edge trained models and an unlabeled set of data by employing a generative adversarial training procedure, generating data samples by the trained generator model, training the federated classifier with the data samples from the generator model, and deploying the trained model back to the plurality of edge devices.

Abstract: Generating soft labels used for training a unified model is achieved by unification of models having respective target classes with distillation. A collection of samples is prepared. Predictions are generated by individual trained models. Individual trained models have an individual class set to form a unified class set that includes target classes. The unified soft labels are estimated for each sample over the target classes in the unified class set from the predictions using a relation connecting a first output of each individual trained model and a second output of the unified model. The unified soft labels are output to train a unified model having the unified class set.

Abstract: A computer-implemented method is presented for constructing a trained model for a plurality of edge classifiers in a network having a federated classifier, a generator, and a discriminator. The method includes obtaining edge trained models from the plurality of edge devices, each edge trained model being trained independently with data from private data of each edge, training the generator model and discriminator model by employing the edge trained models and an unlabeled set of data by employing a generative adversarial training procedure, generating data samples by the trained generator model, training the federated classifier with the data samples from the generator model, and deploying the trained model back to the plurality of edge devices.

Abstract: A computer-implemented method for building a 3D map, includes: obtaining plural videos and plural video-related data units, each of the plural video-related data units indicating a feature of radio wave signals received at a place where a corresponding video has been taken; reconstructing plural 3D models, respectively, based on the plural videos; selecting a pair of 3D models from the plural 3D models based on similarity between a corresponding pair of video-related data units; and merging the pair of 3D models to obtain the 3D map.

Abstract: A computer-implemented method for building a 3D map, includes: obtaining plural videos and plural video-related data units, each of the plural video-related data units indicating a feature of radio wave signals received at a place where a corresponding video has been taken; reconstructing plural 3D models, respectively, based on the plural videos; selecting a pair of 3D models from the plural 3D models based on similarity between a corresponding pair of video-related data units; and merging the pair of 3D models to obtain the 3D map.