Abstract: The decision process of a first machine learning (ML) model may be explained based on a second ML model implemented on an apparatus. The apparatus may obtain a prediction about an image made based on the first ML model. The apparatus may further determine visual concepts associated with the image that may have been used by the first ML model to make the prediction, and determine respective contributions of the visual concepts to the prediction made by the first ML model. The apparatus may then generate, based on the second ML model, a textual description that explains the respective contributions of the visual concepts to the prediction made by the first ML model. The second ML model may determine respective image features associated with the visual concepts, map the determined image features to corresponding text features, and generate the textual description based at least on the text features.

Abstract: An apparatus in accordance with embodiments of the present disclosure may obtain an image depicting one or more hands of a person in a medical environment; and detect, using a first machine learning (ML) model, a plurality of 2D landmarks associated with a hand of the person depicted in the image. The apparatus may further determine, using a second ML model, 3D features of the hand of the person based on the plurality of 2D landmarks. The apparatus may determine a gesture indicated by the hand of the person based on the 3D features of the hand predicted using the second ML model. Alternatively, in determining the 3D features of the hand, the system may stack the plurality of 2D landmarks across a sequence of image frames in a video, and use a third ML model to determine the 3D features of the hand based on the stacked 2D landmarks.

Abstract: Multiple predictions about the position of an object during a time period may each indicate the position of the object at a respective time during the time period. Respective validity indications corresponding to the multiple predictions may each indicate an accuracy of the corresponding prediction. Whether a change has occurred in a distribution of the predictions from a first subset of predictions to a second subset of predictions during the time period may be determined. If the change has occurred, a prediction from the first subset of predictions or the second subset of predictions may be selected, based on the validity of the predictions and/or the detection of a motion, as a best indication of the position of the object.

Abstract: Described herein are machine learning (ML) based on systems, methods, and instrumentalities associated with image search and/or retrieval. An apparatus as described herein may obtain a query image and a textual description associated with the query image, and generate, using an artificial neural network (ANN), a feature representation that may represent the image and the textual description as an associated pair. Based on the feature representation, the apparatus may identify one or more images from an image repository and provide an indication regarding the one or more identified images, for example, as a ranked list.

Abstract: The physical characteristics of one or more anatomical structures of a person may change in accordance with conditions surrounding the determination of such physical characteristics. Machine learning based techniques may be used to determine a template representation of the one or more anatomical structures that may indicate the physical characteristics of the one or more anatomical structures free of the impact imposed by changing conditions. The template representation may then be used to predict the physical characteristics of the one or more anatomical structures under a new set of conditions, without subjecting the person to additional medical scans.

Abstract: Detecting motions associated with a body part of a patient may include using an image sensor installed inside a medical scanner to capture first and second images of the patient inside the medical scanner, wherein the first image may depict the patient in a first state and the second image may depict the patient in a second state. A first area, in the first image, that corresponds to the body part of the patient may be identified and a second area, in the second image, that corresponds to the body part may also be identified so that a first plurality of features may be extracted from the first area of the first image and a second plurality of features may be extracted from the second area of the second image. A motion associated with the body part of the patient may be determined based on the first and second pluralities of features.

Abstract: Disclosed herein are systems, methods and instrumentalities associated with multi-person joint location and pose estimation based on an image that depicts multiple people in a scene, where at least some of the joint locations of a person may be blocked or obstructed by other people or objects in the scene. The estimation may be performed by detecting and grouping joint locations in the image using a bottom-up approach, and refining each group of detected joint locations by recovering obstructed joint location(s) that may be missing from the group. The detection, grouping, and/or refinement may be accomplished based on one or more machine learning (ML) models that may be implemented using artificial neural networks such as convolutional neural networks.

Abstract: Automatic hand gesture determination may be a challenging task considering the complex anatomy and high dimensionality of the human hand. Disclosed herein are systems, methods, and instrumentalities associated with recognizing a hand gesture in spite of the challenges. An apparatus in accordance with embodiments of the present disclosure may use machine learning based techniques to identify the area of an image that may contain a hand and to determine an orientation of the hand relative to a pre-defined direction. The apparatus may then adjust the area of the image containing the hand to align the orientation of the hand with the pre-defined direction and/or to scale the image area to a pre-defined size. Based on the adjusted image area, the apparatus may detect a plurality of hand landmarks and predict a gesture indicated by the hand based on the plurality of detected landmarks.

Abstract: A person's privacy is protected by the law in many settings and disclosed herein are systems, methods, and instrumentalities associated with anonymizing an image of a person while still preserving the visual saliency and/or utility of the image for one or more downstream tasks. These objectives may be accomplished using various machine-learning (ML) techniques such as ML models trained for extracting identifying and residual features from the input image as well as ML models trained for transforming the identifying features into identity-concealing features and for preserving the utility features of the image. An output image may be generated based on the various ML models, wherein the identity of the person may be substantially disguised in the output image while the background and utility attributes of the original image may be substantially maintained in the output image.

Abstract: A display method includes: processing a plurality of images to form base compositions; and presenting the base compositions with different images in a spatial arrangement which as a whole form a composite image viewable to a naked-eye viewer; wherein a subset of the plurality of images are selectively viewable as a modulated view to a user with an optical modulation device.

Abstract: An image processing method includes: determining chrominance information and luminance information of an input color image; iteratively solving one or more equations to obtain bases of the luminance and modulation weights of a plurality of basis element frames of the input color image based on the luminance information; determining bases of the chrominance of the basis element frames based on the modulation weights and the chrominance information; and converting the bases of the luminance and the bases of the chrominance of the basis element frames into an output image.

Abstract: An image processing method includes: determining chrominance information and luminance information of an input color image; iteratively solving one or more equations to obtain bases of the luminance and modulation weights of a plurality of basis element frames of the input color image based on the luminance information; determining bases of the chrominance of the basis element frames based on the modulation weights and the chrominance information; and converting the bases of the luminance and the bases of the chrominance of the basis element frames into an output image.

Abstract: A display method includes: processing a plurality of images to form base compositions; and presenting the base compositions with different images in a spatial arrangement which as a whole form a composite image viewable to a naked-eye viewer; wherein a subset of the plurality of images are selectively viewable as a modulated view to a user with an optical modulation device.