Abstract: A method for automated calibration is provided. The method may include obtaining a plurality of interest points based on prior information regarding a device and image data of the device captured by a visual sensor. The method may include identifying at least a portion of the plurality of interest points from the image data of the device. The method may also include determining a transformation relationship between a first coordinate system and a second coordinate system based on information of at least a portion of the identified interest points in the first coordinate system and in the second coordinate system that is applied to the visual sensor or the image data of the device.

Abstract: The shape and/or location of an organ may change in accordance with changes in the body shape and/or pose of a patient. Described herein are systems, methods, and instrumentalities for automatically determining, using an artificial neural network (ANN), the shape and/or location of the organ based on human models that reflect the body shape and/or pose the patient. The ANN may be trained to learn the spatial relationship between the organ and the body shape or pose of the patient. Then, at an inference time, the ANN may be used to determine the relationship based on a first patient model and a first representation (e.g., a point cloud) of the organ so that given a second patient model thereafter, the ANN may automatically determine the shape and/or location of the organ corresponding to the body shape or pose of the patient indicated by the second patient model.

Abstract: Systems, methods, and computer-readable media are described for performing weakly supervised semantic segmentation of input images that utilizes self-guidance on attention maps during training to cause a guided attention inference network (GAIN) to focus attention on an object in an input image in a holistic manner rather than only on the most discriminative parts of the image. The self-guidance is provided jointly by a classification loss function and an attention mining loss function. Extra supervision can also be provided by using a select number pixel-level labeled input images to enhance the semantic segmentation capabilities of the GAIN.

Abstract: Automatically generating an explanation for a decision prediction from a machine learning algorithm includes using a first processor of a computing device to run the machine learning algorithm using one or more input data; generating a decision prediction output based on the one or more input data; using a second processor to access the decision prediction output of the first processor; generating additional information that identifies one or more causal relationships between the prediction of the first algorithm and the one or more input data; and providing the additional information as the explanation in a user-understandable format on a display of the computing device.

Abstract: Automated patient positioning and modelling includes a hardware processor to obtain image data from an imaging sensor, classify the image data, using a first machine learning model, as a patient pose based on one or more pre-defined protocols for patient positioning, provide a confidence score based on the classification of the image data and if the confidence score is less than a pre-determined value, re-classify the image data using a second machine learning model; or if the confidence score is greater than a pre-determined value, identify the image data as corresponding to a patient pose based on one or more pre-defined protocols for patient positioning during a scan procedure.

Abstract: The pose and shape of a human body may be recovered based on joint location information associated with the human body. The joint location information may be derived based on an image of the human body or from an output of a human motion capture system. The recovery of the pose and shape of the human body may be performed by a computer-implemented artificial neural network (ANN) trained to perform the recovery task using training datasets that include paired joint location information and human model parameters. The training of the ANN may be conducted in accordance with multiple constraints designed to improve the accuracy of the recovery and by artificially manipulating the training data so that the ANN can learn to recover the pose and shape of the human body even with partially observed joint locations.

Abstract: Systems, methods, and instrumentalities are described herein for constructing a multi-view patient model (e.g., a 3D human mesh model) based on multiple single-view models of the patient. Each of the single-view models may be generated based on images captured by a sensing device and, dependent on the field of the view of the sensing device, may depict some keypoints of the patient's body with a higher accuracy and other keypoints of the patient's body with a lower accuracy. The multi-view patient model may be constructed using respective portions of the single-view models that correspond to accurately depicted keypoints. This way, a comprehensive and accurate depiction of the patient's body shape and pose may be obtained via the multi-view model even if some keypoints of the patient's body are blocked from a specific sensing device.

Abstract: A three-dimensional (3D) model of a person may be obtained using a pre-trained neural network based on one or more images of the person. Such a model may be subject to estimation bias and/or other types of defects or errors. Described herein are systems, methods, and instrumentalities for refining the 3D model and/or the neural network used to generate the 3D model. The proposed techniques may extract information such as key body locations and/or a body shape from the images and refine the 3D model and/or the neural network using the extracted information. In examples, the 3D model and/or the neural network may be refined by minimizing a difference between the key body locations and/or body shape extracted from the images and corresponding key body locations and/or body shape determined from the 3D model. The refinement may be performed in an iterative and alternating manner.

Abstract: Systems, methods and instrumentalities are described herein for automating a medical environment. The automation may be realized using one or more sensing devices and at least one processing device. The sensing devices may be configured to capture images of the medical environment and provide the images to the processing device. The processing device may determine characteristics of the medical environment based on the images and automate one or more aspects of the operations in the medical environment. These characteristics may include, e.g., people and/or objects present in the images and respective locations of the people and/or objects in the medical environment. The operations that may be automated may include, e.g., maneuvering and/or positioning a medical device based on the location of a patient, determining and/or adjusting the parameters of a medical device, managing a workflow, providing instructions and/or alerts to a patient or a physician, etc.

Abstract: A method for enhancing facial/object recognition includes receiving a query image, and providing a database of object images, including images relevant to the query image, each image having a first attribute and a second attribute with each of the first attribute and the second attribute having a first state and a second state. The method also includes creating an augmented database by generating a plurality of artificial images for each image in the database, the artificial images cooperating with the image to define a set of images including every combination of the first attribute and the second attribute in each of the first state and the second state, and comparing the query image to the images in the augmented database to find one or more matches.

Abstract: A method for image processing may include: obtaining an original image of a first style, the original image being generated by a first imaging device; obtaining a target transformation model; and generating a transferred image of a second style by transferring the first style of the original image using the target transformation model. The second style may be substantially similar to a target style of one or more other images generated by a second imaging device. The second style may be different from the first style.

Abstract: A system comprising a first computing apparatus in communication with multiple second computing apparatuses. The first computing apparatus may obtain a plurality of first trained machine learning models for a task from the multiple second computing apparatuses. At least a portion of parameter values of the plurality of first trained machine learning models may be different from each other. The first computing apparatus may also obtain a plurality of training samples. The first computing apparatus may further determine, based on the plurality of training samples, a second trained machine learning model by learning from the plurality of first trained machine learning models.

Abstract: The present disclosure relates to systems and methods for scanning a patient in an imaging system. The imaging system may include at least one camera directed at the patient. The systems and methods may obtain a plurality of images of the patient that are captured by the at least one camera. Each of the plurality of images may correspond to one of a series of time points. The systems and methods may also determine a motion of the patient over the series of time points based on the plurality of images of the patient. The systems and methods may further determine whether the patient is ready for scan based on the motion of the patient, and generate control information of the imaging system for scanning the patient in response to determining that the patient is ready for scan.

Abstract: The present disclosure relates to a method for scanning a patient in an imaging system. The imaging system may include one or more cameras directed at the patient. The method may include obtaining a position of each of the camera(s) relative to the imaging system. The method may also include obtain image data of the patient captured by the camera(s), wherein the image data may correspond to a first view with respect to the patient. The method may further include generating projection image data of the patient based on the image data and the position of each of the camera(s) relative to the imaging system, wherein the projection image data may correspond to a second view with respect to the patient different from the first view. The method may further include generating control information for scanning the patient based on the projection image data of the patient.

Abstract: Healthcare services can be automated utilizing a system that recognizes at least one characteristic of a patient based on images of the patient acquired by an image capturing device. Relying on information extracted from these images, the system may automate multiple aspects of a medical procedure such as patient identification and verification, positioning, diagnosis and/or treatment planning using artificial intelligence or machine learning techniques. By automating these operations, healthcare services can be provided remotely and/or with minimum physical contact between the patient and a medical professional.

Abstract: Image-based key points detection using a convolutional neural network (CNN) may be impacted if the key points are occluded in the image. Images obtained from additional imaging modalities such as depth and/or thermal images may be used in conjunction with RGB images to reduce or minimize the impact of the occlusion. The additional images may be used to determine adjustment values that are then applied to the weights of the CNN so that the convolution operations may be performed in a modality aware manner to increase the robustness, accuracy, and efficiency of key point detection.

Abstract: A method for automated calibration is provided. The method may include obtaining a plurality of interest points based on prior information regarding a device and image data of the device captured by a visual sensor. The method may include identifying at least a portion of the plurality of interest points from the image data of the device. The method may also include determining a transformation relationship between a first coordinate system and a second coordinate system based on information of at least a portion of the identified interest points in the first coordinate system and in the second coordinate system that is applied to the visual sensor or the image data of the device.

Abstract: The pose and shape of a human body may be recovered based on joint location information associated with the human body. The joint location information may be derived based on an image of the human body or from an output of a human motion capture system. The recovery of the pose and shape of the human body may be performed by a computer-implemented artificial neural network (ANN) trained to perform the recovery task using training datasets that include paired joint location information and human model parameters. The training of the ANN may be conducted in accordance with multiple constraints designed to improve the accuracy of the recovery and by artificially manipulating the training data so that the ANN can learn to recover the pose and shape of the human body even with partially observed joint locations.

Abstract: Aspects include receiving a request to perform an image classification task in a target domain. The image classification task includes identifying a feature in images in the target domain. Classification information related to the feature is transferred from a source domain to the target domain. The transferring includes receiving a plurality of pairs of task-irrelevant images that each includes a task-irrelevant image in the source domain and in the target domain. The task-irrelevant image in the source domain has a fixed correspondence to the task-irrelevant image in the target domain. A target neural network is trained to perform the image classification task in the target domain. The training is based on the plurality of pairs of task-irrelevant images. The image classification task is performed in the target domain and includes applying the target neural network to an image in the target domain and outputting an identified feature.

Abstract: Systems, methods and instrumentalities are described herein for automating a medical environment. The automation may be realized using one or more sensing devices and at least one processing device. The sensing devices may be configured to capture images of the medical environment and provide the images to the processing device. The processing device may determine characteristics of the medical environment based on the images and automate one or more aspects of the operations in the medical environment. These characteristics may include, e.g., people and/or objects present in the images and respective locations of the people and/or objects in the medical environment. The operations that may be automated may include, e.g., maneuvering and/or positioning a medical device based on the location of a patient, determining and/or adjusting the parameters of a medical device, managing a workflow, providing instructions and/or alerts to a patient or a physician, etc.