Abstract: Images captured by different image capturing devices may have different fields of views and/or resolutions. One or more of these images may be aligned based on an image template, and additional details for the adapted images may be predicted using a machine-learned data recovery model and added to the adapted images such that the images may have the same field of view or the same resolution.

Abstract: Described herein are systems, methods, and instrumentalities associated with automatically annotating a 3D image dataset. The 3D automatic annotation may be accomplished based on a 2D annotation provided by an annotator and by propagating the 2D annotation through multiple images of a sequence of 2D images associated with the 3D image dataset. The automatically annotated 3D image dataset may then be used to annotate other 3D image datasets based on similarities between the first 3D image dataset and the other 3D image datasets. The automatic annotation of the first 3D image dataset and/or the other 3D image datasets may be conducted based on one or more machine-learning models trained for performing those tasks.

Abstract: The pose and shape of a human body may be recovered based on joint location information associated with the human body. The joint location information may be derived based on an image of the human body or from an output of a human motion capture system. The recovery of the pose and shape of the human body may be performed by a computer-implemented artificial neural network (ANN) trained to perform the recovery task using training datasets that include paired joint location information and human model parameters. The training of the ANN may be conducted in accordance with multiple constraints designed to improve the accuracy of the recovery and by artificially manipulating the training data so that the ANN can learn to recover the pose and shape of the human body even with partially observed joint locations.

Abstract: The present disclosure generally provides systems and methods for situation awareness. When executing a set of instructions stored in at least one non-transitory storage medium, at least one processor may be configured to cause the system to perform operations including obtaining, from at least one of one or more sensors, environmental data associated with an environment corresponding to a first time point, generating a first static global representation of an environment corresponding to the first time point based at least in part on the environmental data, generating a first dynamic global representation of the environment corresponding to the first time point based at least in part on the environmental data, and estimating, based on the first static global representation and the first dynamic global representation, a target state of the environment at a target time point using a target estimation model.

Abstract: Disclosed is a method and a system for automatic positioning of a medical equipment with respect to a patient. The method includes obtaining sensor data related to the patient, from a plurality of sensors fixed relative to the medical equipment. The method further includes processing the sensor data to determine at least one pose characteristic of the patient and at least one shape characteristic of the patient. The method further includes determining at least one adjustment parameter for the medical equipment based on the at least one pose characteristic of the patient and the at least one shape characteristic of the patient. The method further includes adjusting the medical equipment based on the at least one adjustment parameter.

Abstract: Systems, methods, and instrumentalities are described herein for constructing a multi-view patient model (e.g., a 3D human mesh model) based on multiple single-view models of the patient. Each of the single-view models may be generated based on images captured by a sensing device and, dependent on the field of the view of the sensing device, may depict some keypoints of the patient's body with a higher accuracy and other keypoints of the patient's body with a lower accuracy. The multi-view patient model may be constructed using respective portions of the single-view models that correspond to accurately depicted keypoints. This way, a comprehensive and accurate depiction of the patient's body shape and pose may be obtained via the multi-view model even if some keypoints of the patient's body are blocked from a specific sensing device.

Abstract: A three-dimensional (3D) model of a person may be obtained using a pre-trained neural network based on one or more images of the person. Such a model may be subject to estimation bias and/or other types of defects or errors. Described herein are systems, methods, and instrumentalities for refining the 3D model and/or the neural network used to generate the 3D model. The proposed techniques may extract information such as key body locations and/or a body shape from the images and refine the 3D model and/or the neural network using the extracted information. In examples, the 3D model and/or the neural network may be refined by minimizing a difference between the key body locations and/or body shape extracted from the images and corresponding key body locations and/or body shape determined from the 3D model. The refinement may be performed in an iterative and alternating manner.

Abstract: Systems, methods and instrumentalities are described herein for automating a medical environment. The automation may be realized using one or more sensing devices and at least one processing device. The sensing devices may be configured to capture images of the medical environment and provide the images to the processing device. The processing device may determine characteristics of the medical environment based on the images and automate one or more aspects of the operations in the medical environment. These characteristics may include, e.g., people and/or objects present in the images and respective locations of the people and/or objects in the medical environment. The operations that may be automated may include, e.g., maneuvering and/or positioning a medical device based on the location of a patient, determining and/or adjusting the parameters of a medical device, managing a workflow, providing instructions and/or alerts to a patient or a physician, etc.

Abstract: Automated patient positioning and modelling includes a hardware processor to obtain image data from an imaging sensor, classify the image data, using a first machine learning model, as a patient pose based on one or more pre-defined protocols for patient positioning, provide a confidence score based on the classification of the image data and if the confidence score is less than a pre-determined value, re-classify the image data using a second machine learning model; or if the confidence score is greater than a pre-determined value, identify the image data as corresponding to a patient pose based on one or more pre-defined protocols for patient positioning during a scan procedure.

Abstract: A system for patient positioning is provided. The system may acquire image data relating to a patient holding a posture and a plurality of patient models. Each patient model may represent a reference patient holding a reference posture, and include at least one reference interest point of the referent patient and a reference representation of the reference posture. The system may also identify at least one interest point of the patient from the image data using an interest point detection model. The system may further determine a representation of the posture of the patient based on a comparison between the at least one interest point of the patient and the at least one reference interest point in each of the plurality of patient models.

Abstract: Described herein are systems, methods, and instrumentalities associated with automatic image annotation. The annotation may be performed based on one or more manually annotated first images of an object and a machine-learned (ML) model trained to extract first features from the one or more first images. To automatically annotate a second, un-annotated image of the object, the ML model may be used to extract second features from the second image, determine information that may be indicative of the characteristics of the object in the second image based on the first and second features, and generate an annotation of the object for the second image using the determined information. The images may be obtained from various sources including, for example, sensors and/or medical scanners, and the object of interest may include anatomical structures such as organs, tumors, etc. The annotated images may be used for multiple purposes including machine learning.

Abstract: Human mesh model recovery may utilize prior knowledge of the hierarchical structural correlation between different parts of a human body. Such structural correlation may be between a root kinematic chain of the human body and a head or limb kinematic chain of the human body. Shape and/or pose parameters relating to the human mesh model may be determined by first determining the parameters associated with the root kinematic chain and then using those parameters to predict the parameters associated with the head or limb kinematic chain. Such a task can be accomplished using a system comprising one or more processors and one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to implement one or more neural networks trained to perform functions related to the task.

Abstract: An apparatus is configured to receive input image data corresponding to output image data of a first radiology scanner device, translate the input image data into a format corresponding to output image data of a second radiology scanner device and generate an output image corresponding to the translated input image data on a post processing imaging device associated with the first radiology scanner device. Medical images from a new scanner can be translate to look as if they came from a scanner of another vendor.

Abstract: An automated process for data annotation of medical images includes obtaining image data from an imaging sensor, partitioning the image data, identifying an object of interest in the partitioned image data, generating an initial contour with one or more control points with respect to the object of interest, identifying a manual adjustment of one of the control points, automatically adjust a position of at least one other control point within a predetermined range of the manually adjusted control point to a new position, the new position of the at least one other control point and manually adjusted control point defining a new contour, and generating an updated image with the new contour and corresponding control points.

Abstract: Automatically validating the calibration of an visual sensor network includes acquiring image data from visual sensors that have partially overlapping fields of view, extracting a representation of an environment in which the visual sensors are disposed, calculating one or more geometric relationships between the visual sensors, comparing the calculated one or more geometric relationships with previously obtained calibration information of the visual sensors, and verifying a current calibration of the visual sensors based on the comparison.

Abstract: An apparatus for automated collision avoidance includes a sensor configured to detect an object of interest, predicting a representation of the object of interest at a future point in time, calculating an indication of a possibility of a collision with the object of interest based on the representation of the object of interest at the future point in time, and executing a collision avoidance action based on the indication.

Abstract: Human model recovery may be realized utilizing pre-trained artificially neural networks. A first neural network may be trained to determine body keypoints of a person based on image(s) of a person. A second neural network may be trained to predict pose parameters associated with the person based on the body keypoints. A third neural network may be trained to predict shape parameters associated with the person based on depth image(s) of the person. A 3D human model may then be generated based on the pose and shape parameters respectively predicted by the second and third neural networks. The training of the second neural network may be conducted using synthetically generated body keypoints and the training of the third neural network may be conducted using normal maps. The pose and shape parameters predicted by the second and third neural networks may be further optimized through an iterative optimization process.

Abstract: A method for automated calibration is provided. The method may include obtaining a plurality of interest points based on prior information regarding a device and image data of the device captured by a visual sensor. The method may include identifying at least a portion of the plurality of interest points from the image data of the device. The method may also include determining a transformation relationship between a first coordinate system and a second coordinate system based on information of at least a portion of the identified interest points in the first coordinate system and in the second coordinate system that is applied to the visual sensor or the image data of the device.

Abstract: The shape and/or location of an organ may change in accordance with changes in the body shape and/or pose of a patient. Described herein are systems, methods, and instrumentalities for automatically determining, using an artificial neural network (ANN), the shape and/or location of the organ based on human models that reflect the body shape and/or pose the patient. The ANN may be trained to learn the spatial relationship between the organ and the body shape or pose of the patient. Then, at an inference time, the ANN may be used to determine the relationship based on a first patient model and a first representation (e.g., a point cloud) of the organ so that given a second patient model thereafter, the ANN may automatically determine the shape and/or location of the organ corresponding to the body shape or pose of the patient indicated by the second patient model.

Abstract: Automatically generating an explanation for a decision prediction from a machine learning algorithm includes using a first processor of a computing device to run the machine learning algorithm using one or more input data; generating a decision prediction output based on the one or more input data; using a second processor to access the decision prediction output of the first processor; generating additional information that identifies one or more causal relationships between the prediction of the first algorithm and the one or more input data; and providing the additional information as the explanation in a user-understandable format on a display of the computing device.