Abstract: Systems and methods are disclosed for performing operations comprising: receiving a plurality of training images representing different phases of a periodic motion of a target region in a patient; applying a model to the plurality of training images to generate a lower-dimensional feature space representation of the plurality of training images; clustering the lower-dimensional feature space representation of the plurality of training images into a plurality of groups corresponding to the different phases of the periodic motion; and classifying a motion phase associated with a new image of the target region in the patient based on the plurality of groups of the clustered lower-dimensional feature space representation of the plurality of training images.

Abstract: Systems and methods are disclosed for performing automatic contour adaptation. The systems and methods receive first and second images depicting an anatomy of a subject and obtain a segmentation associated with the first image. The systems and methods apply a trained neural network to estimate the adapted segmentation corresponding to the anatomy depicted in the second image, the trained network consisting of two sub-networks: a registration sub-network, generating an initial segmentation estimate representing a deformation of the segmentation associated with the first image to fit the anatomy depicted in the second image, and a segmentation refinement sub-network, predicting a refined segmentation for the second image given the initial segmentation estimate.

Abstract: Systems and methods are disclosed for monitoring and estimating an anatomic position of a human subject for a radiotherapy treatment session, based on use of an artificial intelligence (AI) model (e.g., a generative AI model comprising a Transformer deep learning neural network), are described. An example method of monitoring anatomic position with a trained AI model includes: receiving position information corresponding to observed positions of a tracked anatomical area of a patient, observed during the radiotherapy treatment session; providing the position information as an input to a trained model trained with temporal sequences of observed anatomical positions from training data; determining an estimated position of the tracked anatomical area of the patient at a future time, based on output of the trained model; and controlling the radiotherapy treatment session based on the estimated position of the tracked anatomical area of the patient.

Abstract: Joint training techniques to train multiple models across clinical datasets for automatic contouring. Rather than using separate deep neural networks that are trained independently for each different dataset (e.g., a different image contrast or anatomy), joint training can be used to train multiple models simultaneously across clinical datasets for automatic contouring. By taking advantage of commonalities between two or more datasets, the techniques effectively take advantage of data that would otherwise be considered irrelevant to the task—allowing the user to train more performant models while requiring less training data per dataset.

Abstract: Systems and methods are disclosed for monitoring anatomic position of a human subject and modifying a radiotherapy treatment based on anatomic position changes, as determined with a regression model trained to estimate movement of a region of interest. Example operations for movement monitoring and therapy control include: obtaining 3D image data for a subject, which provides a reference volume and at least one defined region of interest; obtaining real-time 2D image data corresponding to the subject, captured during the radiotherapy treatment session; extracting features from the 2D image data; producing a relative motion estimation of a region of interest with a machine learning regression model, the model trained to estimate a spatial transformation from the 2D image data based on training from the reference volume; and controlling a radiotherapy beam of a radiotherapy machine used in the radiotherapy session, based on the relative motion estimation.

Abstract: Systems and methods may be used for estimating instantaneous patient motion (a patient state). The patient state may be estimated based on a 3D reference volume and a stream of images, for example from an image acquisition device. The stream of images may be received in real-time, for example during a radiation therapy treatment. An example method may include encoding the 3D reference volume using a 3D encoder branch of a patient state generator network, encoding the stream of images using a 2D encoder branch of the patient state generator network, and combining the encoded 3D reference volume and the encoded real-time stream of images. The method may include estimating a 3D spatial transform that maps the 3D reference volume to a current patient state by decoding the combined encoding using a 3D decoder branch of the patient state generator network.

Abstract: Systems and methods are disclosed for performing operations comprising: receiving first and second images depicting an anatomy of a subject; obtaining a segmentation associated with the first image; applying a trained neural network to estimate the adapted segmentation corresponding to the anatomy depicted in the second image, the trained network consisting of three sub-networks: a registration sub-network, generating an initial segmentation estimate representing a deformation of the segmentation associated with the first image to fit the anatomy depicted in the second image, a segmentation sub-network, generating a second initial segmentation estimate for the second image, and a third refinement sub-network, combining the two initial segmentations and generating a refined segmentation for the second image.

Abstract: Systems and methods are disclosed for performing operations comprising: receiving a plurality of training images representing different phases of a periodic motion of a target region in a patient; applying a model to the plurality of training images to generate a lower-dimensional feature space representation of the plurality of training images; clustering the lower-dimensional feature space representation of the plurality of training images into a plurality of groups corresponding to the different phases of the periodic motion; and classifying a motion phase associated with a new image of the target region in the patient based on the plurality of groups of the clustered lower-dimensional feature space representation of the plurality of training images.

Abstract: Systems and methods are disclosed for monitoring anatomic position of a human subject and modifying a radiotherapy treatment based on anatomic position changes, as determined with a regression model trained to estimate movement of a region of interest. Example operations for movement monitoring and therapy control include: obtaining 3D image data for a subject, which provides a reference volume and at least one defined region of interest; obtaining real-time 2D image data corresponding to the subject, captured during the radiotherapy treatment session; extracting features from the 2D image data; producing a relative motion estimation of a region of interest with a machine learning regression model, the model trained to estimate a spatial transformation from the 2D image data based on training from the reference volume; and controlling a radiotherapy beam of a radiotherapy machine used in the radiotherapy session, based on the relative motion estimation.

Abstract: Systems and methods are disclosed for monitoring anatomic position of a human subject for a radiotherapy treatment session, based on use of a regression model trained to estimate movement of a region of interest based on 2D image data input. Example operations for movement estimation include: obtaining 3D image data for a subject, which provides a reference volume and at least one defined region of interest; obtaining 2D image data corresponding to the subject, captured in real time (during the radiotherapy treatment session); extracting features from the 2D image data; analyzing the extracted features with a machine learning regression model, trained to estimate a spatial transformation in the three dimensions of the reference volume; and outputting and using a relative motion estimation of the at least one region of interest, produced from the machine learning regression model, the relative motion estimation being estimated from the extracted features.

Abstract: Systems and methods are disclosed for performing operations comprising: receiving a current fraction image and a reference image each depicting an anatomy of a subject, the reference image corresponding to a previous radiotherapy treatment fraction, the current fraction image corresponding to a current radiotherapy treatment fraction; obtaining a reference segmentation associated with the reference image; encoding, by a current fraction encoder, the current fraction image to generate a first set of features; encoding, by a reference encoder, the reference image and the reference segmentation to generate a second set of features; decoding, by a decoder, a combination of the first set of features and the second set of features to generate a predicted segmentation of the current fraction image; and configuring a radiotherapy treatment parameter based on the predicted segmentation of the current fraction image.

Abstract: Systems and methods are disclosed for performing operations comprising: receiving first and second images depicting an anatomy of a subject; obtaining a segmentation associated with the first image; applying a trained neural network to estimate the adapted segmentation corresponding to the anatomy depicted in the second image, the trained network consisting of two sub-networks: a registration sub-network, generating an initial segmentation estimate representing a deformation of the segmentation associated with the first image to fit the anatomy depicted in the second image, and a segmentation refinement sub-network, predicting a refined segmentation for the second image given the initial segmentation estimate.

Abstract: Systems and methods are disclosed for performing operations comprising: receiving first and second images depicting an anatomy of a subject; obtaining a segmentation associated with the first image; applying a trained neural network to estimate the adapted segmentation corresponding to the anatomy depicted in the second image, the trained network consisting of three sub-networks: a registration sub-network, generating an initial segmentation estimate representing a deformation of the segmentation associated with the first image to fit the anatomy depicted in the second image, a segmentation sub-network, generating a second initial segmentation estimate for the second image, and a third refinement sub-network, combining the two initial segmentations and generating a refined segmentation for the second image.