Abstract: Systems and methods for performing a medical imaging analysis task using a foundation model are provided. One or more 3D (three-dimensional) medical images each comprising a plurality of 2D (two-dimensional) slices are received. A first set of features is extracted from the plurality of 2D slices of the one or more 3D medical images using a machine learning based encoding network. Each respective feature of the first set of features is resampled based on a spatial location of one or more pixels of the 2D slices from which the respective feature was extracted. The resampled first set of features is encoded into a second set of features. A medical imaging analysis task is performed based on the second set of features. Results of the medical imaging analysis task are output.

Abstract: Systems and methods for performing a medical imaging analysis task using a universal foundation model are provided. 1) one or more input medical images each in a domain and 2) a domain code for each of the one or more input medical images identifying its domain are provided. For each respective one of the domain codes, one or more weights are determined based on the respective domain code. One or more parameters of a dynamic convolutional layer are updated based on the one or more weights. A first set of features is extracted from the one or more input medical images using the dynamic convolutional layer with the one or more updated parameters. The first set of features are encoded into a second set of features using a machine learning based encoder. A medical imaging analysis task is performed based on the second set of features. Results of the medical imaging analysis task are output.

Abstract: Systems and methods for performing a medical imaging analysis task based on pixelwise positionally encoded features are provided. One or more input medical images are received. One or more pixelwise positional embedding images are generated for the one or more input medical images using a spatially varying function. Patches are extracted from the one or more input medical images and the one or more pixelwise positional embedding images. The patches extracted from the one or more input medical images are encoded with corresponding ones of the patches extracted from the one or more pixelwise positional embedding images into pixelwise positionally encoded features. A medical imaging analysis task is performed using a machine learning based network based on the pixelwise positionally encoded features. Results of the medical imaging analysis task are output.

Abstract: Systems and methods for performing a medical imaging analysis task are provided. 1) a plurality of medical images acquired at a plurality of acquisition orientations and in one or more domains and 2) a domain code for each particular acquisition orientation of the plurality of acquisition orientations are received. Each of the domain codes identify a presence of the one or more domains of the plurality of medical images that were acquired at the particular acquisition orientation. For each of the particular acquisition orientations, the domain code for the particular acquisition orientation are encoded, features are extracted from the plurality of medical images that were acquired at the particular acquisition orientation, and the encoded domain code and the extracted features are combined to generate image features for the particular acquisition orientation. A medical imaging analysis task is performed based on the image features for each of the particular acquisition orientations.

Abstract: Systems and methods for performing a medical imaging analysis task based on pixelwise positionally encoded features are provided. One or more input medical images are received. One or more pixelwise positional embedding images are generated for the one or more input medical images using a spatially varying function. Patches are extracted from the one or more input medical images and the one or more pixelwise positional embedding images. The patches extracted from the one or more input medical images are encoded with corresponding ones of the patches extracted from the one or more pixelwise positional embedding images into pixelwise positionally encoded features. A medical imaging analysis task is performed using a machine learning based network based on the pixelwise positionally encoded features. Results of the medical imaging analysis task are output.