Abstract: Techniques are described for generating multimodal training data cohorts tailored to specific clinical machine learning (ML) model inferencing tasks. In an embodiment, a method comprises accessing, by a system comprising a processor, multimodal clinical data for a plurality of subjects included in one or more clinical data sources. The method further comprises selecting, by the system, datasets from the multimodal clinical data based on the datasets respectively comprising subsets of the multimodal clinical data that satisfy criteria determined to be relevant to a clinical processing task. The method further comprises generating, by the system, a training data cohort comprising the datasets for training a clinical inferencing model to perform the clinical processing task.