Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like that performs inference operations on an individual AI model using dynamically selected parallelism, where the parallelism selection can be based on one or more computational workload conditions associated with the individual AI model. For example, the AI model system of some example embodiments switches between two or more different parallelism modes of operation with respect to an individual AI model based on one or more inference request patterns or based on one or more batch characteristics of a set of inference requests received by the AI model system.