Abstract: Techniques for shared memory spaces in data and model parallelism are provided to improve memory efficiency and memory access speed. A shared memory space may be established at a host system or in a hardware memory agent. The shared memory may store training data or model parameters for an artificial intelligence model at a memory address in one or more memory circuits. Data for the artificial intelligence model may be processed across a plurality of artificial intelligence accelerators using the training data or the model parameters of the shared memory space. That is, multiple accelerators access one copy of the data from the shared memory space instead of accessing their own separate memory space.