Abstract: A system comprises a memory that stores computer executable components; and a processor that executes the computer executable components stored in the memory. A communication component receives a stream of medical images from a medical image data source via a network for displaying in one or more viewports of a medical image visualization application that provides interactive functionalities in association with the displaying. A buffer component stores the medical images, in one or more buffers, as the respective medical images are received. A monitoring component monitors user activity with respect to a rendered subset of the medical images in the one or more viewports in association with usage of the interactive functionalities. A buffer management component regulates storing of the medical images in the one or more buffers by the buffer component as a function of the user activity to facilitate seamless rendering of the medical images.