Abstract: An augmented reality system for displaying a medical image (in particular an image stream) and a respective method are disclosed. The augmented reality system includes an optical head mounted display, a medical imaging device, a tracking system that is adapted to measure data concerning a position and orientation of at least one object, and a processing unit. The processing unit is configured to transform an image from a coordinate system in which the image was taken to a coordinate system of the display. This transformation is performed using data measured by the tracking system, where the ARS is adapted to display the transformed image on the display in a position, orientation and scale that corresponds to the perspective of a position and orientation of the bearer of the display.