Abstract: A method and system for visually annotating an image. Annotations and notes to images, such as digital medical and healthcare images, may be stored in a structured vector representation alongside image information in a single, non-volatile and portable file or in a separate file from the image. The annotations may be composed of point, line and polygon drawings and text symbols, labels or definitions and captions or descriptions. The annotations may be structured in a manner that facilitates grouping and manipulation as user defined groups. The annotations may be related to an image but not inextricably bound such that the original image is completely preserved. Annotations may further be selectively displayed on the image for context appropriate viewing. The annotations may be retrieved for purposes such as editing, printing, display, indexing and reporting for example, and may be displayed on an image for interactive use with an embedded self-contained user interface.