Abstract: A data structure representation of an image called an image structure map (ISM) accurately and explicitly represents the geometry, topology and signal properties of regions in an original image and allows for efficient and accurate spatial indexing of the image and those regions. The ISM may serve as the basis for an image interaction system in which a user interacts with a rendered view of an original image that is produced from the ISM data structure. The rendered view of the original image represents the geometry and topology of the original image as represented in the ISM. The user interacts with the rendered view to make alterations to, or to query, the structures in the ISM. The user may also modify the ISM by interacting directly with a displayed view of the original image. The user may further invoke automatic operations to be performed using the original image data structure that produce image region data, such as signal property descriptors and boundary data, that may be used to modify the ISM.