Abstract: A system is provided to classify the gaze direction of an individual obseng a number of surrounding objects. The system utilizes a qualitative approach in which frequently occurring head poses of the individual are automatically identified and labelled according to their association with the surrounding objects. In conjunction with processing of eye pose, this enables the classification of gaze direction. In one embodiment, each observed head pose of the individual is automatically associated with a bin in a "pose-space histogram". This histogram records the frequency of different head poses over an extended period of time. Given observations of a car driver, for example, the pose-space histogram develops peaks over time corresponding to the frequently viewed directions of toward the dashboard, toward the mirrors, toward the side window, and straight-ahead.