Abstract: A cardiac view of a medical ultrasound image is automatically identified. By grouping different views into sub-categories, a hierarchal classifier identifies the views. For example, apical views are distinguished from parasternal views. Specific types of apical or parasternal views are identified based on distinguishing between images of the geneses. Different features are used for classifying, such as gradients, functions of the gradients, statistics of an average frame of data from a clip or sequence of frames, or a number of edges along a given direction. The number of features used may be compressed, such as by classifying a plurality of features into a new feature. For example, alpha weights in a model of features and classes are determined and used as features for classification.