Abstract: Embodiments are directed towards displaying an animated video emoticon by augmenting features identified in a video stream. Augmenting features identified in the video stream may include modifying, in whole or in part, some aspects of the identified features but not other aspects. For example, a user may select an animated video emoticon indicating surprise. Surprise may be conveyed by detecting the location of the user's eyes in the video stream, enlarging a size aspect of the eyes so as to appear ‘wide-eyed’, but leaving other aspects such as color and shape unchanged. Then, the location and/or orientation of the eyes in the video stream are tracked, and the augmentation is applied to the eyes at each tracked location and/or orientation. In another embodiment, identified features may be removed from the video stream and replaced with images, graphics, video, and the like.