Abstract: A system that performs emotion detection, prediction, and coaching system is disclosed. The system includes a media feature extractor, which extracts features from different media modalities, a perceiver module which detects or predicts emotional response for a given audience, a coaching module that generates context-based, personalized coaching in the form of commentary, retrospective analysis, tips kudos, events, and scores. The intermediate results, like media feature, perceiver output, and context, are stored in an emotion association database, which can be used as reference data by the system.