Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting engagement. One of the methods is a method of determining an engagement score for each of a plurality of participants in a video conference call, and includes, for each of the plurality of participants: receiving an encoded video stream of the participant, the encoded video stream representing an encoding of a sequence of one or more video frames; decoding the encoded video stream to determine the sequence of video frames; for each of a plurality of engagement categories representing respective different modes by which the participant may be engaged in the conference call, processing one or more respective video frames from the sequence of video frames to generate a sub-score for the engagement category; and combining the sub-scores for the plurality of engagement categories to generate the engagement score for the participant.