Abstract: A processing server assists in identifying user consumption of information. The processing server includes a hardware processor, and memory storing computer instructions that when executed perform receiving image data comprising a series of screen image snapshots being presented on a screen of a client device, the screen image snapshots comprising respective timestamps; receiving audio data captured by a microphone of the client device; receiving metadata comprising image metadata, audio metadata, and client device metadata; determining one or more first matches to the image data; determining one or more second matches to the audio data; and aggregating the one or more first matches and the one or more second matches based on the metadata.

Abstract: A client device assists in identifying user consumption of information. The client device comprises a hardware processor; a screen; memory storing computer instructions that when executed perform capturing a series of screen image snapshots being presented on the screen; reducing resolution of each screen image snapshot in the series of screen image snapshots; capturing metadata associated with each screen image snapshot in the series of screen image snapshots, the metadata at least including a timestamp; identifying a duplicate in the series of screen image snapshots; discarding the duplicate from the series of screen image snapshots; and uploading the series of captured screen image snapshots to a processing server for processing.