Abstract: Video processing, encoding and decoding systems are described. A processor receives video frames representative of a sequence of images captured by a video sensor and the video frames are encode according to a desired video encoding standard. A video analytics processor receives video analytics metadata generated by the video encoder from the sequence of images and produces video analytics messages for transmission to a client device which performs client side video analytics processing. The video analytics metadata may comprise pixel domain video analytics information directly from an analog-to-digital front end or directly from an encoding engine as the engine is performing compression.