Abstract: A Media Analytics Co-optimizer (MAC) engine that utilizes available motion and scene information to increase the activation sparsity in artificial intelligence (AI) visual media applications. In an example, the MAC engine receives video frames and associated video characteristics determined by a video decoder and reformats the video frames by applying a threshold level of motion to the video frames and zeroing out areas that fall below the threshold level of motion. In some examples, the MAC engine further receives scene information from an optical flow engine or event processing engine and reformats further based thereon. The reformatted video frames are consumed by the first stage of AI inference.

Abstract: In one embodiment, a compute device includes interface circuitry and processing circuitry. The processing circuitry receives, via the interface circuitry, a current frame of a video stream to be encoded. The processing circuitry then determines whether a scene change occurs at the current frame. If a scene change occurs at the current frame, the processing circuitry detects the scene in the current frame by performing pixel segmentation on the current frame. If a scene change does not occur at the current frame, the processing circuitry detects the scene in the current frame by performing motion estimation on the current frame relative to a previous frame in which the scene was detected. Based on the scene detected in the current frame, the processing circuitry then generates one or more encoding parameters and provides those parameters to a video encoder to encode the current frame.