Abstract: Content transparency control implemented on a central processing unit, or a graphics processing unit to post-process live video content requires direct access to double data rate memory and can demand significant amounts of power. A solution addresses this issue by exploiting temporal noise reduction hardware and software already included in an image signal processor. The temporal noise reduction hardware and software are modified to support content transparency control in tandem with temporal noise reduction. Specifically, the feedback weight map and the output weight map used in temporal noise reduction to blend an input image and a retrieved reference image can be controlled based on a semantic map to effectuate content transparency control.

Abstract: Systems and methods for improving a temporal noise reducer (TNR) architecture that improves TNR performance and IQ. Temporal noise reduction is a core feature of a video processing pipeline, where TNR can be used to decrease noise in video streams. TNRs generally includes two main steps: motion analysis and blending. Motion analysis includes identifying moving elements, and can include generating a motion map indicating regions of the input image that are static versus regions with movement. Blending includes blending the current input image frame with the previous temporally-denoised frame. An architecture is provided that separates the motion analysis from the blending step. In particular, the architecture includes a motion analysis block that operates on the raw image at the start of the pipeline, while the blending operation is completed on the processed image at the end of the image processing pipeline.

Abstract: Framing a video effectively and efficiently in video conferencing, video recording, and live streaming can be a challenge. The framing decision preferably captures users and their actions without degrading the user experience. To address this challenge, a framing pipeline can be implemented to include a plurality of detectors, a multi-detector fusion, and a motion planner. Multi-detector fusion can merge detections produced by different detectors at different rates. Motion planner can change the framing decision based on a merged region of interest and a target region of interest produced by multi-detector fusion. The framing pipeline includes one or more features that can achieve a robust, temporally stable, and visually pleasing framed video frames that smoothly tracks users over time without causing unpleasant artifacts in the video conferencing experience.