Abstract: Techniques for content-adaptive encoder configuration are described herein. In accordance with various embodiments, a device (e.g., a content-based toolset configurator) including a processor and a non-transitory memory receives one or more frames in a media stream and a performance target of an encoder. The content-based toolset configurator performs cycles of pre-analysis of the one or more frames to generate content features within the performance target, assigns a content class to the one or more frames based on the content features, a previous classification, and the performance target, and sets configurations of the encoder for encoding the one or more frames corresponding to the content class and the performance target.

Abstract: Techniques for content-adaptive encoder configuration are described herein. In accordance with various embodiments, a device (e.g., a content-based toolset configurator) including a processor and a non-transitory memory receives one or more frames in a media stream and a performance target of an encoder. The content-based toolset configurator performs cycles of pre-analysis of the one or more frames to generate content features within the performance target, assigns a content class to the one or more frames based on the content features, a previous classification, and the performance target, and sets configurations of the encoder for encoding the one or more frames corresponding to the content class and the performance target.

Abstract: Techniques for machine learning based coded size estimation in rate control of video encoding are described. An encoder in accordance with various embodiments uses one or more machine learning approaches to facilitate the rate control of video encoding. When training one or more neural network models, the relationship between the coded size and unit characteristics (e.g., picture pixels and/or picture features) is learned from past encoding. The encoder then uses the trained model(s) to estimate the coded size through model inference with improved accuracy. In some embodiments, the trained model(s) are integrated into the encoder for direct model inference. The direct model inference reduces the overhead of referencing application programming interfaces (APIs) provided by a separate machine learning platform, thus making the rate control methods and systems described herein useful in real time video encoding.