Abstract: A learning model is trained for rate-distortion behavior prediction against a corpus of a video hosting platform and used to determine optimal bitrate allocations for video data given video content complexity across the corpus of the video hosting platform. Complexity features of the video data are processed using the learning model to determine a rate-distortion cluster prediction for the video data, and transcoding parameters for transcoding the video data are selected based on that prediction. The rate-distortion clusters are modeled during the training of the learning model, such as based on rate-distortion curves of video data of the corpus of the video hosting platform and based on classifications of such video data. This approach minimizes total corpus egress and/or storage while further maintaining uniformity in the delivered quality of videos by the video hosting platform.

Abstract: A training dataset that includes a first dataset and a second dataset is received. The first dataset includes a first subset of first videos corresponding to a first context and respective first ground truth quality scores of the first videos, and the second dataset includes a second subset of second videos corresponding to a second context and respective second ground truth quality scores of the second videos. A machine learning model is trained to predict the respective first ground truth quality scores and the respective second ground truth quality scores. Training the model includes training it to obtain a global quality score for one of the videos; and training it to map the global quality score to context-dependent predicted quality scores. The context-dependent predicted quality scores include a first context-dependent predicted quality score corresponding to the first context and a second context-dependent predicted quality score corresponding to the second context.

Abstract: Processing a spherical video using denoising is described. Video content comprising the spherical video is received. Whether a camera geometry or a map projection, or both, used to generate the spherical video is available is then determined. The spherical video is denoised using a first technique responsive to a determination that the camera geometry, the map projection, or both is available. Otherwise, the spherical video is denoised using a second technique. At least some steps of the second technique can be different from steps of the first technique. The denoised spherical video can be encoded for transmission or storage using less data than encoding the spherical video without denoising.

Abstract: A method includes training a first model to measure the banding artefacts, training a second model to deband the image, and generating a debanded image for the image using the second model. Training the first model can include selecting a first set of first training images, generating a banding edge map for a first training image, where the map includes weights that emphasize banding edges and de-emphasize true edges in the first training image, and using the map and a luminance plane of the first training image as input to the first model. Training the second model can include selecting a second set of second training images, generating a debanded training image for a second training image, generating a banding score for the debanded training image using the first model, and using the banding score in a loss function used in training the second model.

Abstract: Signals of an immersive multimedia item are jointly considered for optimizing the quality of experience for the immersive multimedia item. During encoding, portions of available bitrate are allocated to the signals (e.g., a video signal and an audio signal) according to the overall contribution of those signals to the immersive experience for the immersive multimedia item. For example, in the spatial dimension, multimedia signals are processed to determine spatial regions of the immersive multimedia item to render using greater bitrate allocations, such as based on locations of audio content of interest, video content of interest, or both. In another example, in the temporal dimension, multimedia signals are processed in time intervals to adjust allocations of bitrate between the signals based on the relative importance of such signals during those time intervals. Other techniques for bitrate optimizations for immersive multimedia streaming are also described herein.

Abstract: Methods, systems, and media for determining perceptual quality indicators of video content items are provided.

Abstract: First video frames that include a visual object and a non-spatialized first audio segment that includes an auditory event are received. If that second video frames do not include the visual object and a first time difference between the first video frames and the second video frames does not exceed a certain time, a motion vector of the visual object is used to assign a spatial location to the auditory event in at least one of the second video frames. A second audio segment that includes the auditory event and third video frames are received. If the third video frames do not include the visual object and a second time difference between the first video frames and the third video frames exceeds the certain time, the auditory event is assigned to a diffuse sound field. An audio output that conveys spatial locations of the visual object is output.

Abstract: Denoising video content includes identifying a three-dimensional flat frame block of multiple frames of the video content, wherein the three-dimensional flat frame block comprises flat frame blocks, each flat frame block is located within a respective frame of the multiple frames, and the flat frame blocks have a spatial and temporal intensity variance that is less than a threshold. Denoising video content also includes determining an average intensity value of the three-dimensional flat frame block, determining a noise model that represents noise characteristics of the three-dimensional flat frame block, generating a denoising function using the average intensity value and the noise model, and denoising the multiple frames using the denoising function.

Abstract: Assigning spatial information to audio segments is disclosed. A method includes receiving a first audio segment that is non-spatialized and is associated with first video frames; identifying visual objects in the first video frames; identifying auditory events in the first audio segment; identifying a match between a visual object of the visual objects and an auditory event of the auditory events; and assigning a spatial location to the auditory event based on a location of the visual object.

Abstract: A method includes training a first model to measure the banding artefacts, training a second model to deband the image, and generating a debanded image for the image using the second model. Training the first model can include selecting a first set of first training images, generating a banding edge map for a first training image, where the map includes weights that emphasize banding edges and de-emphasize true edges in the first training image, and using the map and a luminance plane of the first training image as input to the first model. Training the second model can include selecting a second set of second training images, generating a debanded training image for a second training image, generating a banding score for the debanded training image using the first model, and using the banding score in a loss function used in training the second model.

Abstract: Image data is processed for noise reduction before encoding and subsequent decoding. For an input image in a spatial domain, two-dimensional (2-D) wavelet coefficients at multiple levels are generated. Each level includes multiple subbands, each associated with a respective subband type in a wavelet domain. For respective levels, a flat region of a subband is identified, which flat region includes blocks of the subband having a variance no higher than a first threshold variance. A flat block set for the subband type associated with the subband is identified, which includes blocks common to respective flat regions of the subband. A second threshold variance is determined using variances of the flat block set, and is then used for thresholding at least some of the 2-D wavelet coefficients to remove noise. After thresholding, a denoised image is generated in the spatial domain using the levels.

Abstract: Video streams uploaded to a video hosting platform are transcoded using quality-normalized transcoding parameters dynamically selected using a learning model. Video frames of a video stream are processed using the learning model to determine bitrate and quality score pairs for some or all possible transcoding resolutions. The listing of bitrate and quality score pairs determined for each resolution is processed to determine a set of transcoding parameters for transcoding the video stream into each resolution. The bitrate and quality score pairs of a given listing may be processed using one or more predefined thresholds, which may, in some cases, refer to a weighted distribution of resolutions according to watch times of videos of the video hosting platform. The video stream is then transcoded into the various resolutions using the set of transcoding parameters selected for each resolution.

Abstract: A learning model is trained for rate-distortion behavior prediction against a corpus of a video hosting platform and used to determine optimal bitrate allocations for video data given video content complexity across the corpus of the video hosting platform. Complexity features of the video data are processed using the learning model to determine a rate-distortion cluster prediction for the video data, and transcoding parameters for transcoding the video data are selected based on that prediction. The rate-distortion clusters are modeled during the training of the learning model, such as based on rate-distortion curves of video data of the corpus of the video hosting platform and based on classifications of such video data. This approach minimizes total corpus egress and/or storage while further maintaining uniformity in the delivered quality of videos by the video hosting platform.

Abstract: A trained model is retrained for video quality assessment and used to identify sets of adaptive compression parameters for transcoding user generated video content. Using transfer learning, the model, which is initially trained for image object detection, is retrained for technical content assessment and then again retrained for video quality assessment. The model is then deployed into a transcoding pipeline and used for transcoding an input video stream of user generated content. The transcoding pipeline may be structured in one of several ways. In one example, a secondary pathway for video content analysis using the model is introduced into the pipeline, which does not interfere with the ultimate output of the transcoding should there be a network or other issue. In another example, the model is introduced as a library within the existing pipeline, which would maintain a single pathway, but ultimately is not expected to introduce significant latency.

Abstract: Denoising video content includes identifying a three-dimensional flat frame block of multiple frames of the video content, wherein the three-dimensional flat frame block comprises flat frame blocks, each flat frame block is located within a respective frame of the multiple frames, and the flat frame blocks have a spatial and temporal intensity variance that is less than a threshold. Denoising video content also includes determining an average intensity value of the three-dimensional flat frame block, determining a noise model that represents noise characteristics of the three-dimensional flat frame block, generating a denoising function using the average intensity value and the noise model, and denoising the multiple frames using the denoising function.

Abstract: Processing a spherical video using denoising is described. Video content comprising the spherical video is received. Whether a camera geometry or a map projection, or both, used to generate the spherical video is available is then determined. The spherical video is denoised using a first technique responsive to a determination that the camera geometry, the map projection, or both is available. Otherwise, the spherical video is denoised using a second technique. At least some steps of the second technique can be different from steps of the first technique. The denoised spherical video can be encoded for transmission or storage using less data than encoding the spherical video without denoising.

Abstract: A method for denoising video content includes identifying a first frame block associated with a first frame of the video content. The method also includes estimating a first noise model that represents characteristics of the first frame block. The method also includes identifying at least one frame block adjacent to the first frame block. The method also includes generating a second noise model that represents characteristics of the at least one frame block adjacent to the first frame block by adjusting the first noise model based on at least one characteristic of the at least one frame block adjacent to the first frame block. The method also includes denoising the at least one frame block adjacent to the first frame block using the second noise model.

Abstract: A method for denoising video content includes identifying a first frame block of a plurality of frame blocks associated with a first frame of the video content. The method also includes determining an average intensity value for the first frame block. The method also includes determining a first noise model that represents characteristics of the first frame block. The method also includes generating a denoising function using the average intensity value and the first noise model for the first frame block. The method further includes denoising the plurality of frame blocks using the denoising function.

Abstract: Adaptive filtering is used video stream for bitrate reduction. A first copy of the input video stream is encoded to a reference bitstream. Each of a number of candidate filters is applied to each frame of a second copy of the input video stream to produce a filtered second copy of the input video stream. The filtered second copy is encoded to a candidate bitstream. A cost value for the candidate filter is determined based on distortion value and bitrate differences between the candidate bitstream and the reference bitstream. The candidate bitstream corresponding to the candidate filter with a lowest one of the cost values is selected as the output bitstream, which is then output or stored. Processing the input video stream using the adaptive filter and before the encoding may result in bitrate reduction, thereby improving compression, decompression, and other performance.

Abstract: Signals of an immersive multimedia item are jointly considered for optimizing the quality of experience for the immersive multimedia item. During encoding, portions of available bitrate are allocated to the signals (e.g., a video signal and an audio signal) according to the overall contribution of those signals to the immersive experience for the immersive multimedia item. For example, in the spatial dimension, multimedia signals are processed to determine spatial regions of the immersive multimedia item to render using greater bitrate allocations, such as based on locations of audio content of interest, video content of interest, or both. In another example, in the temporal dimension, multimedia signals are processed in time intervals to adjust allocations of bitrate between the signals based on the relative importance of such signals during those time intervals. Other techniques for bitrate optimizations for immersive multimedia streaming are also described herein.