Abstract: A system includes a hardware processor, a memory storing software code, and a machine learning (ML) model-based video frame interpolator. The hardware processor executes the software code to provide first and second frames of a video sequence including a plurality of frames, respective binary masks for the first and second frames, and optionally an intermediate frame of the video sequence between the first and second frames and a binary mask for the intermediate frame, as interpolation inputs to the ML model-based video frame interpolator. The hardware processor further executes the software code to generate, using the ML model-based video frame interpolator and the interpolation inputs, an interpolated frame and an error map for the interpolated frame, wherein generating the interpolated frame and the error map includes a cross-backward warping of respective latent feature representations of each of the plurality of frames.

Abstract: One embodiment of the present invention sets forth a technique for performing frame interpolation. The technique includes generating (i) a first set of feature maps based on a first set of rendering features associated with a first key frame, (ii) a second set of feature maps based on a second set of rendering features associated with a second key frame, and (iii) a third set of feature maps based on a third set of rendering features associated with a target frame. The technique also includes applying one or more neural networks to the first, second, and third set of feature maps to generate a set of mappings from a first set of pixels in the first key frame to a second set of pixels in the target frame. The technique further includes generating the target frame based on the set of mappings.