Abstract: Methods, systems, and computer readable media for rendering context-aware and interactive artificial intelligence-generated videos in real time. A talking face (“TF”) model may traverse from a first node to a second node of a state graph via an edge based on a TF instruction generated by an interaction model. The TF model may retrieve a transition video associated with the edge and a pre-computed video template associated with the second node from one or more TF databases. The pre-computed video template may include a plurality of masked video frames and a plurality of pre-computed mouth positions for each masked video frame. The TF model may inpaint a pre-computed mouth position into a masked region of each masked video frame to form a video frame stream. The interaction model may generate a video from the transition video and the video frame stream and present the video on a user device.

Abstract: Methods, systems, and computer readable media for rendering context-aware and interactive artificial intelligence-generated videos in real time. A talking face (“TF”) model may traverse from a first node to a second node of a state graph via an edge based on a TF instruction generated by an interaction model. The TF model may retrieve a transition video associated with the edge and a pre-computed video template associated with the second node from one or more TF databases. The pre-computed video template may include a plurality of masked video frames and a plurality of pre-computed mouth positions for each masked video frame. The TF model may inpaint a pre-computed mouth position into a masked region of each masked video frame to form a video frame stream. The interaction model may generate a video from the transition video and the video frame stream and present the video on a user device.