Abstract: Image generation using a hierarchical, model-based system includes generating a first region of an image using a first neural network model. The first region of the image is provided to a second neural network model as input. A second region of the image is generated using the second neural network model. The second region of the image shares a boundary with at least a portion of the first region of the image.

Abstract: Synthesizing high-resolution data includes distorting, with a distortion function, a region of interest (ROI) within an input of inferential data. The distorting generates distortion data within which the ROI is enhanced relative to other regions of the distortion data. A generative artificial intelligence (AI) model generates synthetic data in response to input of the distortion data. The generative AI model is trained against a distorted ground truth generated using the distortion function to distort one or more regions of interest ROI within source data used to guide the generative AI model in generating the synthetic data.

Abstract: Synthesizing high-resolution input for rendering a digital human includes generating, with a generative artificial intelligence (AI) model, a distorted image of the digital human by enhancing a region of interest (ROI) within the distorted image relative to other regions of the distorted image. The generative AI model is previously trained against a distorted control image generated using a distortion function to distort a control image used to guide image generation by the generative AI model. The distorted control image is generated by reconfiguring and augmenting pixels of the control image. An undistorted image of the digital human is generated using a reverse distortion function to reverse distortion of the distorted image.

Abstract: Autoregressive content rendering for temporally coherent video generation includes generating, by an autoencoder, a plurality of predicted images. The plurality of predicted images is fed back to the autoencoder network. The plurality of predicted images may be encoded by the autoencoder network to generate a plurality of encoded predicted images. The autoencoder network encodes a plurality of keypoint images to generate a plurality of encoded keypoint images. One or more predicted images of the plurality of predicted images are generated by the autoencoder network by decoding a selected encoded keypoint image of the plurality of encoded keypoint images with an encoded predicted image of the plurality of encoded predicted images of a prior iteration of the autoencoder network.

Abstract: Generating images includes generating encoded data by encoding input data into a latent space. The encoded data is decoded through a first decoder having first decoder layers by processing the encoded data through one or more of the first decoder layers. The encoded data is decoded through a second decoder having second decoder layers by processing the encoded data through one or more of the second decoder layers. An updated feature map is generated by replacing at least a portion of a feature map output from a selected layer of the first decoder layers with at least a portion of a feature map output from a selected layer of the second decoder layers. An image is generated by further decoding the updated feature map through one or more additional layers of the first decoder layers.

Abstract: Synthesizing speech and movement of a virtual human includes capturing supplemental data generated by a transducer. The supplemental data specifies one or more attributes of a user. The capturing is performed in substantially real-time with the user providing input to a conversational platform. A behavior determiner generates behavioral data based on the supplemental data and an audio response generated by the conversational platform in response to the input to the conversation platform. Based on the behavioral data and the audio response, a rendering network generates a video rendering of a virtual human engaging in a conversation with the user, the video rendering synchronized with the audio response.

Abstract: Image-to-image translations using 1D inputs includes concatenating multiple 1D vectors forming a concatenated 1D vector. The multiplicity of 1D vectors includes 1D vectors of at least two different modalities. An encoded 1D vector is generated by encoding the concatenated 1D vector. An encoded 2D array of features is generated by reshaping an arrangement of features of the encoded 1D feature vector. An image of a virtual human is generated by decoding the encoded 2D array.

Abstract: Multimodal disentanglement can include generating a set of silhouette images corresponding to a human face, the generating undoing a correlation between an upper portion and a lower portion of the human face depicted by each silhouette image. A unimodal machine learning model can be trained with the set of silhouette images. As trained, the unimodal machine learning model can generate synthetic images of the human face. The synthetic images generated by the unimodal machine learning model once trained can be used to train a multimodal rendering network. The multimodal rendering network can be trained to generate a voice-animated digital human. Training the multimodal rendering network can be based on minimizing differences between the synthetic images and images generated by the multimodal rendering network.

Abstract: Creating images and animations of lip motion from mouth shape data includes providing, as one or more input features to a neural network model, a vector of a plurality of coefficients. Each vector of the plurality of coefficients corresponds to a different mouth shape. Using the neural network model, a data structure output specifying a visual representation of a mouth including lips having a shape corresponding to the vector is generated.

Abstract: Image generation using a hierarchical, model-based system includes generating a first region of an image using a first neural network model. The first region of the image is provided to a second neural network model as input. A second region of the image is generated using the second neural network model. The second region of the image shares a boundary with at least a portion of the first region of the image.