Abstract: Techniques are disclosed herein for generating an image. The techniques include performing one or more first denoising operations based on a first machine learning model and an input image that includes a first object to generate a mask that indicates a spatial arrangement associated with a second object interacting with the first object, and performing one or more second denoising operations based on a second machine learning model, the input image, and the mask to generate an image of the second object interacting with the first object.

Abstract: Computer vision systems and methods for compositional pixel prediction are provided. The system receives an input image frame having a plurality of entities where each entity has a location at a first time step. The system processes the input image frame to extract a representation of each entity. The system utilizes an entity predictor to determine a predicted representation of each extracted entity representation at a next time step based on each extracted entity representation and a latent variable and utilizes a frame decoder to generate a predicted frame based on the input image frame and the predicted entity representations. The system trains an encoder to predict a distribution over the latent variable based on the input image frame and a final frame of a ground truth video associated with the input image frame.