Abstract: Techniques are described for an autoregressive user input-to-video (e.g., text-to-video or image-to-video) generation. Using these approaches, a video of unlimited length and without temporal inconsistencies is generated based on the user input. In an implementation, the system receives user input, having particular content, to generate a sequence of video frames, having the same content. The system generates an output sequence of video frames, having the same particular content, by iteratively denoising the frames and conditioning the generation based on the user input. The system may additionally integrate initial anchor frame features with the user input when conditioning the generation of the output sequence of frames. The system may additionally condition each denoising iteration of the output sequence of video frames based on the features of the previous sequence of output video frames.

Abstract: Techniques are described for upscaling low-resolution image data. In an embodiment, input image data of a low-resolution image is received to generate output image data for a high-resolution image of the low-resolution image. The input image data is interpolated to match the output size, and, based on the original input image data, the residual image data is determined. The interpolated image data is combined with the residual image data to generate the output image data for the high-resolution image of the output size. The techniques further include training one or more learning models for determining the residual image data based on the original input image data.

Abstract: Techniques are described for upscaling low-resolution image data. In an embodiment, input image data of a low-resolution image is received to generate output image data for a high-resolution image of the low-resolution image. The input image data is interpolated to match the output size, and, based on the original input image data, the residual image data is determined. The interpolated image data is combined with the residual image data to generate the output image data for the high-resolution image of the output size. The techniques further include training one or more learning models for determining the residual image data based on the original input image data.

Abstract: Aspects presented herein include systems and methods for editing images (still or video images). In embodiments, edit information is captures and associate with an edited image (e.g., a “remix” image). The remix image and its associated edit information may be readily shared with other users. In embodiments, users can see the creator's editing steps used to achieve the end result via interactive “cards” that may be displayed with the remix image. In embodiments, a player application uses the captured edit information to allow users to “replay” some or all of those edits on an image. The remix-replay embodiments provide: (1) unique ways for capturing edits and parameter adjustments for being applied onto a different image; (2) unique ways for observing how the image was edited for learning how to replicate edits; and (3) unique ways for applying some or all of those edits during editing.

Abstract: Aspects presented herein include systems and methods for editing images (still or video images). In embodiments, edit information is captures and associate with an edited image (e.g., a “remix” image). The remix image and its associated edit information may be readily shared with other users. In embodiments, users can see the creator's editing steps used to achieve the end result via interactive “cards” that may be displayed with the remix image. In embodiments, a player application uses the captured edit information to allow users to “replay” some or all of those edits on an image. The remix-replay embodiments provide: (1) unique ways for capturing edits and parameter adjustments for being applied onto a different image; (2) unique ways for observing how the image was edited for learning how to replicate edits; and (3) unique ways for applying some or all of those edits during editing.

Abstract: Techniques are described for inpainting of image data with a missing region. In an embodiment, at each iteration, the process determines a corresponding missing boundary region of the missing region and generates a collection of boundary patches for the missing boundary region. Based on comparing a boundary patch from the collection to source patches from a known source region of image data, the process generates replacement patches for the missing boundary region. When a boundary pixel data unit corresponds to multiple replacement pixel data units from different replacement patches, the process aggregates the multiple replacement pixel data units to generate an updated boundary pixel data unit. In an embodiment, the process performs convolution using the updated and previously known region of the image data.

Abstract: Techniques are described for the inpainting of an image with a missing region. In an embodiment, the process transforms an input image tensor of the image into a spectral tensor of the image. A machine learning-generated filter is applied to the spectral tensor of the image. Additionally or alternatively, a spectral tensor is spectrally split into at least high-frequency and low-frequency sub-tensors. The output tensor(s) are then inverse-transformed into the spatial domain, yielding pixel data for the unknown region that accurately regenerates both low-level detail information as well as high-level detail information.

Abstract: Techniques are described for adapting the format of streaming media based on received interactions with the media. In an embodiment, a client computer system receives from media streaming service an initial version of media in an initial media format. After the media is presented, a request to interact with the initial version of the media may be received. Such a request may cause the client system to generate a change request data for the initial version of media. Based at least in part on the change request data for the initial version of the media, the new format for streaming the media to the client computer system is determined. The client computer system receives the media streamed in the new format.

Abstract: Techniques are described for the inpainting of an image with a missing region. In an embodiment, the process transforms an input image tensor of the image into a spectral tensor of the image. A machine learning-generated filter is applied to the spectral tensor of the image. Additionally or alternatively, a spectral tensor is spectrally split into at least high-frequency and low-frequency sub-tensors. The output tensor(s) are then inverse-transformed into the spatial domain, yielding pixel data for the unknown region that accurately regenerates both low-level detail information as well as high-level detail information.

Abstract: Techniques are described for inpainting of image data with a missing region. In an embodiment, at each iteration, the process determines a corresponding missing boundary region of the missing region and generates a collection of boundary patches for the missing boundary region. Based on comparing a boundary patch from the collection to source patches from a known source region of image data, the process generates replacement patches for the missing boundary region. When a boundary pixel data unit corresponds to multiple replacement pixel data units from different replacement patches, the process aggregates the multiple replacement pixel data units to generate an updated boundary pixel data unit. In an embodiment, the process performs convolution using the updated and previously known region of the image data.

Abstract: Aspects presented herein include systems and methods for editing images (still or video images). In embodiments, edit information is captures and associate with an edited image (e.g., a “remix” image). The remix image and its associated edit information may be readily shared with other users. In embodiments, users can see the creator's editing steps used to achieve the end result via interactive “cards” that may be displayed with the remix image. In embodiments, a player application uses the captured edit information to allow users to “replay” some or all of those edits on an image. The remix-replay embodiments provide: (1) unique ways for capturing edits and parameter adjustments for being applied onto a different image; (2) unique ways for observing how the image was edited for learning how to replicate edits; and (3) unique ways for applying some or all of those edits during editing.

Abstract: Techniques are described for inpainting of image data with a missing region. In an embodiment, at each iteration, the process determines a corresponding missing boundary region of the missing region and generates a collection of boundary patches for the missing boundary region. Based on comparing a boundary patch from the collection to source patches from a known source region of image data, the process generates replacement patches for the missing boundary region. When a boundary pixel data unit corresponds to multiple replacement pixel data units from different replacement patches, the process aggregates the multiple replacement pixel data units to generate an updated boundary pixel data unit. In an embodiment, the process performs convolution using the updated and previously known region of the image data.

Abstract: Techniques are described for inpainting of image data with a missing region. In an embodiment, at each iteration, the process determines a corresponding missing boundary region of the missing region and generates a collection of boundary patches for the missing boundary region. Based on comparing a boundary patch from the collection to source patches from a known source region of image data, the process generates replacement patches for the missing boundary region. When a boundary pixel data unit corresponds to multiple replacement pixel data units from different replacement patches, the process aggregates the multiple replacement pixel data units to generate an updated boundary pixel data unit. In an embodiment, the process performs convolution using the updated and previously known region of the image data.

Abstract: Aspects presented herein include systems and methods for editing images (still or video images). In embodiments, edit information is captures and associate with an edited image (e.g., a “remix” image). The remix image and its associated edit information may be readily shared with other users. In embodiments, users can see the creator's editing steps used to achieve the end result via interactive “cards” that may be displayed with the remix image. In embodiments, a player application uses the captured edit information to allow users to “replay” some or all of those edits on an image. The remix-replay embodiments provide: (1) unique ways for capturing edits and parameter adjustments for being applied onto a different image; (2) unique ways for observing how the image was edited for learning how to replicate edits; and (3) unique ways for applying some or all of those edits during editing.

Abstract: Techniques are described for upscaling low-resolution image data. In an embodiment, input image data of a low-resolution image is received to generate output image data for a high-resolution image of the low-resolution image. The input image data is interpolated to match the output size, and, based on the original input image data, the residual image data is determined. The interpolated image data is combined with the residual image data to generate the output image data for the high-resolution image of the output size. The techniques further include training one or more learning models for determining the residual image data based on the original input image data.

Abstract: Aspects presented herein include systems and methods for editing images (still or video images). In embodiments, edit information is captures and associate with an edited image (e.g., a “remix” image). The remix image and its associated edit information may be readily shared with other users. In embodiments, users can see the creator's editing steps used to achieve the end result via interactive “cards” that may be displayed with the remix image. In embodiments, a player application uses the captured edit information to allow users to “replay” some or all of those edits on an image. The remix-replay embodiments provide: (1) unique ways for capturing edits and parameter adjustments for being applied onto a different image; (2) unique ways for observing how the image was edited for learning how to replicate edits; and (3) unique ways for applying some or all of those edits during editing.