Abstract: There are provided a method, an apparatus, a device, and a computer-readable storage medium for generating skeleton data. The method proposed includes: sampling, based on shape data of an object, a plurality of surface points of the object; generating a set of shape units by encoding the plurality of surface points; providing the set of shape units to a model to generate a set of skeleton units; and generating skeleton data of the object based on the set of skeleton units, the skeleton data indicating a plurality of joint points of the object and a connection relationship between the plurality of joint points.

Abstract: Systems and methods are disclosed for selecting target objects within digital images utilizing a multi-modal object selection neural network trained to accommodate multiple input modalities. In particular, in one or more embodiments, the disclosed systems and methods generate a trained neural network based on training digital images and training indicators corresponding to various input modalities. Moreover, one or more embodiments of the disclosed systems and methods utilize a trained neural network and iterative user inputs corresponding to different input modalities to select target objects in digital images. Specifically, the disclosed systems and methods can transform user inputs into distance maps that can be utilized in conjunction with color channels and a trained neural network to identify pixels that reflect the target object.

Abstract: Techniques for creating high-quality and animatable three-dimensional (3D) avatars are provided. A 3D human representation with a pre-defined pose is generated in a canonical space by adopting a coarse-to-fine pipeline. The 3D human representation corresponds to a 3D avatar to be created based on input text. The 3D human representation is deformed to a target pose defined by a Skinned Multi-Person Linear (SMPL) parameter in a deformed space. The 3D avatar is created by implementing iterative optimization of the 3D human representation in the canonical space and the deformed space by applying a Score Distillation Sampling (SDS) loss based on the input text and information indicative of poses.

Abstract: According to an embodiment of the disclosure, a method, apparatus, device and storage medium for training an image generation model is provided. The method includes: obtaining a reference image and a target image; providing, to an image generation model, the reference image and pose information corresponding to the target image to generate an intermediate image, the pose information describing a pose of a target object in the target image; determining a first region in the intermediate image corresponding to a predetermined part of the target object; and training the image generation model based at least on a difference between the first region and a second region in the target image corresponding to the predetermined part.

Abstract: Methods of customizing generation of objects using diffusion models are provided. One or more parameters (e.g., a conditioning signal, network weights, or an initial or starting noise) of the diffusion model can be optimized by a backpropagation process, which can be performed by solving an augmented adjoint ordinary differential equation (ODE) based on an adjoint sensitivity method. The customized diffusion model can generate stylized objects, generate objects with specific visual effect(s), and provide adversary examples to audit security of an object generation system.

Abstract: The present disclosure describes techniques for implementing drag-based image editing. Feature maps are generated based on latent representations of an image by a first sub-model of a machine learning model. The first sub-model is configured to preserve an identity of the image. Embeddings corresponding to at least one pair of points are generated by a second sub-model of the machine learning model. Each pair of points comprises a handle point and a target point. The handle point identifies an area of the image. The target point indicates a target location to which the area is to be relocated. The feature maps and the embeddings are injected into a third sub-model of the machine learning model to guide a process of generating a target image by the third sub-model. The target image depicts the area of the image relocated at the target location.

Abstract: Generating an object using a diffusion model includes obtaining a first input and a second input, and synthesizing an output object from the first input and the second input. The synthesizing of the output object includes generating a layout of the output object from the first input, injecting the second input as a content conditioner to the layout of the output object, and de-noising the layout of the output object injected with the content conditioner to generate a content of the output object.

Abstract: Generating a multi-dimensional video using a multi-dimensional video generative model for, including, but not limited to, at least one of static portrait animation, video reconstruction, or motion editing. The method including providing data into the multi-dimensionally aware generator of the multi-dimensional video generative model, and generating the multi-dimensional video from the data by the multi-dimensionally aware generator.

Abstract: Embodiments of the present disclosure disclose an image editing method and apparatus, an electronic device, and a storage medium. The method includes: receiving an image to be edited and an editing theme; generating, by using a preset vision-language model, an editing instruction and an editing position corresponding to the editing instruction based on the image to be edited and the editing theme; and editing the image to be edited based on the editing instruction and the editing position, to obtain a target image.

Abstract: A computing system including one or more processing devices configured to receive an image generation prompt and a reference image. Over a plurality of denoising timesteps, the one or more processing devices compute a guided image by applying denoising updates to a generated image at a denoising diffusion model. At a subset of the denoising timesteps, computing the guided image further includes applying guidance updates to the generated image based on the image generation prompt, the reference image, and a generated image set. The one or more processing devices compute each guidance update by performing a forward pass and a backward pass in first and second integration timesteps. A size of the generated image set and numbers of the first and second integration timesteps are each equal to a predefined integration timestep count. The one or more processing devices output a final generated image computed in a final denoising timestep.

Abstract: A computing system includes a processor and a storage device holding instructions executable by the processor to receive an initial image segmentation mask for an image. The initial image segmentation mask is input to a diffusion model trained to change pixel values of a plurality of mask pixels of the image segmentation mask to thereby generate a refined image segmentation mask for the image. The refined image segmentation mask is output.

Abstract: Embodiments of the present disclosure relate to validation of unsupervised adaptive models. According to example embodiments of the present disclosure, unlike methods validating with the seen target data, the present disclosure synthesizes new samples by mixing the target samples and pseudo labels. The accuracy between model predictions of mixed samples and the mixed labels are measured for model selection, and the accuracy score may be called PseudoMix. PseudoMix enjoys the combined inductive bias of previous methods. Experiments demonstrate that PseudoMix can keep state-of-the-art performance across different validation settings.

Abstract: Generating a multi-dimensional video using a multi-dimensional video generative model for, including, but not limited to, at least one of static portrait animation, video reconstruction, or motion editing. The method including providing data into the multi-dimensionally aware generator of the multi-dimensional video generative model, and generating the multi-dimensional video from the data by the multi-dimensionally aware generator.

Abstract: A computing system is described herein that implements a diffusion-based framework for animating reference images. The computing system includes a video diffusion model that is utilized to encode temporal information. The computing system further includes a novel appearance encoder that is utilized to retain the intricate details of the reference image and maintain appearance coherence across frames. The computing system further employs a video fusion technique to smooth transitions between animated segments in long video animation. Potential benefits of the computing system include enhanced temporal consistency, faithful preservation of reference images, and improved animation fidelity in the generated animation sequences.

Abstract: Embodiments of the present disclosure disclose an image generation method, an apparatus, an electronic device, and a storage medium.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for training and utilizing scale-diverse segmentation neural networks to analyze digital images at different scales and identify different target objects portrayed in the digital images. For example, in one or more embodiments, the disclosed systems analyze a digital image and corresponding user indicators (e.g., foreground indicators, background indicators, edge indicators, boundary region indicators, and/or voice indicators) at different scales utilizing a scale-diverse segmentation neural network. In particular, the disclosed systems can utilize the scale-diverse segmentation neural network to generate a plurality of semantically meaningful object segmentation outputs. Furthermore, the disclosed systems can provide the plurality of object segmentation outputs for display and selection to improve the efficiency and accuracy of identifying target objects and modifying the digital image.

Abstract: Methods of customizing generation of objects using diffusion models are provided. One or more parameters (e.g., a conditioning signal, network weights, or an initial or starting noise) of the diffusion model can be optimized by a backpropagation process, which can be performed by solving an augmented adjoint ordinary differential equation (ODE) based on an adjoint sensitivity method. The customized diffusion model can generate stylized objects, generate objects with specific visual effect(s), and provide adversary examples to audit security of an object generation system.

Abstract: Generating a multi-dimensional video using a multi-dimensional video generative model for, including, but not limited to, at least one of static portrait animation, video reconstruction, or motion editing. The method including providing data into the multi-dimensionally aware generator of the multi-dimensional video generative model, and generating the multi-dimensional video from the data by the multi-dimensionally aware generator.

Abstract: Embodiments of the present disclosure relate to validation of unsupervised adaptive models. According to example embodiments of the present disclosure, unlike methods validating with the seen target data, the present disclosure synthesizes new samples by mixing the target samples and pseudo labels. The accuracy between model predictions of mixed samples and the mixed labels are measured for model selection, and the accuracy score may be called PseudoMix. PseudoMix enjoys the combined inductive bias of previous methods. Experiments demonstrate that PseudoMix can keep state-of-the-art performance across different validation settings.

Abstract: Generating an object using a diffusion model includes obtaining a first input and a second input, and synthesizing an output object from the first input and the second input. The synthesizing of the output object includes generating a layout of the output object from the first input, injecting the second input as a content conditioner to the layout of the output object, and de-noising the layout of the output object injected with the content conditioner to generate a content of the output object.