Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure obtain a low-resolution image and a text description of the low-resolution image. A mapping network generates a style vector representing the text description of the low-resolution image. An adaptive convolution component generates an adaptive convolution filter based on the style vector. An image generation network generates a high-resolution image corresponding to the low-resolution image based on the adaptive convolution filter.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize machine learning to generate a sequence of transition frames for a gap in a clipped digital video. For example, the disclosed system receives a clipped digital video that includes a pre-cut frame prior to a gap in the clipped digital video and a post-cut frame following the gap in the clipped digital video. Moreover, the disclosed system utilizes a natural motion sequence model to generates a sequence of transition keypoint maps between the pre-cut frame and the post-cut frame. Furthermore, using a generative neural network, the disclosed system generates a sequence of transition frames for the gap in the clipped digital video from the sequence of transition keypoint maps.

Abstract: Provided herein are novel anti-CLDN 18.2 antibodies and chimeric antigen receptors (CAR), cells or compositions comprising the same, vector or plasmid encoding anti-CLDN 18.2 CAR, anti-CLDN 18.2 antibody-drug conjugates (ADCs), bispecific antibodies containing anti-CLDN 18.2 antibodies, and methods for producing the same, or using the same for detecting or treating ovarian cancer or prostate cancer. Also provided herein are anti-CLDN 18.2 antibodies, compositions comprising the same, nucleic acid sequence encoding the same, and a kit for detecting CLDN 18.2.

Abstract: Systems and methods for fine-tuning diffusion models are described. Embodiments of the present disclosure obtain an input text indicating an element to be included in an image; generate a synthetic image depicting the element based on the input text using a diffusion model trained by comparing synthetic images depicting the element to training images depicting elements similar to the element and updating selected parameters corresponding to an attention layer of the diffusion model based on the comparison.

Abstract: Systems and methods for image generation include obtaining an input image and an attribute value representing an attribute of the input image to be modified; computing a modified latent vector for the input image by applying the attribute value to a basis vector corresponding to the attribute in a latent space of an image generation network; and generating a modified image based on the modified latent vector using the image generation network, wherein the modified image includes the attribute based on the attribute value.

Abstract: Systems and methods for image generation are described. Embodiments of the present disclosure obtain user input that indicates a target color and a semantic label for a region of an image to be generated. The system also generates of obtains a noise map including noise biased towards the target color in the region indicated by the user input. A diffusion model generates the image based on the noise map and the semantic label for the region. The image can include an object in the designated region that is described by the semantic label and that has the target color.

Abstract: Systems and methods for image editing are described. Embodiments of the present disclosure include obtaining an image and a prompt for editing the image. A diffusion model is tuned based on the image to generate different versions of the image. The prompt is then encoded to obtain a guidance vector, and the diffusion model generates a modified image based on the image and the encoded text prompt.

Abstract: Aspects of the methods, apparatus, non-transitory computer readable medium, and systems include obtaining a noise map and a global image code encoded from an original image and representing semantic content of the original image; generating a plurality of image patches based on the noise map and the global image code using a diffusion model; and combining the plurality of image patches to produce an output image including the semantic content.

Abstract: Systems and methods dynamically adjust an available range for editing an attribute in an image. An image editing system computes a metric for an attribute in an input image as a function of a latent space representation of the input image and a filtering vector for editing the input image. The image editing system compares the metric to a threshold. If the metric exceeds the threshold, then the image editing system selects a first range for editing the attribute in the input image. If the metric does not exceed the threshold, a second range is selected. The image editing system causes display of a user interface for editing the input image comprising an interface element for editing the attribute within the selected range.

Abstract: Various disclosed embodiments are directed to changing parameters of an input image or multidimensional representation of the input image based on a user request to change such parameters. An input image is first received. A multidimensional image that represents the input image in multiple dimensions is generated via a model. A request to change at least a first parameter to a second parameter is received via user input at a user device. Such request is a request to edit or generate the multidimensional image in some way. For instance, the request may be to change the light source position or camera position from a first set of coordinates to a second set of coordinates.

Abstract: An image processing system uses a depth-conditioned autoencoder to generate a modified image from an input image such that the modified image maintains an overall structure from the input image while modifying textural features. An encoder of the depth-conditioned autoencoder extracts a structure latent code from an input image and depth information for the input image. A generator of the depth-conditioned autoencoder generates a modified image using the structure latent code and a texture latent code. The modified image generated by the depth-conditioned autoencoder includes the structural features from the input image while incorporating textural features of the texture latent code. In some aspects, the autoencoder is depth-conditioned during training by augmenting training images with depth information. The autoencoder is trained to preserve the depth information when generating images.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.

Abstract: One example method involves operations for receiving a request to transform an input image into a target image. Operations further include providing the input image to a machine learning model trained to adapt images. Training the machine learning model includes accessing training data having a source domain of images and a target domain of images with a target style. Training further includes using a pre-trained generative model to generate an adapted source domain of adapted images having the target style. The adapted source domain is generated by determining a rate of change for parameters of the target style, generating weighted parameters by applying a weight to each of the parameters based on their respective rate of change, and applying the weighted parameters to the source domain. Additionally, operations include using the machine learning model to generate the target image by modifying parameters of the input image using the target style.

Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.

Abstract: A monolithic dielectric coating composed of microscale periodic high-contrast gratings on multilayers of high and low refractive index optical materials is described, which is deposited on metal thin-films of flexible polymer insulation sheeting. The emittance can be minimized to any blackbody temperature, using parameter optimization of high-contrast grating phase-shift mode conditions. The high-low refractive index infrared-transparent multilayer is based on Fabry-Pérot cavity, but with non-quarter-wave thicknesses to achieve multilayer insulation conductance below that of metal films. This ultralow emittance coating is most relevant to thermal management of refrigeration and electronic components.

Abstract: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.

Abstract: Embodiments are disclosed for automatic object re-colorization in images. In some embodiments, a method of automatic object re-colorization includes receiving a request to recolor an object in an image, the request including an object identifier and a color identifier, identifying an object in the image associated with the object identifier, generating a mask corresponding to the object in the image, providing the image, the mask, and the color identifier to a color transformer network, the color transformer network trained to recolor objects in input images, and generating, by the color transformer network, a recolored image, wherein the object in the recolored image has been recolored to a color corresponding to the color identifier.

Abstract: Semantic fill techniques are described that support generating fill and editing images from semantic inputs. A user input, for example, is received by a semantic fill system that indicates a selection of a first region of a digital image and a corresponding semantic label. The user input is utilized by the semantic fill system to generate a guidance attention map of the digital image. The semantic fill system leverages the guidance attention map to generate a sparse attention map of a second region of the digital image. A semantic fill of pixels is generated for the first region based on the semantic label and the sparse attention map. The edited digital image is displayed in a user interface.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for combining digital images. In particular, in one or more embodiments, the disclosed systems combine latent codes of a source digital image and a target digital image utilizing a blending network to determine a combined latent encoding and generate a combined digital image from the combined latent encoding utilizing a generative neural network. In some embodiments, the disclosed systems determine an intersection face mask between the source digital image and the combined digital image utilizing a face segmentation network and combine the source digital image and the combined digital image utilizing the intersection face mask to generate a blended digital image.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for latent-based editing of digital images using a generative neural network. In particular, in one or more embodiments, the disclosed systems perform latent-based editing of a digital image by mapping a feature tensor and a set of style vectors for the digital image into a joint feature style space. In one or more implementations, the disclosed systems apply a joint feature style perturbation and/or modification vectors within the joint feature style space to determine modified style vectors and a modified feature tensor. Moreover, in one or more embodiments the disclosed systems generate a modified digital image utilizing a generative neural network from the modified style vectors and the modified feature tensor.