Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly generating modified digital images utilizing a guided inpainting approach that implements a patch match model informed by a deep visual guide. In particular, the disclosed systems can utilize a visual guide algorithm to automatically generate guidance maps to help identify replacement pixels for inpainting regions of digital images utilizing a patch match model. For example, the disclosed systems can generate guidance maps in the form of structure maps, depth maps, or segmentation maps that respectively indicate the structure, depth, or segmentation of different portions of digital images. Additionally, the disclosed systems can implement a patch match model to identify replacement pixels for filling regions of digital images according to the structure, depth, and/or segmentation of the digital images.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems generate utilizing a segmentation neural network, an object mask for each object of a plurality of objects of a digital image. The disclosed systems detect a first user interaction with an object in the digital image displayed via a graphical user interface. The disclosed systems surface, via the graphical user interface, the object mask for the object in response to the first user interaction. The disclosed systems perform an object-aware modification of the digital image in response to a second user interaction with the object mask for the object.

Abstract: Systems and methods for upsampling low-resolution content within a high-resolution image include obtaining a composite image and a mask. The composite image includes a high-resolution region and a low-resolution region. An upsampling network identifies the low-resolution region of the composite image based on the mask and generates an upsampled composite image based on the composite image and the mask. The upsampled composite image comprises higher frequency details in the low-resolution region than the composite image.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, in one or more embodiments, the disclosed systems receive an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and the disclosed systems retrieve the image search results utilizing a weighted combination of the queries. In some implementations, the disclosed systems generate an input embedding for the search input (e.g., the multi-modal search input) and retrieve the image search results using the input embedding.

Abstract: Embodiments are disclosed for performing wire segmentation of images using machine learning. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input image, generating, by a first trained neural network model, a global probability map representation of the input image indicating a probability value of each pixel including a representation of wires, and identifying regions of the input image indicated as including the representation of wires. The disclosed systems and methods further comprise, for each region from the identified regions, concatenating the region and information from the global probability map to create a concatenated input, and generating, by a second trained neural network model, a local probability map representation of the region based on the concatenated input, indicating pixels of the region including representations of wires. The disclosed systems and methods further comprise aggregating local probability maps for each region.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.

Abstract: A method, apparatus, and non-transitory computer readable medium for image processing are described. Embodiments of the present disclosure obtain an image and an input text including a subject from the image and a location of the subject in the image. An image encoder encodes the image to obtain an image embedding. A text encoder encodes the input text to obtain a text embedding. An image processing apparatus based on the present disclosure generates an output text based on the image embedding and the text embedding. In some examples, the output text includes a relation of the subject to an object from the image and a location of the object in the image.

Abstract: A compressor includes a shell, a non-orbiting scroll disposed within the shell, an orbiting scroll disposed within the shell and meshed with the non-orbiting scroll, a driveshaft operable to drive the orbiting scroll relative to the non-orbiting scroll, and a ring-shaped counterweight assembly positioned on the driveshaft. The counterweight assembly has two radial surfaces and a circumferential edge between the two radial surfaces, and the counterweight assembly includes a counterweight fixed on the driveshaft and a cover attached to one of the counterweight and the driveshaft via a snap fit connection. The counterweight and the cover cooperate to define the counterweight assembly.

Abstract: Embodiments of the present disclosure include obtaining a text prompt describing an element, layout information indicating a target region for the element, and a precision level corresponding to the element. Some embodiments generate a text feature pyramid based on the text prompt, the layout information, and the precision level, wherein the text feature pyramid comprises a plurality of text feature maps at a plurality of scales, respectively. Then, an image is generated based on the text feature pyramid. In some cases, the image includes an object corresponding to the element of the text prompt at the target region. Additionally, a shape of the object corresponds to a shape of the target region based on the precision level.

Abstract: This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that perform language guided digital image editing utilizing a cycle-augmentation generative-adversarial neural network (CAGAN) that is augmented using a cross-modal cyclic mechanism. For example, the disclosed systems generate an editing description network that generates language embeddings which represent image transformations applied between a digital image and a modified digital image. The disclosed systems can further train a GAN to generate modified images by providing an input image and natural language embeddings generated by the editing description network (representing various modifications to the digital image from a ground truth modified image). In some instances, the disclosed systems also utilize an image request attention approach with the GAN to generate images that include adaptive edits in different spatial locations of the image.

Abstract: A method, non-transitory computer readable medium, apparatus, and system for image generation are described. An embodiment of the present disclosure includes obtaining an input image, an inpainting mask, and a plurality of content preservation values corresponding to different regions of the inpainting mask, and identifying a plurality of mask bands of the inpainting mask based on the plurality of content preservation values. An image generation model generates an output image based on the input image and the inpainting mask. The output image is generated in a plurality of phases. Each of the plurality of phases uses a corresponding mask band of the plurality of mask bands as an input.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for transferring global style features between digital images utilizing one or more machine learning models or neural networks. In particular, in one or more embodiments, the disclosed systems receive a request to transfer a global style from a source digital image to a target digital image, identify at least one target object within the target digital image, and transfer the global style from the source digital image to the target digital image while maintaining an object style of the at least one target object.

Abstract: Systems, methods, and non-transitory computer-readable media implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, one or more embodiments involve receiving an input digital image and search input and further modifying the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and one or more embodiments involve retrieving the image search results utilizing a weighted combination of the queries. Some implementations involve generating an input embedding for the search input (e.g., the multi-modal search input) and retrieving the image search results using the input embedding.

Abstract: Techniques for performing a digital operation on a digital image are described along with methods and systems employing such techniques. According to the techniques, an input (e.g., an input stroke) is received by, for example, a processing system. Based upon the input, an area of the digital image upon which a digital operation (e.g., for removal of a distractor within the area) is to be performed is determined. In an implementation, one or more metrics of an input stroke are analyzed, typically in real time, to at least partially determine the area upon which the digital operation is to be performed. In an additional or alternative implementation, the input includes a first point, a second point and a connector, and the area is at least partially determined by a location of the first point relative to a location of the second point and/or by locations of the first point and/or second point relative to one or more edges of the digital image.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing a cascaded modulation inpainting neural network. For example, the disclosed systems utilize a cascaded modulation inpainting neural network that includes cascaded modulation decoder layers. For example, in one or more decoder layers, the disclosed systems start with global code modulation that captures the global-range image structures followed by an additional modulation that refines the global predictions. Accordingly, in one or more implementations, the image inpainting system provides a mechanism to correct distorted local details. Furthermore, in one or more implementations, the image inpainting system leverages fast Fourier convolutions block within different resolution layers of the encoder architecture to expand the receptive field of the encoder and to allow the network encoder to better capture global structure.

Abstract: Various disclosed embodiments are directed to resizing, via down-sampling and up-sampling, a high-resolution input image in order to meet machine learning model low-resolution processing requirements, while also producing a high-resolution output image for image inpainting via a machine learning model. Some embodiments use a refinement model to refine the low-resolution inpainting result from the machine learning model such that there will be clear content with high resolution both inside and outside of the mask region in the output. Some embodiments employ new model architecture for the machine learning model that produces the inpainting result—an advanced Cascaded Modulated Generative Adversarial Network (CM-GAN) that includes Fast Fourier Convolution (FCC) layers at the skip connections between the encoder and decoder.

Abstract: Systems and methods provide editing operations in a smart editing system that may generate a focal point within a mask of an object for each frame of a video segment and perform editing effects on the frames of the video segment to quickly provide users with natural video editing effects. An eye-gaze network may produce a hotspot map of predicted focal points in a video frame. These predicted focal points may then be used by a gaze-to-mask network to determine objects in the image and generate an object mask for each of the detected objects. This process may then be repeated to effectively track the trajectory of objects and object focal points in videos. Based on the determined trajectory of an object in a video clip and editing parameters, the editing engine may produce editing effects relative to an object for the video clip.

Abstract: Systems and methods for multi-modal representation learning are described. One or more embodiments provide a visual representation learning system trained using machine learning techniques. For example, some embodiments of the visual representation learning system are trained using cross-modal training tasks including a combination of intra-modal and inter-modal similarity preservation objectives. In some examples, the training tasks are based on contrastive learning techniques.

Abstract: This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that trains (and utilizes) an image color editing diffusion neural network to generate a color edited digital image(s) for a digital image. In particular, in one or more implementations, the disclosed systems identify a digital image depicting content in a first color style. Moreover, the disclosed systems generate, from the digital image utilizing an image color editing diffusion neural network, a color-edited digital image depicting the content in a second color style different from the first color style. Further, the disclosed systems provide, for display within a graphical user interface, the color-edited digital image.