Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure, via a multi-modal encoder of an image processing apparatus, encodes a text prompt to obtain a text embedding. A color encoder of the image processing apparatus encodes a color prompt to obtain a color embedding. A diffusion prior model of the image processing apparatus generates an image embedding based on the text embedding and the color embedding. A latent diffusion model of the image processing apparatus generates an image based on the image embedding, where the image includes an element from the text prompt and a color from the color prompt.

Abstract: Systems and methods for image tagging are provided. One aspect of the systems and methods includes encoding an image and a tag of the image using a multimodal encoder to obtain an image embedding and a text embedding, respectively. Another aspect of the systems and methods includes generating training data for a machine learning model by filtering a plurality of image-tag pairs based on a similarity between the image embedding and the text embedding. Another aspect of the systems and methods includes training the machine learning model using the training data.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a diffusion neural network for mask aware image and typography editing. For example, in one or more embodiments the disclosed systems utilize a text-image encoder to generate a base image embedding from a base digital image. Moreover, the disclosed systems generate a mask-segmented image by combining a shape mask with the base digital image. In one or more implementations, the disclosed systems utilize noising steps of a diffusion noising model to generate a mask-segmented image noise map from the mask-segmented image. Furthermore, the disclosed systems utilize a diffusion neural network to create a stylized image corresponding to the shape mask from the base image embedding and the mask-segmented image noise map.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure obtain a text prompt for text guided image generation. A multi-modal encoder of an image processing apparatus encodes the text prompt to obtain a text embedding. A diffusion prior model of the image processing apparatus converts the text embedding to an image embedding. A latent diffusion model of the image processing apparatus generates an image based on the image embedding, wherein the image includes an element described by the text prompt.

Abstract: Systems and methods for image processing are provided. One aspect of the systems and methods includes obtaining a text prompt in a first language. Another aspect of the systems and methods includes encoding the text prompt using a multilingual encoder to obtain a multilingual text embedding. Yet another aspect of the systems and methods includes processing the multilingual text embedding using a diffusion prior model to obtain an image embedding, wherein the diffusion prior model is trained to process multilingual text embeddings from the first language and a second language based on training data from the first language and the second language. Yet another aspect of the systems and methods includes generating an image using a diffusion model based on the image embedding, wherein the image includes an element corresponding to the text prompt.

Abstract: A method, apparatus, non-transitory computer readable medium, and system for image generation include obtaining a text embedding of a text prompt and an image embedding of an image prompt. Some embodiments map the text embedding into a joint embedding space to obtain a joint text embedding and map the image embedding into the joint embedding space to obtain a joint image embedding. Some embodiments generate a synthetic image based on the joint text embedding and the joint image embedding.

Abstract: Embodiments are disclosed for machine learning-based generation of recommended layouts. The method includes receiving a set of design elements for performing generative layout recommendation. A number of each type of design element from the set of design elements is determined. A set of recommended layouts are generated using a trained generative layout model and the number and type of design elements. The set of recommended layouts are output.

Abstract: The present disclosure describes systems and methods for information retrieval. Embodiments of the disclosure provide a color embedding network trained using machine learning techniques to generate embedded color representations for color terms included in a text search query. For example, techniques described herein are used to represent color text in a same space as color embeddings (e.g., an embedding space created by determining a histogram of LAB based colors in a three-dimensional (3D) space). Further, techniques are described for indexing color palettes for all the searchable images in the search space. Accordingly, color terms in a text query are directly converted into a color palette and an image search system can return one or more search images with corresponding color palettes that are relevant to (e.g., within a threshold distance from) the color palette of the text query.

Abstract: Embodiments of the technology described herein, provide improved visual search results by combining a visual similarity and a textual similarity between images. In an embodiment, the visual similarity is quantified as a visual similarity score and the textual similarity is quantified as a textual similarity score. The textual similarity is determined based on text, such as a title, associated with the image. The overall similarity of two images is quantified as a weighted combination of the textual similarity score and the visual similarity score. In an embodiment, the weighting between the textual similarity score and the visual similarity score is user configurable through a control on the search interface. In one embodiment, the aggregate similarity score is the sum of a weighted visual similarity score and a weighted textual similarity score.

Abstract: Systems and methods for product retrieval are described. One or more aspects of the systems and methods include receiving a query that includes a text description of a product associated with a brand; identifying the product based on the query by comparing the text description to a product embedding of the product, wherein the product embedding is based on a brand embedding of the brand; and displaying product information for the product in response to the query, wherein the product information includes the brand.

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure identify a plurality of candidate concepts in a knowledge graph (KG) that correspond to an image tag of an image; generate an image embedding of the image using a multi-modal encoder; generate a concept embedding for each of the plurality of candidate concepts using the multi-modal encoder; select a matching concept from the plurality of candidate concepts based on the image embedding and the concept embedding; and generate association data between the image and the matching concept.

Abstract: The present disclosure describes systems and methods for information retrieval. Embodiments of the disclosure provide a color embedding network trained using machine learning techniques to generate embedded color representations for color terms included in a text search query. For example, techniques described herein are used to represent color text in a same space as color embeddings (e.g., an embedding space created by determining a histogram of LAB based colors in a three-dimensional (3D) space). Further, techniques are described for indexing color palettes for all the searchable images in the search space. Accordingly, color terms in a text query are directly converted into a color palette and an image search system can return one or more search images with corresponding color palettes that are relevant to (e.g., within a threshold distance from) the color palette of the text query.

Abstract: Embodiments of the technology described herein, provide improved visual search results by combining a visual similarity and a textual similarity between images. In an embodiment, the visual similarity is quantified as a visual similarity score and the textual similarity is quantified as a textual similarity score. The textual similarity is determined based on text, such as a title, associated with the image. The overall similarity of two images is quantified as a weighted combination of the textual similarity score and the visual similarity score. In an embodiment, the weighting between the textual similarity score and the visual similarity score is user configurable through a control on the search interface. In one embodiment, the aggregate similarity score is the sum of a weighted visual similarity score and a weighted textual similarity score.