Abstract: Systems and methods for removing objects from images are disclosed. An image processing application identifies a boundary of each object of a set of objects in an image. The image processing application identifies a completed boundary for each object of the set of objects by providing the object to a trained model. The image processing application determines a set of masks. Each mask corresponds to an object of the set of objects and represents a region of the image defined by an intersection of the boundary of the object and the boundary of a target object to be removed from the image. The image processing application updates each mask by separately performing content filling on the corresponding region. The image processing application creates an output image by merging each of the updated masks with portions of the image.

Abstract: A system identifies a video comprising frames associated with content tags. The system detects features for each frame of the video. The system identifies, based on the detected features, scenes of the video. The system determines, for each frame for each scene, a frame score that indicates a number of content tags that match the other frames within the scene. The system selects, for each scene, a set of key frames that represent the scene based on the determined frame scores. The system receives a search query comprising a keyword. The system generates, for display, search results responsive to the search query including a dynamic preview of the video. The dynamic preview comprises an arrangement of frames of the video corresponding to each scene of the video. Each of the arrangement of frames is selected from the selected set of key frames representing the respective scene of the video.

Abstract: Digital image ordering based on object position and aesthetics is leveraged in a digital medium environment. According to various implementations, an image analysis system is implemented to identify visual objects in digital images and determine aesthetics attributes of the digital images. The digital images can then be arranged in way that prioritizes digital images that include relevant visual objects and that exhibit optimum visual aesthetics.

Abstract: Techniques are disclosed for generating image recommendations to facilitate the sale of a product. An example methodology includes identifying a product category associated with an image of the product provided by the seller, and a product sub-category associated with the product image. The method further includes retrieving one or more images of for-sale items. The retrieval is based on a search of for-sale listings using the identified product category and the identified product sub-category. The method further includes clustering the retrieved images of for-sale items into groups, each group associated with a perspective viewpoint of the for-sale item. The method further includes providing a selected image from each group as an image recommendation. The selection is based on a value score associated with each of the images of the for-sale items. A graphical status indicating completeness of the seller's image set is updated in response to recommended images being adopted.

Abstract: Certain embodiments involve adaptive search results for multimedia search queries to provide dynamic previews. For instance, a computing system receives a search query that includes a keyword. The computing system identifies, based on the search query, a video file having keyframes with content tags that match the search query. The computing system determines matching scores for respective keyframes of the identified video file. The computing system generates a dynamic preview from at least two keyframes having the highest matching scores.

Abstract: Techniques are provided for identifying objects (such as products within a physical store) within a captured video scene and indicating which of object in the captured scene matches a desired object requested by a user. The matching object is then displayed in an accentuated manner to the user in real-time (via augmented reality). Object identification is carried out via a multimodal methodology. Objects within the captured video scene are identified using a neural network trained to identify different types of objects. The identified objects can then be compared against a database of pre-stored images of the desired product to determine if a close match is found. Additionally, text on the identified objects is analyzed and compared to the text of the desired object. Based on either or both identification methods, the desired object is indicated to the user on their display, via an augmented reality graphic.

Abstract: The present disclosure describes systems, non-transitory computer-readable media, and methods that can generate contextual identifiers indicating context for frames of a video and utilize those contextual identifiers to generate translations of text corresponding to such video frames. By analyzing a digital video file, the disclosed systems can identify video frames corresponding to a scene and a term sequence corresponding to a subset of the video frames. Based on images features of the video frames corresponding to the scene, the disclosed systems can utilize a contextual neural network to generate a contextual identifier (e.g. a contextual tag) indicating context for the video frames. Based on the contextual identifier, the disclosed systems can subsequently apply a translation neural network to generate a translation of the term sequence from a source language to a target language. In some cases, the translation neural network also generates affinity scores for the translation.

Abstract: Structural modifications to a person's face in a reference image are captured and automatically applied to the person's face in another image. The reference image is processed to compute landmark information on the person's face and apply a mesh to the reference image. When structural modifications are made to the person's face in the reference image, the mesh is modified, and the modified mesh is stored in association with the landmark information. Another image is analyzed to compute landmark information on the person's face in that image and apply a mesh to the image. A transformation matrix is computed using the landmark information from the reference image and current image, and the modified mesh from the reference image is transformed using the transformation matrix. The mesh in the current image is modified using the transformed mesh, thereby applying the structural modification to the person's face in the current image.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that generate dynamic tool-based animated tutorials. In particular, in one or more embodiments, the disclosed systems generate an animated tutorial in response to receiving a request associated with an image editing tool. The disclosed systems then extract steps from existing general tutorials that pertain to the image editing tool to generate tool-specific animated tutorials. In at least one embodiment, the disclosed systems utilize a clustering algorithm in conjunction with image parameters to provide a set of these generated animated tutorials that showcase diverse features and/or attributes of the image editing tool based on measured aesthetic gains resulting from application of the image editing tool within the animated tutorials.

Abstract: Techniques are disclosed for generating image recommendations to facilitate the sale of a product. An example methodology includes identifying a product category associated with an image of the product provided by the seller, and a product sub-category associated with the product image. The method further includes retrieving one or more images of for-sale items. The retrieval is based on a search of for-sale listings using the identified product category and the identified product sub-category. The method further includes clustering the retrieved images of for-sale items into groups, each group associated with a perspective viewpoint of the for-sale item. The method further includes providing a selected image from each group as an image recommendation. The selection is based on a value score associated with each of the images of the for-sale items. A graphical status indicating completeness of the seller's image set is updated in response to recommended images being adopted.

Abstract: Techniques are disclosed for generating image recommendations to facilitate the sale of a product. An example methodology includes identifying a product category associated with an image of the product provided by the seller, and a product sub-category associated with the product image. The method further includes retrieving one or more images of for-sale items. The retrieval is based on a search of for-sale listings using the identified product category and the identified product sub-category. The method further includes clustering the retrieved images of for-sale items into groups, each group associated with a perspective viewpoint of the for-sale item. The method further includes providing a selected image from each group as an image recommendation. The selection is based on a value score associated with each of the images of the for-sale items. A graphical status indicating completeness of the seller's image set is updated in response to recommended images being adopted.

Abstract: Techniques are provided for identifying objects (such as products within a physical store) within a captured video scene and indicating which of object in the captured scene matches a desired object requested by a user. The matching object is then displayed in an accentuated manner to the user in real-time (via augmented reality). Object identification is carried out via a multimodal methodology. Objects within the captured video scene are identified using a neural network trained to identify different types of objects. The identified objects can then be compared against a database of pre-stored images of the desired product to determine if a close match is found. Additionally, text on the identified objects is analyzed and compared to the text of the desired object. Based on either or both identification methods, the desired object is indicated to the user on their display, via an augmented reality graphic.

Abstract: Techniques are provided for identifying objects (such as products within a physical store) within a captured video scene and indicating which of object in the captured scene matches a desired object requested by a user. The matching object is then displayed in an accentuated manner to the user in real-time (via augmented reality). Object identification is carried out via a multimodal methodology. Objects within the captured video scene are identified using a neural network trained to identify different types of objects. The identified objects can then be compared against a database of pre-stored images of the desired product to determine if a close match is found. Additionally, text on the identified objects is analyzed and compared to the text of the desired object. Based on either or both identification methods, the desired object is indicated to the user on their display, via an augmented reality graphic.

Abstract: Techniques are provided herein for more efficiently storing images that have a common subject, such as product images that share the same product in the image. Each image undergoes an adaptive tiling procedure to split the image into a plurality of tiles, with each tile identifying a region of the image having pixels with the same content. The tiles across multiple images can then be clustered together and those tiles having identical content are removed. Once all duplicate tiles have been removed from the set of all tiles across the images, the tiles are once again clustered based on their encoding scheme and certain encoding parameters. Tiles within each cluster are compressed using the best compression technique for the tiles in each corresponding cluster. By removing duplicative tile content between numerous images of the same subject, the total amount of data that needs to be stored is reduced.

Abstract: An image editing program can include a content-aware selection system. The content-aware selection system can enable a user to select an area of an image using a label or a tag that identifies object in the image, rather than having to make a selection area based on coordinates and/or pixel values. The program can receive a digital image and metadata that describes an object in the image. The program can further receive a label, and can determine from the metadata that the label is associated with the object. The program can then select a bounding box for the object, and identify in the bounding box, pixels that represent the object. The program can then output a selection area that surrounds the pixels.

Abstract: In some embodiments, contextual image variations are generated for an input image. For example, a contextual composite image depicting a variation is generated based on a input image and a synthetic image component. The synthetic image component includes contextual features of a target object from the input image, such as shading, illumination, or depth that are depicted on the target object. The synthetic image component also includes a pattern from an additional image, such as a fabric pattern. In some cases, a mesh is determined for the target object. Illuminance values are determined for each mesh block. An adjusted mesh is determined based on the illuminance values. The synthetic image component is based on a combination of the adjusted mesh and the pattern from the additional image, such as a depiction of the fabric pattern with stretching, folding, or other contextual features from the target image.

Abstract: A method of automatically computing tool properties of a virtual paint brush includes obtaining an edge map from a digital image, wherein boundaries of objects in the image are detected, computing contours for the edge map, receiving a brush stroke of the virtual paint brush from a user at a point on a contour, and finding an object boundary that corresponds to the contour that received the brush stroke, computing a tangential slope of the object boundary at the point of the brush stroke, adjusting tool properties of the virtual paint brush based on a change in the tangential slope of the object boundary, and visually displaying the adjusted tool properties dynamically while the user is moving brush strokes around the object boundaries.

Abstract: Described herein is a system and techniques for classification of subjects within image information. In some embodiments, a set of subjects may be identified within image data obtained at two different points in time. For each of the subjects in the set of subjects, facial landmark relationships may be assessed at the two different points in time to determine a difference in facial expression. That difference may be compared to a threshold value. Additionally, contours of each of the subjects in the set of subjects may be assessed at the two different points in time to determine a difference in body position. That difference may be compared to a different threshold value. Each of the subjects in the set of subjects may then be classified based on the comparison between the differences and the threshold values.

Abstract: An image modification system receives image features of a base image and an additional image. The base image and the additional image depict an apparel item displayed on a mannequin. A first feature pair from the base image and a second feature pair from the additional images are determined. A first distance is calculated between the first feature pair and a second distance is calculated between the second feature pair. Based on a ratio including the first and second distances, a matching relationship between the first and second feature pairs is determined. A pixel of the base image is identified within an image area occluded by the mannequin. Based on the matching relationship, image data is identified for a corresponding additional pixel from the additional image. A modified base image including a ghost mannequin effect is generated by modifying the pixel to include the image data of the additional pixel.

Abstract: Methods, systems, and computer storage media for providing tool tutorials based on tutorial information that is dynamically integrated into tool tutorial shells using graphics editing system operations in a graphics editing systems. In operation, an image is received in association with a graphics editing application. Tool parameters (e.g., image-specific tool parameters) are generated based on processing the image. The tool parameters are generated for a graphics editing tool of the graphics editing application. The graphics editing tool (e.g., object removal tool or spot healing tool) can be a premium version of a simplified version of the graphics editing tool in a freemium application service. Based on the tool parameters and the image, a tool tutorial data file is generated by incorporating the tool parameters and the image into a tool tutorial shell. The tool tutorial data file can be selectively rendered in an integrated interface of the graphics editing application.