Abstract: Modeling semantic concepts in an embedding space as distributions is described. In the embedding space, both images and text labels are represented. The text labels describe semantic concepts that are exhibited in image content. In the embedding space, the semantic concepts described by the text labels are modeled as distributions. By using distributions, each semantic concept is modeled as a continuous cluster which can overlap other clusters that model other semantic concepts. For example, a distribution for the semantic concept “apple” can overlap distributions for the semantic concepts “fruit” and “tree” since can refer to both a fruit and a tree. In contrast to using distributions, conventionally configured visual-semantic embedding spaces represent a semantic concept as a single point. Thus, unlike these conventionally configured embedding spaces, the embedding spaces described herein are generated to model semantic concepts as distributions, such as Gaussian distributions, Gaussian mixtures, and so on.

Abstract: Digital content interaction prediction and training techniques that address imbalanced classes are described. In one or more implementations, a digital medium environment is described to predict user interaction with digital content that addresses an imbalance of numbers included in first and second classes in training data used to train a model using machine learning. The training data is received that describes the first class and the second class. A model is trained using machine learning. The training includes sampling the training data to include at least one subset of the training data from the first class and at least one subset of the training data from the second class. Iterative selections are made of a batch from the sampled training data. The iteratively selected batches are iteratively processed by a classifier implemented using machine learning to train the model.

Abstract: Techniques for image captioning with word vector representations are described. In implementations, instead of outputting results of caption analysis directly, the framework is adapted to output points in a semantic word vector space. These word vector representations reflect distance values in the context of the semantic word vector space. In this approach, words are mapped into a vector space and the results of caption analysis are expressed as points in the vector space that capture semantics between words. In the vector space, similar concepts with have small distance values. The word vectors are not tied to particular words or a single dictionary. A post-processing step is employed to map the points to words and convert the word vector representations to captions. Accordingly, conversion is delayed to a later stage in the process.

Abstract: Techniques for image captioning with weak supervision are described herein. In implementations, weak supervision data regarding a target image is obtained and utilized to provide detail information that supplements global image concepts derived for image captioning. Weak supervision data refers to noisy data that is not closely curated and may include errors. Given a target image, weak supervision data for visually similar images may be collected from sources of weakly annotated images, such as online social networks. Generally, images posted online include “weak” annotations in the form of tags, titles, labels, and short descriptions added by users. Weak supervision data for the target image is generated by extracting keywords for visually similar images discovered in the different sources. The keywords included in the weak supervision data are then employed to modulate weights applied for probabilistic classifications during image captioning analysis.

Abstract: Embodiments of the present invention relate to learning image representation by distilling from multi-task networks. In implementation, more than one single-task network is trained with heterogeneous labels. In some embodiments, each of the single-task networks is transformed into a Siamese structure with three branches of sub-networks so that a common triplet ranking loss can be applied to each branch. A distilling network is trained that approximates the single-task networks on a common ranking task. In some embodiments, the distilling network is a Siamese network whose ranking function is optimized to approximate an ensemble ranking of each of the single-task networks. The distilling network can be utilized to predict tags to associate with a test image or identify similar images to the test image.

Abstract: Content creation and sharing integration techniques and systems are described. In one or more implementations, techniques are described in which modifiable versions of content (e.g., images) are created and shared via a content sharing service such that image creation functionality used to create the images is preserved to permit continued creation using this functionality. In one or more additional implementations, image creation functionality employed by a creative professional to create content is leveraged to locate similar images from a content sharing service.

Abstract: Image search techniques and systems involving emotions are described. In one or more implementations, a digital medium environment of a content sharing service is described for image search result configuration and control based on a search request that indicates an emotion. The search request is received that includes one or more keywords and specifies an emotion. Images are located that are available for licensing by matching one or more tags associated with the image with the one or more keywords and as corresponding to the emotion. The emotion of the images is identified using one or more models that are trained using machine learning based at least in part on training images having tagged emotions. Output is controlled of a search result having one or more representations of the images that are selectable to license respective images from the content sharing service.

Abstract: Content update and suggestion techniques are described. In one or more implementations, techniques are implemented to generate suggestions that are usable to guide creative professionals in updating content such as images, video, sound, multimedia, and so forth. A variety of techniques are usable to generate suggestions for the content professionals. In one example, suggestions are based on shared characteristics of images licensed by users of a content sharing service, e.g., licensed by the users. In another example, suggestions are based on metadata of the images licensed by the users, the metadata describing characteristics of how the images are created. These suggestions are then used to guide transformation of a user's image such that the image exhibits these characteristics and thus has an increased likelihood of being desired for licensing by customers of the service.

Abstract: Content creation and sharing integration techniques and systems are described. In one or more implementations, techniques are described in which modifiable versions of content (e.g., images) are created and shared via a content sharing service such that image creation functionality used to create the images is preserved to permit continued creation using this functionality. In one or more additional implementations, image creation functionality employed by a creative professional to create content is leveraged to locate similar images from a content sharing service.

Abstract: Content creation collection and navigation techniques and systems are described. In one example, a representative image is used by a content sharing service to interact with a collection of images provided as part of a search result. In another example, a user interface image navigation control is configured to support user navigation through images based on one or more metrics. In a further example, a user interface image navigation control is configured to support user navigation through images based on one or more metrics identified for an object selected from the image. In yet another example, collections of images are leveraged as part of content creation. In another example, data obtained from a content sharing service is leveraged to indicate suitability of images of a user for licensing as part of the service.

Abstract: Content creation collection and navigation techniques and systems are described. In one example, a representative image is used by a content sharing service to interact with a collection of images provided as part of a search result. In another example, a user interface image navigation control is configured to support user navigation through images based on one or more metrics. In a further example, a user interface image navigation control is configured to support user navigation through images based on one or more metrics identified for an object selected from the image. In yet another example, collections of images are leveraged as part of content creation. In another example, data obtained from a content sharing service is leveraged to indicate suitability of images of a user for licensing as part of the service.

Abstract: Font recognition and similarity determination techniques and systems are described. In a first example, localization techniques are described to train a model using machine learning (e.g., a convolutional neural network) using training images. The model is then used to localize text in a subsequently received image, and may do so automatically and without user intervention, e.g., without specifying any of the edges of a bounding box. In a second example, a deep neural network is directly learned as an embedding function of a model that is usable to determine font similarity. In a third example, techniques are described that leverage attributes described in metadata associated with fonts as part of font recognition and similarity determinations.

Abstract: Font recognition and similarity determination techniques and systems are described. In a first example, localization techniques are described to train a model using machine learning (e.g., a convolutional neural network) using training images. The model is then used to localize text in a subsequently received image, and may do so automatically and without user intervention, e.g., without specifying any of the edges of a bounding box. In a second example, a deep neural network is directly learned as an embedding function of a model that is usable to determine font similarity. In a third example, techniques are described that leverage attributes described in metadata associated with fonts as part of font recognition and similarity determinations.

Abstract: Font recognition and similarity determination techniques and systems are described. In a first example, localization techniques are described to train a model using machine learning (e.g., a convolutional neural network) using training images. The model is then used to localize text in a subsequently received image, and may do so automatically and without user intervention, e.g., without specifying any of the edges of a bounding box. In a second example, a deep neural network is directly learned as an embedding function of a model that is usable to determine font similarity. In a third example, techniques are described that leverage attributes described in metadata associated with fonts as part of font recognition and similarity determinations.

Abstract: Image editing techniques are disclosed that support a number of physically-based image editing tasks, including object insertion and relighting. The techniques can be implemented, for example in an image editing application that is executable on a computing system. In one such embodiment, the editing application is configured to compute a scene from a single image, by automatically estimating dense depth and diffuse reflectance, which respectively form the geometry and surface materials of the scene. Sources of illumination are then inferred, conditioned on the estimated scene geometry and surface materials and without any user input, to form a complete 3D physical scene model corresponding to the image. The scene model may include estimates of the geometry, illumination, and material properties represented in the scene, and various camera parameters. Using this scene model, objects can be readily inserted and composited into the input image with realistic lighting, shadowing, and perspective.

Abstract: Example systems and methods for classifying visual patterns into a plurality of classes are presented. Using reference visual patterns of known classification, at least one image or visual pattern classifier is generated, which is then employed to classify a plurality of candidate visual patterns of unknown classification. The classification scheme employed may be hierarchical or nonhierarchical. The types of visual patterns may be fonts, human faces, or any other type of visual patterns or images subject to classification.

Abstract: Systems and methods are discussed to separate the specular reflectivity and/or the diffuse reflectivity from an input image. Embodiments of the invention can be used to determine the specular chromaticity by iteratively solving one or more objective functions. An objective function can include functions that take into account the smooth gradient of the specular chromaticity. An objective function can take into account the interior chromatic homogeneity of the diffuse chromaticity and/or the sharp changes between chromaticity. Embodiments of the invention can also be used to determine the specular chromaticity of an image using a pseudo specular-free image that is calculated from the input image and a dark channel image that can be used to iteratively solve an objective function(s).

Abstract: Robust techniques for self-calibration of a moving camera observing a planar scene. Plane-based self-calibration techniques may take as input the homographies between images estimated from point correspondences and provide an estimate of the focal lengths of all the cameras. A plane-based self-calibration technique may be based on the enumeration of the inherently bounded space of the focal lengths. Each sample of the search space defines a plane in the 3D space and in turn produces a tentative Euclidean reconstruction of all the cameras that is then scored. The sample with the best score is chosen and the final focal lengths and camera motions are computed. Variations on this technique handle both constant focal length cases and varying focal length cases.

Abstract: Embodiments of the present invention relate to finding semantic parts in images. In implementation, a convolutional neural network (CNN) is applied to a set of images to extract features for each image. Each feature is defined by a feature vector that enables a subset of the set of images to be clustered in accordance with a similarity between feature vectors. Normalized cuts may be utilized to help preserve pose within each cluster. The images in the cluster are aligned and part proposals are generated by sampling various regions in various sizes across the aligned images. To determine which part proposal corresponds to a semantic part, a classifier is trained for each part proposal and semantic part to determine which part proposal best fits the correlation pattern given by the true semantic part. In this way, semantic parts in images can be identified without any previous part annotations.

Abstract: Deep convolutional neural networks receive local and global representations of images as inputs and learn the best representation for a particular feature through multiple convolutional and fully connected layers. A double-column neural network structure receives each of the local and global representations as two heterogeneous parallel inputs to the two columns. After some layers of transformations, the two columns are merged to form the final classifier. Additionally, features may be learned in one of the fully connected layers. The features of the images may be leveraged to boost classification accuracy of other features by learning a regularized double-column neural network.