Abstract: Systems, methods, and computer program products for identifying a relevant candidate product in an electronic marketplace. Embodiments perform a visual similarity comparison between candidate product image visual content and input query image visual content, process formal and informal natural language user inputs, and coordinate aggregated past user interactions with the marketplace stored in a knowledge graph. Visually similar items and their corresponding product categories, aspects, and aspect values can determine suggested candidate products without discernible delay during a multi-turn user dialog. The user can then refine the search for the most relevant items available for purchase by providing responses to machine-generated prompts that are based on the initial search results from visual, voice, and/or text inputs. An intelligent online personal assistant can thus guide a user to the most relevant candidate product more efficiently than existing search tools.

Abstract: A machine is configured to generate in real time personalized online banner images for users based on data pertaining to user behavior in relation to an image of a product. For example, the machine receives a user selection indicating one or more data features associated with the user. The one or more data features include a data feature pertaining to user behavior in relation to an image of a product. The machine generates, using a machine learning algorithm, a data representation of the machine learning algorithm based on the one or more data features including the data feature pertaining to user behavior in relation to the image of the product. The data representation includes one or more data features pertaining to one or more characteristics of online banner images. The machine generates an online banner image for the user based on the data representation.

Abstract: Techniques and systems are described that leverage computer vision as part of search to expand functionality of a computing device available to a user and increase operational computational efficiency as well as efficiency in user interaction. In a first example, user interaction with items of digital content is monitored. Computer vision techniques are used to identify digital images in the digital content, objects within the digital images, and characteristics of those objects. This information is used to assign a user to a user segment of a user population which is then used to control output of subsequent digital content to the user, e.g., recommendations, digital marketing content, and so forth.

Abstract: Camera platform and object inventory control techniques are described. In an implementation a live feed of digital images is output in a user interface by a computing device. A user selection is received through interaction with the user interface of at least one of the digital images. An object, included within the at least one digital image, is recognized using machine learning. Metadata is then obtained that pertains to the recognized object. Augmented reality digital content is generated based at least in part of the obtained metadata. The augmented reality digital content is displayed as part of the live feed of digital images as associated with the object.

Abstract: Various embodiments use a neural network to analyze images for aspects that characterize the images, to present locations of those aspects on the images, and, additionally, to permit a user to interact with those locations on the images. For example, a user may interact with a visual cue over one of those locations to modify, refine, or filter the results of a visual search, performed on a publication corpus, that uses an input image (e.g., one captured using a mobile device) as a search query.

Abstract: Systems and methods configured to perform monitoring of one or more user activities are described. In one embodiment, a plurality of sensors is configured for contact-free monitoring of at least one bodily function. A signal processing module communicatively coupled to the sensors is configured to receive data from the sensors. A first sensor is configured to generate a first set of quantitative data associated with a user speed and a user position. A second sensor is configured to generate a second set of quantitative data associated with a user action. A third sensor is configured to generate a third set of quantitative data associated with a user movement. The signal processing module is configured to process the three sets of quantitative data using a machine learning module, and identify a user activity and detect a condition associated with the user.

Abstract: Privacy-preserving activity monitoring systems and methods are described. In one embodiment, a plurality of sensors is configured for contact-free monitoring of at least one user state. A signal processing module communicatively coupled to the sensors is configured to receive data from the sensors. A first sensor is configured to generate a first set of quantitative data associated with a first user state. A second sensor is configured to generate a second set of quantitative data associated with a second user state. A third sensor is configured to generate a third set of quantitative data associated with a third user state. The signal processing module is configured to process the three sets of quantitative data using a machine learning module, and identify a user activity and detect a condition associated with the user, where no user-identifying information is communicated more than 100 meters to or from the signal processing module.

Abstract: In various example embodiments, a system and method for determining an item that has confirmed characteristics are described herein. An image that depicts an object is received from a client device. Structured data that corresponds to characteristics of one or more items are retrieved. A set of characteristics is determined, the set of characteristics being predicted to match with the object. An interface that includes a request for confirmation of the set of characteristics is generated. The interface is displayed on the client device. Confirmation that at least one characteristic from the set of characteristics matches with the object depicted in the image is received from the client device.

Abstract: An image is passed through an image identifier to identify a coarse category for the image and a bounding box for a categorized object. A mask is used to identify the portion of the image that represents the object. Given the foreground mask, the convex hull of the mask is located and an aligned rectangle of minimum area that encloses the hull is fitted. The aligned bounding box is rotated and scaled, so that the foreground object is roughly moved to a standard orientation and size (referred to as calibrated). The calibrated image is used as an input to a fine-grained categorization module, which determines the fine category within the coarse category for the input image.

Abstract: A machine is configured to generate in real time personalized online banner images for users based on data pertaining to user behavior in relation to an image of a product. For example, the machine receives a user selection indicating one or more data features associated with the user. The one or more data features include a data feature pertaining to user behavior in relation to an image of a product. The machine generates, using a machine learning algorithm, a data representation of the machine learning algorithm based on the one or more data features including the data feature pertaining to user behavior in relation to the image of the product. The data representation includes one or more data features pertaining to one or more characteristics of online banner images. The machine generates an online banner image for the user based on the data representation.

Abstract: Methods, systems, and computer programs are presented for adding new features to a network service. A method includes receiving an image depicting an object of interest. A category set is determined for the object of interest and an image signature is generated for the image. Using the category set and the image signature, the method identifies a set of publications within a publication database and assigns a rank to each publication. The method causes presentation of the ranked list of publications at a computing device from which the image was received.

Abstract: In various example embodiments, a system and method for determining an item that has confirmed characteristics are described herein. An image that depicts an object is received from a client device. Structured data that corresponds to characteristics of one or more items are retrieved. A set of characteristics is determined, the set of characteristics being predicted to match with the object. An interface that includes a request for confirmation of the set of characteristics is generated. The interface is displayed on the client device. Confirmation that at least one characteristic from the set of characteristics matches with the object depicted in the image is received from the client device.

Abstract: In various example embodiments, a system and method for projecting visual aspects into a vector space are presented. A query that includes visual data is received by the system from a client device. A visual aspect indicated in the visual data is analyzed. One or more symbols that correspond to the analyzed visual aspect is generated by the system. The analyzed visual aspect is projected into a vector space using the one or more symbols. A group of projections are identified, the group of projections being within a predetermined distance from the projected visual aspect in the vector space. An interface that depicts the further visual aspects is generated. The interface is displayed on the client device.

Abstract: Various embodiments described herein utilize multiple levels of generative adversarial networks (GANs) to facilitate generation of digital images based on user-provided images. Some embodiments comprise a first generative adversarial network (GAN) and a second GAN coupled to the first GAN, where the first GAN includes an image generator and at least two discriminators, and the second GAN includes an image generator and at least one discriminator. According to some embodiments, the (first) image generator of the first GAN is trained by processing a user-provided image using the first GAN. For some embodiments, the user-provided image and the first generated image, generated by processing the user-provided image using the first GAN, are combined to produce a combined image. For some embodiments, the (second) image generator of the second GAN is trained by processing the combined image using the second GAN.

Abstract: Aspects of the present disclosure relate to a network-based crowd assisted query system that includes a client device in communication with an application server executing the crowd assisted query system over a network. For example, the crowd assisted query system may be or include a group of one or more server machines. Users of the crowd assisted query system are presented with a graphical user interface (GUI) configured to receive queries that include data objects, wherein the data objects include representations of unidentified items of interest to the user. The data objects may include media content, such as graphical images as well as audio data, and in some example embodiments may further include text data describing the unidentified items.

Abstract: In various example embodiments, a system and method for projecting visual aspects into a vector space are presented. A query that includes visual data is received by the system from a client device. A visual aspect indicated in the visual data is analyzed. One or more symbols that correspond to the analyzed visual aspect is generated by the system. The analyzed visual aspect is projected into a vector space using the one or more symbols. A group of projections are identified, the group of projections being within a predetermined distance from the projected visual aspect in the vector space. An interface that depicts the further visual aspects is generated. The interface is displayed on the client device.

Abstract: A machine is configured to generate in real time personalized online banner images for users based on data pertaining to user behavior in relation to an image of a product. For example, the machine receives a user selection indicating one or more data features associated with the user. The one or more data features include a data feature pertaining to user behavior in relation to an image of a product. The machine generates, using a machine learning algorithm, a data representation of the machine learning algorithm based on the one or more data features including the data feature pertaining to user behavior in relation to the image of the product. The data representation includes one or more data features pertaining to one or more characteristics of online banner images. The machine generates an online banner image for the user based on the data representation.

Abstract: Camera platform techniques are described. In an implementation, a plurality of digital images and data describing times, at which, the plurality of digital images are captured is received by a computing device. Objects of clothing are recognized from the digital images by the computing device using object recognition as part of machine learning. A user schedule is also received by the computing device that describes user appointments and times, at which, the appointments are scheduled. A user profile is generated by the computing device by training a model using machine learning based on the recognized objects of clothing, times at which corresponding digital images are captured, and the user schedule. From the user profile, a recommendation is generated by processing a subsequent user schedule using the model as part of machine learning by the computing device.

Abstract: Various embodiments described herein utilize multiple levels of generative adversarial networks (GANs) to facilitate generation of digital images based on user-provided images. Some embodiments comprise a first generative adversarial network (GAN) and a second GAN coupled to the first GAN, where the first GAN includes an image generator and at least two discriminators, and the second GAN includes an image generator and at least one discriminator. According to some embodiments, the (first) image generator of the first GAN is trained by processing a user-provided image using the first GAN. For some embodiments, the user-provided image and the first generated image, generated by processing the user-provided image using the first GAN, are combined to produce a combined image. For some embodiments, the (second) image generator of the second GAN is trained by processing the combined image using the second GAN.

Abstract: Camera platform techniques are described. In an implementation, a plurality of digital images and data describing times, at which, the plurality of digital images are captured is received by a computing device. Objects of clothing are recognized from the digital images by the computing device using object recognition as part of machine learning. A user schedule is also received by the computing device that describes user appointments and times, at which, the appointments are scheduled. A user profile is generated by the computing device by training a model using machine learning based on the recognized objects of clothing, times at which corresponding digital images are captured, and the user schedule. From the user profile, a recommendation is generated by processing a subsequent user schedule using the model as part of machine learning by the computing device.