Abstract: The present concepts relate to matching data of two different modalities using two stages of attention. First data is encoded as a set of first vectors representing components of the first data, and second data is encoded as a set of second vectors representing components of the second data. In the first stage, the components of the first data are attended by comparing the first vectors and the second vectors to generate a set of attended vectors. In the second stage, the components of the second data are attended by comparing the second vectors and the attended vectors to generate a plurality of relevance scores. Then, the relevance scores are pooled to calculate a similarity score that indicates a degree of similarity between the first data and the second data.

Abstract: Representative embodiments disclose mechanisms to perform visual intent classification or visual intent detection or both on an image. Visual intent classification utilizes a trained machine learning model that classifies subjects in the image according to a classification taxonomy. The visual intent classification can be used as a pre-triggering mechanism to initiate further action in order to substantially save processing time. Example further actions include user scenarios, query formulation, user experience enhancement, and so forth. Visual intent detection utilizes a trained machine learning model to identify subjects in an image, place a bounding box around the image, and classify the subject according to the taxonomy. The trained machine learning model utilizes multiple feature detectors, multi-layer predictions, multilabel classifiers, and bounding box regression.

Abstract: A visual search system comprised of a computing device, the computing device including an image processing engine for generating a feature vector representing a user-selected object in an image input, an object detection engine for locating one or more objects in the image input and for determining a category of a user-selected object from objects in the image input, the object detection engine using the category to generate a plurality of attributes for the user-selected object, a product data store for storing a plurality of tables storing one or more attributes associated with a category of the user-selected object, an attribute generation engine for generating a plurality of attribute options for each of the attributes of the user-selected object, and an attribute matching engine for comparing attributes and attribute options of the user-selected object with attributes and attribute options of visually similar products and images.

Abstract: Methods, systems, and computer programs are presented for identifying the brand and model of products embedded within an image. One method includes operations for receiving, via a graphical user interface (GUI), a selection of an image, and for analyzing the image to determine a location within the image of one or more products. For each product in the image, determining a unique identification of the product is determined, the unique identification including a manufacturer of the product and a model identifier. The method further includes an operation for presenting information about the one or more products in the GUI with a selection option for selecting each of the one or more products. Additionally, the method includes operations for receiving a product selection for one of the one or more products, and presenting shopping options in the GUI for purchasing the selected product.

Abstract: A method for using a speech signal to augment a visual search includes processing the image data to determine an image search intent. Concurrently with processing the image data, the method processes the speech signal to determine at least one speech search intent. The method generates a search query by combining keywords and/or the image from the image search intent with keywords from the speech search intent. The method then performs a search based on the generated query and reports the results of the search. The method generates the image search intent by applying the image data to a knowledge base and generates the speech search intent by converting the speech to text and applying the text to a cognition service.

Abstract: The description relates to diversified hybrid image annotation for annotating images. One implementation includes generating first image annotations for a query image using a retrieval-based image annotation technique. Second image annotations can be generated for the query image using a model-based image annotation technique. The first and second image annotations can be integrated to generate a diversified hybrid image annotation result for the query image.

Abstract: Systems and methods for identifying search results in response to a search query are presented. More particularly, images are selected as search results, at least in part, according to an attractiveness value associated with the images. Upon receiving a search query, a set of content is identified according to the query intent of the search query and includes at least one image. The identified set of content is ordered according an overall score determined according to relevance and, in the case of the at least one image, according to an attractiveness value. A search results generator selects items from the set of content according to their overall scores, including the at least one image, generates a search results page, and returns the search results page to the requesting party.

Abstract: Non-limiting examples of the present disclosure relate to object detection processing of image content that categorically classifies specific objects within image content. Exemplary object detection processing may be utilized to enhance visual search processing including content retrieval and curation, among other technical advantages. An exemplary object detection model is implemented to categorically classify an object. In doing, so an exemplary object detection model may classify objects based on: analysis of specific objects within image content, positioning of the objects within the image content and intent associated with the image content, among other examples. The object detection model generates exemplary categorical classification(s) for specific data objects, which may be propagated to enhance processing efficiency and accuracy during visual search processing.

Abstract: Systems and methods can be implemented to conduct a visual search as a service in a variety of applications. In various embodiments, a system is configured to provide searching capabilities of content provided by a first entity in response to a search request by a second entity. An image provided by the second entity can be used by the system as a query image to search the content of the first entity. In an embodiment, the first entity can be a commercial entity providing such a system with image related content regarding its products and services such that any number of individual consumers can search for particular products and services of the commercial entity via their communication enabled devices. In addition, such systems can be arranged for other embodiments to provide customized searches of a single source by many individual devices. Additional systems and methods are disclosed.

Abstract: Systems and methods can be implemented to conduct searches based on images used as queries in a variety of applications. In various embodiments, a set of visual words representing a query image are generated from features extracted from the query image and are compared with visual words of index images. A set of candidate images is generated from the index images resulting from matching one or more visual words in the comparison. A multi-level ranking is conducted to sort the candidate images of the set of candidate images, and results of the multi-level ranking are returned to a user device that provided the query image. Additional systems and methods are disclosed.

Abstract: A technique for hyperparameter tuning can be performed via a hyperparameter tuning tool. In the technique, computer-readable values for each of one or more machine learning hyperparameters can be received. Multiple computer-readable hyperparameter value sets can be defined using different combinations of the values. In response to a request to start, an overall hyperparameter tuning operation can be performed via the tool, with the overall operation including a tuning job for each of the hyperparameter sets. A computer-readable comparison of the results of the parameter tuning operations can be generated for the hyperparameter sets, with the comparison indicating effectiveness of the hyperparameter sets, as compared to each other, in the tuning jobs.