Abstract: A computing device can obtain data describing at least one document, the at least one document referencing at least one media object, wherein a portion of the at least one media object includes one or more characters. The computing device can obtain data describing the one or more characters in the at least one media object in the at least one document. The computing device can generate an updated copy of the at least one document that includes the data describing the one or more characters in the at least one media object. The computing device can present, on a display screen of the computing device and through an interface, the updated copy of the at least one document, wherein the one or more characters in the at least one media object are able to be selected or searched.

Abstract: An image creation and editing tool can use the data produced from training a neural network to add stylized representations of an object to an image. An object classification will correspond to an object representation, and pixel values for the object representation can be added to, or blended with, the pixel values of an image in order to add a visualization of a type of object to the image. Such an approach can be used to add stylized representations of objects to existing images or create new images based on those representations. The visualizations can be used to create patterns and textures as well, as may be used to paint or fill various regions of an image. Such patterns can enable regions to be filled where image data has been deleted, such as to remove an undesired object, in a way that appears natural for the contents of the image.

Abstract: Images of various views of objects can be captured. An object mesh structure can be created based at least in part on the object images. The object mesh structure represents the three-dimensional shape of the object. Alpha masks indicating which pixels are associated with the object can be used to refine the object mesh structure. A request can be made to view the object from an arbitrary viewpoint which differs from the viewpoints associated with the captured images. A subset of the captured images can be used to create a synthetic image. Different weights can be assigned to the captured image to render a synthetic image that represents the view from the arbitrary viewpoint selected. The input images for the synthetic image can be prefetched, or loaded into memory before the arbitrary view is requested. The images can also be cached for future use or to avoid reloading them for another synthetic image.

Abstract: Users can view images or renderings of items placed (virtually) within a physical space. For example, a rendering of an item can be placed within a live camera view of the physical space. A snapshot of the physical space can be captured and the snapshot can be customized, shared, etc. The renderings can be represented as two-dimensional images, e.g., virtual stickers or three-dimensional models of the items. Users can have the ability to view different renderings, move those items around, and develop views of the physical space that may be desirable. The renderings can link to products offered through an electronic marketplace and those products can be consumed. Further, collaborative design is enabled through modeling the physical space and enabling users to view and move around the renderings in a virtual view of the physical space.

Abstract: Objects can be rendered in three dimensions and viewed and manipulated in an augmented reality environment. A number of object images, a number of segmentation masks, and an object mesh structure are used by a client device to render the object in three dimensions. The object images and segmentation masks can be sequenced into frames. The object images and segmentation masks can be partitioned into patches and sequenced, or ordered, within each patch, and a keyframe can be assigned in each patch. Then, the object images and segmentation masks can be encoded into video files and sent to a client device. The client device can quickly retrieve a requested object image and segmentation mask based at least in part on identifying the keyframe in the same patch as the object image and segmentation mask.

Abstract: Various embodiments of systems and methods allow the presentation of a content item with a marketplace section. A system can receive a content item and analyze the content item to identify at least one featured object in the content item. The system can then match the at least one featured object with a marketplace item. If the marketplace item passes various filtering criteria, then the author of the content item can approve the marketplace item being presented with the content item. A system can then receive data for the content item from an application server and data for the marketplace item from a marketplace server and generate a unified presentation of the content item and the marketplace item. A user can then interact with a marketplace section to buy the marketplace item.

Abstract: Visual effects for element of interest can be displayed within a live camera view in real time or substantially using a processing pipeline that does not immediately display an acquired image until it has been updated with the effects. In various embodiments, software-based approaches, such as fast convolution algorithms, and/or hardware-based approaches, such as using a graphics processing unit (GPU), can be used reduce the time between acquiring an image and displaying the image with various visual effects. These visual effects can include automatically highlighting elements, augmenting the color, style, and/or size of elements, casting a shadow on elements, erasing elements, substituting elements, or shaking and jumbling elements, among other effects.

Abstract: Systems and methods are disclosed for a system configured to access first content to determine that textual features common to queries received in a website are available in text within the first content. An identifier within the first content is determined where the identifier references second content that comprises details associated with the textual features. In response to a content query including the textual features received in the website, display content is generated. The display content includes a first area to display matching content indexed from prior user searches and also includes a second area to be modified. A portion of the details and the identifier is provided in the second area.

Abstract: Systems and methods are provided for previewing content that is located behind the links of a search results webpage without having to click on the links and thereby cause webpage changes. Embodiments provide a webpage that presents a first set of search results in response to a search query submitted by a user. This webpage is displayed in a graphical user interface and the first set of search results comprises one or more links to other search results webpages, which are defined dynamically and cannot be pre-computed. In the event a user mouses over one of the links, embodiments execute the search associated with the moused-over link and then render selected portions of the search results in a preview window that is positioned over a portion the webpage that displays the first set of search results.

Abstract: Approaches are described for an interface element that displays progressively more information about a search result in a listing of search results. In an embodiment, a user interacts with a search result, for example by utilizing a gesture on a touch-sensitive display, and a supplemental content item is generated and displayed using, in one example, a subset of available supplemental information associated with the search result. The display of the supplemental content item is accompanied by an animation is some embodiments whereby the supplemental content item visually “slides” over to replace the search result.

Abstract: An advertising system can be configured to determine advertisements that are responsive to targeting segments included in advertisement requests without having to individually evaluate Boolean or logical targeting expressions associated with each advertisement. For example, the advertising system can be configured to determine, for advertisements, one or more terms that correspond to at least a portion of the respective targeting expressions of the advertisement. When an advertisement request is received, a determination can be made as to whether any of the segments (e.g., age group) associated with that request correspond to any of the terms that were derived from the advertisements. The advertisement system can then determine any advertisements that match the term(s) that correspond to the request.

Abstract: A semantic analysis can be performed to determine an intent of a received query. The intent can relate to a primary object of the query, which can be identified through the semantic analysis. Other attributes can be determined from the query that help to focus the object of the intent. A query vector is generated, based on the intent and primary object, and used to search a multi-dimensional semantic space including semantic representations of possible matches. The attributes are used to adjust the query vector in the semantic space. Objects having vectors ending proximate the query vector are identified as potential search results, with the distance from the query vector being used as a ranking mechanism. If refinement is needed, a dialog is used to obtain additional information from the user. Once results are obtained with sufficient confidence, results can be returned as search results for the query.

Abstract: Various embodiments enable an image recognition system reduce the number image match candidates before running a full-fledged pair-wise match on all image match candidates. In order to accomplish this, each inventory image can be assigned to a group. For example, a title for a book sold by an electronic marketplace could be available in multiple languages, in multiple bindings, and the book could be available in print, audio book, or electronic book. Each one of these variations could be associated with its own similarly looking inventory image, each of which could be returned as a valid match to a query image for the book. Accordingly, the inventory images for these variations could be assigned to a group for the book and, instead of geometrically processing an image for each variation, the image match system can process a single image representing all of the variations.

Abstract: Because not all companies register or obtain barcodes (or other encodings or identifiers) from a single, consistent source, and because different companies can be associated with the same barcode at different times, there can be multiple products associated with a specific barcode. If a user requests information associated with a specific barcode, information associated with the user or request can be used to attempt to determine the associated product if more than one matching product is uncovered. In some embodiments, a location of a user device at approximately a time of the request can be used to attempt to determine the appropriate product, such as if the user is in a store that only offers one of the potential products. Other information can be used as well, such as past locations, purchase history, viewing patterns, and the like.

Abstract: The processing of search queries for a customer using a set of resource can balance performance with cost, in order to ensure that the processing satisfies customer performance requirements while attempting to minimize the cost to the customer for obtaining that performance. In addition to dynamically updating the allocation of resources used to process the requests, such as to change the number or sizes of allocated resources, the number of indexes to be searched can be updated as well. For example, a search index can be divided into two sub-indexes against which queries can be executed concurrently in order to reduce the latency by about a half. Adjustments to the indexes and resources can be balanced to minimize cost while retaining performance, which can include rejoining sub-indexes or reducing the number of resource as appropriate.

Abstract: Memory management is provided for processors, such as GPUs used to process data using a trained machine learning model. Requests received to a CPU can be stored to a request queue until the queue is full, or until a timeout value has been reached for periods of lower activity. The requests can then be batched and sent to a GPU as a single message on a single thread. Memory can be pre-allocated, and the trained model loaded into GPU memory once for processing of the relevant batches. The individual requests can be processed by the GPU and the results analyzed to determine at least a subset of results to return to the CPU, which can be provided back as results of the processing.

Abstract: Large scale instance recognition is provided that can take advantage of channel-wise pooling. A received query image is processed to extract a set of features that can be used to generate a set of region proposals. The proposed regions of image data are processed using a trained classifier to classify the regions as object or non-object regions. Extracted features for the object regions are processed using feature correlation against extracted features for a set of object images, each representing a classified object. Matching tensors generated from the comparison are processed using a spatial verification network to determine match scores for the various object images with respect to a specific object region. The match scores are used to determine which objects, or types of objects, are represented in the query image. Information or content associated with the matching objects can be provided as part of a response.

Abstract: Content can be located for items that are stylistically similar to an item of interest. The item of interest can be represented in a query image, which is analyzed to determine one or more regions having an item represented therein. The classification of the item is determined, enabling identification of a trained model to be used to process image data for the region(s) of the query image. The trained model outputs a set of attributes, relating to visual or stylistic attributes, and corresponding confidence or prominence values for the attributes. These attributes and values can be compared against a data repository to locate items determined to be similar based on corresponding attributes and values. A similarity determination algorithm can identify similar items and rank those items by similarity. Content for the most similar items is returned as a result for the query image.

Abstract: Various embodiments provide methods and systems for users and business owners to share content and/or links to visual elements of a place at a physical location, and, in response to a user device pointing at a tagged place, causing the content and/or links to the visual elements of the place to be presented on the user device. In some embodiments, content and links are tied to specific objects at a place based at least in part upon one of Global Positioning System (GPS) locations, Inertial Measurement Unit (IMU) orientations, compass data, or one or more visual matching algorithms. Once the content and links are attached to the specific objects of the place, they can be discovered by a user with a portable device pointing at the specific objects in the real world.

Abstract: Approaches herein provide for multi-camera calibration and subsequent application in augmented reality applications. The approach obtains image data of a three-dimensional (3D) calibration object from different directions using the at least one camera. A determination is made for extrinsic parameters associated with a relative position of the 3D calibration object to the at least one camera. A further determination is made for intrinsic parameters associated with the at least one camera relative to the 3D calibration object facing in the different positions. Revised intrinsic parameters are generated from the intrinsic parameters. The revised intrinsic parameters provide a calibration output that may be applied to generate views of an item in an augmented reality application.