Abstract: Image processing techniques for self-captured images are disclosed. An image can be captured during activation of an illumination element of a mobile device. Presence of a representation of the mobile device can be determined in the captured image based at least in part upon locating a reflection of an illumination produced by the illumination element. Dimensions of the representation of the mobile device can be determined and compared to actual dimensions of the mobile device to provide a scaling factor. The scaling factor can be used to determine the size of various objects, including articles of clothing, in the image.

Abstract: Visual effects for element of interest can be displayed within a live camera view in real time or substantially using a processing pipeline that does not immediately display an acquired image until it has been updated with the effects. In various embodiments, software-based approaches, such as fast convolution algorithms, and/or hardware-based approaches, such as using a graphics processing unit (GPU), can be used reduce the time between acquiring an image and displaying the image with various visual effects. These visual effects can include automatically highlighting elements, augmenting the color, style, and/or size of elements, casting a shadow on elements, erasing elements, substituting elements, or shaking and jumbling elements, among other effects.

Abstract: Tasks such as object classification from image data can take advantage of a deep learning process using convolutional neural networks. These networks can include a convolutional layer followed by an activation layer, or activation unit, among other potential layers. Improved accuracy can be obtained by using a generalized linear unit (GLU) as an activation unit in such a network, where a GLU is linear for both positive and negative inputs, and is defined by a positive slope, a negative slope, and a bias. These parameters can be learned for each channel or a block of channels, and stacking those types of activation units can further improve accuracy.

Abstract: The subject technology provides embodiments for a relevance ranking algorithm of point of interests (POIs). Based on a query request from a user of a mapping application, the relevance ranking orders the matching POIs based on either a reverse logistic distance function or a logarithmic distance function that is used as a multiplier for a baseline score of each of the matching POIs. The multiplier is used to boost or discount the respective relevance of each of the matching POIs, especially with respect to a specified area of interest included in the query request or a location of a user included in the query request. A subset of the ranked POIs may then be selected for display to the user.

Abstract: Various embodiments may obtain an image representation of an object for use in image matching and content retrieval. For example, an image matching system processes video content items to determine one or more scenes for one or more video content items. The image matching system can extract, from at least one video frame for a scene, feature descriptors relating to one or more objects represented in the at least one video frame. The image matching system indexes the feature descriptors into a feature index storing information for each of the feature descriptors and respective corresponding video frame. The image matching system correlates the feature descriptors of the feature index to determine one or more groups having similar feature descriptors. The image matching system indexes the one or more groups into a correlation index storing information for each of the one or more groups and respective corresponding feature descriptors.

Abstract: The present technology relates to methods, systems and computer program products for displaying search completion results. A chalkboard interface having search completion results can be displayed in response to inputs entered in a search field of a webpage. One or more search completion options, such as predicted search queries, can be generated based at least on the inputs. The search completion options and corresponding search results can be displayed in the chalkboard interface. If further inputs are entered in the search field, the search completion options are modified, resulting in a dynamic chalkboard interface that displays search completion options with corresponding search results that change with each keystroke or input inputted at the search field. The chalkboard interface can be superimposed over the webpage, except for the search field, so that as inputs are entered in the search field, the search completion options and search results are accordingly modified.

Abstract: Approaches include using a machine learning-based approach to generating search strings and refinements based on a specific item represented in an image. For example, a classifier that is trained on descriptions of images can be provided. An image that includes a representation of an item of interest is obtained. The image is analyzed using the classifier algorithm to determine a first term representing a visual characteristic of the image. Then, the image is analyzed again to determine a second term representing another visual characteristic of the image based at least in part on the first term. Additional terms can be determined to generate a description of the image, including characteristics of the item of interest. Based on the determined characteristics of the item of interest, a search query and one or more refinements can be generated.

Abstract: A user can share digital content with another user or device using dynamically-generated barcodes. A user might request to share an electronic book (“e-book”), stored on a first device, with another user having a second device. The first device can generate a barcode that includes not only information about the identity of the e-book, but also information about the user or the first device, which can help to determine rights and/or access restrictions for the content. The second device can capture an image of the barcode generated on a display of the first device, and use information extracted from that image to attempt to obtain a copy of the e-book. Other types of barcodes can be generated based at least in part upon the context, such as where a user is attempting to locate a physical copy of a type of digital content in a retail location.

Abstract: Techniques to determine the three-dimensional structure of wearable items are disclosed. A scanning device can project waves at a wearable item and receive reflections of the waves. In embodiments, the scanning device projects light pulses and detects reflections of the light pulses. Depth measurements from the scanning device to the wearable item can be used to determine points corresponding to locations on the wearable item. A point cloud can be generated of the points, which can convey the three-dimensional structure of the wearable item. In embodiments, another wearable item can be determined to have a corresponding three-dimensional structure based on point cloud comparisons. The second wearable item can be suggested to a user as a wearable item having an appropriate fit for the user.

Abstract: Approaches provide for using voice information to influence the importance of search result categories for a search query. For example, various embodiments may provide search results for a search query based on a most relevant search result category to the search query. Voice information associated with a subsequent user interaction may be analyzed to identify whether the search result category is correct or if search results from a different category should be provided. Additionally, the voice information may be used to update the relevance score of the search result category to the search query to improve the category matching of future queries.

Abstract: A number of approaches for using a camera or other device to capture an image of an arbitrary signature, drawing and/or other scribble rendered on a package and utilizing the captured image as electronic postage for shipping the package to the recipient destination. A sender wishing to send a package handwrites a signature or other scribble directly onto the package that is to be shipped. A mobile device (e.g., smartphone) can be used to capture an image of the scribble and perform image recognition to determine whether the captured image contains sufficient features to be distinct from other images stored in a database. If the image is sufficiently distinct, the image can be accepted as electronic postage for the package. When the package is picked up, the delivery person scans the scribble and matches it to the image on file to determine the recipient address and other shipping information.

Abstract: Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

Abstract: Techniques for experimentation on live content in a production environment are disclosed. In one embodiment, a method for live experimentation comprises receiving a first request from a client computing device for an instance of content, sending a second request for an instance of content to a content provider, receiving the instance of content from the content provider in response to the second request, associating a test script with the received instance of content, and sending the instance of content with the associated test script to the client computing device. The client computing device is configured to execute the test script on the instance of content without modifying content executed by the content provider.

Abstract: Techniques for updating content created for a third party user are disclosed. In one embodiment, the method for updating content comprises receiving information for an item associated with a third party user, requesting a search including information for a plurality of items offered for purchase, the search request being based on the received information for the at least one item, and updating content for the third party user with one of the items included in the list of items received in response to the search request. Information for the item associated with the third party user may be received from a list of items associated with the third party user and maintained by the third party user or from a content source associated with an entity other than the third party user.

Abstract: Video content can be analyzed to identify particular sections of the video content. Speech to text or similar techniques can be used to obtain a transcription of the video content. The transcription can be indexed (e.g., timestamped) to the video content. Information describing how users are interacting with or consuming the video content (e.g., social media information, viewing history data, etc.) can be collected and used to identify the particular sections. Once the particular sections have been identified, other services can be provided. For example, custom trailers and summaries of the video content can be generated based on the identified sections. Additionally, the video content can be augmented to include additional information relevant to the particular sections, such as production information, actor information, or other information. The additional information can be added so as not to interfere with the important sections.

Abstract: The subject technology provides embodiments for performing fast corner detection in a given image for augmented reality applications. Embodiments disclose a high-speed test that examines intensities of pairs of pixels around a candidate center pixel. In one example, the examined pairs are comprised of pixels that are diametrically opposite ends of a circle formed with the candidate center pixel. Further, a pyramid of images including four rings of surrounding pixels is generated. An orientation of the pixels from the four rings are determined and a vector of discrete values of the pixels are provided. Next, a forest of trees are generated for the vector of discrete values corresponding to a descriptor for a first image. For a second image including a set of descriptors, approximate nearest neighbors are determined from the forest of tree representing closest matching descriptors from the first image.

Abstract: A system is provided for providing location-specific images to a mobile device for display. The system generally comprises three components: a mobile device having a screen, a position identification system (e.g., a GPS receiver) that determines the position of the mobile device, and a database containing location-specific images taken at various locations. Each location-specific image is associated with geographic coordinates of the location at which the image was taken. Based on the position of the mobile device as determined by the position identification system, a location-specific image is selected from the database and displayed on the screen of the mobile device. For example, when a user is using the mobile device as a car navigation system, a location-specific image of the user's destination location can be selected and displayed when the determined position of the mobile device comes within a certain distance from the geographic coordinates of the destination location.

Abstract: Various embodiments provide a method for determining color information for an image. For example, a color descriptor for an image can be determined and compared against color descriptors stored for each of a number of sample images, which each represent a color in a color space. Upon comparison, matching scores can be generated for a color match between the image and each respective sample image. In this example, the number of sample images with a matching score above a threshold value can be summed and the image can be assigned to a color associated with a highest frequency of the number of sample images. Accordingly, the assigned color of the image can then be used in a “query by color” search or a browse-by-color capability.

Abstract: Approaches introduce a pre-processing and post-processing framework to a neural network-based approach to identify items represented in an image. For example, a classifier that is trained on several categories can be provided. An image that includes a representation of an item of interest is obtained. Rotated versions of the image are generated and each of a subset of the rotated images is analyzed to determine a probability that a respective image includes an instance of a particular category. The probabilities can be used to determine a probability distribution of output category data, and the data can be analyzed to select an image of the rotated versions of the image. Thereafter, a categorization tree can then be utilized, whereby for the item of interest represented the image, the category of the item can be determined. The determined category can be provided to an item retrieval algorithm to determine primary content for the item of interest.

Abstract: Various embodiments enable an image recognition system reduce the number image match candidates before running a full-fledged pair-wise match on all image match candidates. In order to accomplish this, each inventory image can be assigned to a group. For example, a title for a book sold by an electronic marketplace could be available in multiple languages, in multiple bindings, and the book could be available in print, audio book, or electronic book. Each one of these variations could be associated with its own similarly looking inventory image, each of which could be returned as a valid match to a query image for the book. Accordingly, the inventory images for these variations could be assigned to a group for the book and, instead of geometrically processing an image for each variation, the image match system can process a single image representing all of the variations.