Abstract: A method including capturing a document object model event from a user accessing a document in an application is provided. The method includes identifying a window for the application in a display, identifying a relative position of a cursor from the user within the window, and detecting a resolution for the display within the window. The method also includes inserting a time-stamp for the document object model event and adding the document object model event, the relative position of the cursor, the resolution for the display, and the time-stamp in a playback code. The method also includes playing back the document object model event in a sequence according to the time-stamp, the relative position of the cursor, and the resolution for the display, in a second display. A system and a non-transitory, computer-readable medium to perform the above method are also provided.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for relational image querying. A system may receive a search query for content from a client device, where the query specifies one or more objects and one or more spatial relationships between the one or more objects. The system may generate a query vector for the query using a computer-operated neural language model. The system may compare the query vector to an indexed vector for each of the one or more spatial relationships between the one or more objects of an image. The system may determine a listing of relational images from a collection of images based on the comparison. The system may determine a ranking for each image in the listing of relational images, and provide search results responsive to the search query to the client device, which may include a prioritized listing of the relational images based on the determined ranking.

Abstract: A method for receiving an image query from a user via a client device is provided. The method includes determining a user personalized data based on a prior user history, generating a synthetic image with a generative tool, based on the image query and the user personalized data, and evaluating a similarity between the synthetic image and a real image in an image database with a discriminative tool. The method also includes providing the synthetic image to the user for selection and storing a user response to the synthetic image in the prior user history. A system and a non-transitory, computer readable medium storing instructions to cause the system to perform the above method are also disclosed.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for language translation based on image search similarities. These aspects include an image retrieval system using a convolutional neural network that is trained to identify a correlation between an image and a language term, and using an image search engine to search against images corresponding to visual words that are responsive to a given search query in a given spoken language. These aspects include access to interaction probability data that identifies user interaction probabilities for the visual words to determine a correlation between the input language terms of the search query and the rate at which users interact with images of a corresponding visual word that is responsive to the search query. The system then provides a prioritized listing of images that is responsive to the given search query based on the identified user interaction probabilities.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for video generation. An input video can be provided to a video server that is coupled to a video database storing hundreds, thousands, millions, or billions of video clips. Each stored video clip includes content that is known to the video server. The video server identifies aspects of the content of the input video and identifies matching video clips stored in the video database, using a trained machine-learning engine. The machine-learning engine can also modify some aspects of the input video while maintaining other aspects of the input video.

Abstract: Computer image clustering systems and methods for conducting effective media searches by grouping multimedia documents tagged by keywords into a hierarchy of images configured to: (1) maintain a first database, (2) maintain an initial occurrence matrix, (3) maintain an occurrence matrix, (4) maintain a media file activation score for each media file in the first database, (5) generate a log version of the occurrence matrix, (6) maintain an inverse media file frequency value for each descriptive term in the first database, (7) generate a descriptive term frequency matrix and generate a list of document vectors in multidimensional space (list), and (8) organize and process each media file in the list into a high activation score category and a low activation score category.

Abstract: Methods for identifying visual portions of visual media files responsive to search queries are provided. In one aspect, a method includes identifying, for each of a plurality of visual media files from a collection of media files, visual portions of the plurality of visual media files responsive to a group of search queries, and providing, in response to a search query from the group of search queries, an identifier of a visual portion of at least one of the plurality of visual media files for display as responsive to the search query. Systems and machine-readable media are also provided.

Abstract: Methods for providing in-browser perspective manipulation are provided. In one aspect, a method includes executing, in a web browser, an image editing application retrieved from a remote server. The method also includes providing for display, in a user interface of the image editing application, a background and an image. The method also includes modifying anchors of the image in response to user input. The method also includes determining an indicator for supporting hardware graphics acceleration in the web browser. When the indicator is true, the method includes calling a 3D perspective transformation function on the image based on the modified anchors. When the indicator is false, the method includes a 2D approximation that divides the image into smaller units for affine transformations, which are then recombined into the transformed image. The method also includes compositing an output image comprising the transformed image in front of the background.

Abstract: A method including receiving a compositional constraint from a user for merging multiple images to create a canvas, is provided. The method includes finding a vector for the canvas in a merged space associated with the compositional constraint, and generating a synthetic image for the canvas based on the vector for the canvas and a generative tool trained in a generative adversarial configuration against a discriminative tool. The method also includes providing the synthetic image for the canvas to a user when the discriminative tool identifies the synthetic image as a real image. A system and a non-transitory, computer readable medium storing instructions to perform the above method are also provided.

Abstract: Methods for style modification of images in search results are provided. In one aspect, a method includes receiving user input identifying a search query from a client device, in which the search query indicates one or more predetermined search terms. The subject system determines a first collection of images that correspond to the one or more predetermined search terms and a second collection of images that exclude images that correspond to the one or more predetermined search terms. The subject system modifies images of the second collection to apply a keyword style that corresponds to the one or more predetermined search terms, and provides a listing of images to the client device. The listing of images includes both the first collection of images and the images of the second collection that are modified with the applied keyword style. Systems and machine-readable media are also provided.

Abstract: A computer-implemented method is provided for retrieving an image from a user in a desired format and for detecting a compression efficiency for the image. When the compression efficiency is above a pre-selected threshold the computer-implemented method includes obtaining a saliency representation of the image, capturing a feature description of a non-salient portion of the image, flattening the non-salient portion in a new image, storing the new image in a selected format in a memory and storing a background descriptor for the image in the memory.

Abstract: Methods for predicting network access probability of data files accessible over a computer network are provided. In one aspect, a method includes generating a primary data vector for a media file based on a stored data representation of the file, and providing the data vector for the file to an algorithm that uses past interaction information for at least one other media file from a collection of media files having a degree of similarity with the media file above a threshold similarity value. The method also includes receiving, as an output of the algorithm, a marketability score for the media file, the score indicative of a likelihood that a user will download the media file. Systems and machine-readable media are also provided.

Abstract: Methods for prioritizing a set of images identified as responsive to an image search query from a user based on features of the images identified as relevant to a geographic region of the user are provided. In one aspect, the method includes submitting a plurality of images to a computer-operated convolutional neural network that is configured to analyze image pixel data for each of the plurality of images to identify features, in each of the plurality of images, influencing a download probability of the corresponding image in a plurality of geographic regions. The method also includes receiving, from the neural network and for each of the plurality of images, a download probability of each image for each of the plurality of geographic regions. Systems and machine-readable media are also provided.

Abstract: A method for receiving an activation request for an application element installed in a client device is provided. The method includes identifying one or more images from a first image database selected by a user in the client device and performing a reverse image search to identify one or more images in a second image database that are visually similar to the images from the first image database. The method also includes providing the one or more images in the second image database to the user via the client device and adding a user-selected image from the one or more images in the second image database to a collection in the second image database, the collection being associated with the user.

Abstract: A method for receiving a first user query from a user for searching an item, forming a first filter based on the first user query, and forming a first filtered item collection is provided. The method includes predicting a new query based on the first user query and a historical query log, forming a second filter for the new query, and applying the second filter to the first filtered item collection to form a second filtered item collection. Further, associating an item score to each of a plurality of items in the first and second filtered item collections, sorting the plurality of items in the first and second filtered item collections according to the item score associated to each of the plurality of items, and providing, to a user display, an item in the plurality of items in the first or second filtered item collections according to a sorting order.

Abstract: A method and system for displaying a dynamic mosaic of media files, the method comprising: receiving media files, determining a fill direction, constructing a layout, determining an edge set from the media files previously placed on the layout, determining a trailing edge, and adjacent edges, constructing a set of closed rectangles an open rectangle from the trailing edge and adjacent edges, providing a first set of frames by iteratively attempting to fill up to one closed rectangle from the set of closed rectangles, providing a second set of frames by filling the open rectangle by placing one searched media file in the open rectangle if no closed rectangle in the set of closed rectangles can be filled, and inserting one of the first set of frames and the second set of frames into a current layout before updating a current edge set based on the inserted frames.

Abstract: Techniques for synonym expansion are provided. An example method includes retrieving one or more multimedia items and one or more keywords associated with the multimedia items, wherein the retrieving is performed for the one or more multimedia items that are associated with both a query term and a synonym from a synonym pair, determining a first set of one or more keywords that are common to a group of multimedia items obtained based on the retrieving, determining, for each keyword from the first set of one or more keywords, a context score quantifying a value of the keyword for the synonym pair, based on the context score, determining from the first set of one or more keywords a second set of keywords that correlate higher to the synonym pair relative to other keywords from the second set of one or more keywords.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for composition aware image querying. A system may receive user input identifying a search query for content from a client device, where the user input indicates one or more queries assigned to one or more regions of a search input page. The system may generate a query vector for each query using a computer-operated neural language model. The system may compare the query vector to an indexed vector for each region of an image. The system may determine a listing of composition-specific images from a collection of images based on the comparison. The system may determine a ranking for each image in the listing of composition-specific images, and provide search results responsive to the search query to the client device. The search results may include a prioritized listing of the composition-specific images based on the determined ranking.

Abstract: Methods and systems for searching images in response to a keyword and color palette selection criteria. The method comprising: searching a first database to locate a set of pertinent images satisfying a keyword and color palette selection criteria; displaying the set of pertinent images and the set of colors; generating a set of suggested color palettes; displaying on the user interface the set of pertinent images which match the keyword and the single color specified by the single color option; displaying the set of pertinent images which match the combination specified by the first set of colors; dynamically extracting and displaying the color palette from a first image in response to receiving an input indicating a cursor hovering over the first image; and performing a first set of operations in response to receiving a user selected color palette extracted from the first image.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for watermarking an identification mark on an image. A system may provide an image to a trained convolutional neural network to generate a saliency map. The saliency map includes saliency information which identifies a salient region of the image and a non-salient region of the image. The system may be configured to determine a level of aggressiveness based on a weight model. The weight model includes information regarding a popularity of the image, a value of the image, a geographic location of the image, and a user account associated with the image. The system is configured to overlap the watermark with one of the identified salient region and the non-salient region based on the level of aggressiveness to generate a watermarked image.