Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating realistic extensions of images. In one aspect, a method comprises providing an input that comprises a provided image to a generative neural network having a plurality of generative neural network parameters. The generative neural network processes the input in accordance with trained values of the plurality of generative neural network parameters to generate an extended image. The extended image has (i) more rows, more columns, or both than the provided image, and (ii) is predicted to be a realistic extension of the provided image. The generative neural network is trained using an adversarial loss objective function.

Abstract: The disclosure provides a multi-object tracking algorithm based on an object detection and feature extraction combination model, including the following steps: S1, adding an object appearance feature extraction network layer behind a prediction feature layer of an object detection tracking network having an FPN structure; S2, calculating object fused loss of the object detection tracking network having the FPN structure and added with the object appearance feature extraction network layer; S3, forming a feature comparison database utilizing a neural network during multi-frame objection detection and tracking process; and S4, comparing current image object appearance features with features in the feature comparison database, drawing an object trajectory if the objects are uniform; else adding the current image object appearance features into the feature comparison database to form a new feature comparison database, and then repeating steps S2 and S3.

Abstract: A device including one or more processors configured to, based on at least one of an external criterion or a signal-based criterion, output a first gain adjusted signal from a first gain adjuster or a second gain adjusted signal from a second gain adjuster as an input to an audio processing operation. The first gain adjusted signal corresponds to a first gain applied to an audio signal from a microphone and the second gain adjusted signal corresponding to a second gain applied to an audio signal from a microphone. The signal-based criterion is based on whether a first signal metric associated with the first gain adjusted signal or a second signal metric associated with the second gain adjusted signal is closer to a target metric.

Abstract: A device including one or more processors configured to, based on at least one of an external criterion or a signal-based criterion, output a first gain adjusted signal from a first gain adjuster or a second gain adjusted signal from a second gain adjuster as an input to an audio processing operation. The first gain adjusted signal corresponds to a first gain applied to an audio signal from a microphone and the second gain adjusted signal corresponding to a second gain applied to an audio signal from a microphone. The signal-based criterion is based on whether a first signal metric associated with the first gain adjusted signal or a second signal metric associated with the second gain adjusted signal is closer to a target metric.

Abstract: Systems and methods of automatically extracting summaries of video content are described herein. A data processing system can access, from a video database, a first video content element including a first plurality of frame. The data processing system can select an intervallic subset of the first plurality of frames of the first video content element. The data processing system can calculate, for each of a plurality of further subsets comprising a predetermined number of frames from the intervallic subset, a score for the further subset. The data processing system can identify, from the plurality of further subsets, a further subset having a highest score. The data processing system can select a portion of the first video content element comprising the frames of the further subset having the highest score. The data processing system can generate a second video content element comprising the selected portion of the first video content element.

Abstract: Systems and methods of automatically extracting summaries of video content are described herein. A data processing system can access, from a video database, a first video content element including a first plurality of frame. The data processing system can select an intervallic subset of the first plurality of frames of the first video content element. The data processing system can calculate, for each of a plurality of further subsets comprising a predetermined number of frames from the intervallic subset, a score for the further subset. The data processing system can identify, from the plurality of further subsets, a further subset having a highest score. The data processing system can select a portion of the first video content element comprising the frames of the further subset having the highest score. The data processing system can generate a second video content element comprising the selected portion of the first video content element.

Abstract: Systems and methods of automatically extracting summaries of video content are described herein. A data processing system can access, from a video database, a first video content element including a first plurality of frame. The data processing system can select an intervallic subset of the first plurality of frames of the first video content element. The data processing system can calculate, for each of a plurality of further subsets comprising a predetermined number of frames from the intervallic subset, a score for the further subset. The data processing system can identify, from the plurality of further subsets, a further subset having a highest score. The data processing system can select a portion of the first video content element comprising the frames of the further subset having the highest score. The data processing system can generate a second video content element comprising the selected portion of the first video content element.

Abstract: Methods and apparatus to reduce compression artifacts in images are disclosed herein. An example method includes separating at least a portion of an image into a first component and a second component, reducing a first artifact in the first component to form a first cleaned component, reducing, using the first cleaned component, a second artifact in the second component to form a second cleaned component, and combining the first cleaned component and the second cleaned component to form a cleaned image.

Abstract: Methods, systems, and apparatus, include computer programs encoded on a computer-readable storage medium, for determining keywords for an image that supports an overlay content item. A method includes identifying, using one or more processors, an image that is to support an overlay content item, the image being presented on a web site and including a portion that is designated as being enabled to receive and display the overlay content item; evaluating pixel data associated with the image including determining one or more labels that are associated with content included within the image; and determining one or more keywords for the image based at least in part on the one or more labels.

Abstract: Methods, systems and apparatus for choosing image labels. In one aspect, a method includes receiving data specifying a first image, receiving text labels for the first image, receiving search results in response to a web search performed using at least some of the text labels as queries, ranking the text labels, at least in part, based on a number of resources referenced by the received search results, wherein at least some of the resources each include an image matching the first image, and selecting an image label for the image from the ranked text labels, the image label being selected based on the ranking.

Abstract: Methods, systems, and apparatus, include computer programs encoded on a computer-readable storage medium, for determining keywords for an image that supports an overlay content item. A method includes identifying, using one or more processors, an image that is to support an overlay content item, the image being presented on a web site and including a portion that is designated as being enabled to receive and display the overlay content item; evaluating pixel data associated with the image including determining one or more labels that are associated with content included within the image; and determining one or more keywords for the image based at least in part on the one or more labels.

Abstract: Methods, systems, and apparatus, include computer programs encoded on a computer-readable storage medium, for determining keywords for an image that supports an overlay content item. A method includes identifying, using one or more processors, an image that is to support an overlay content item, the image being presented on a web site and including a portion that is designated as being enabled to receive and display the overlay content item; evaluating pixel data associated with the image including determining one or more labels that are associated with content included within the image; and determining one or more keywords for the image based at least in part on the one or more labels.

Abstract: Methods, systems and apparatus for choosing image labels. In one aspect, a method includes receiving data specifying a first image, receiving text labels for the first image, receiving search results in response to a web search performed using at least some of the text labels as queries, ranking the text labels, at least in part, based on a number of resources referenced by the received search results, wherein at least some of the resources each include an image matching the first image, and selecting an image label for the image from the ranked text labels, the image label being selected based on the ranking.

Abstract: Methods, systems and apparatus for choosing image labels. In one aspect, a method includes receiving data specifying a first image, receiving text labels for the first image, receiving search results in response to a web search performed using at least some of the text labels as queries, ranking the text labels, at least in part, based on a number of resources referenced by the received search results, wherein at least some of the resources each include an image matching the first image, and selecting an image label for the image from the ranked text labels, the image label being selected based on the ranking.

Abstract: Methods, systems and apparatus for choosing image labels. In one aspect, a method includes receiving data specifying a first image, receiving text labels for the first image, receiving search results in response to a web search performed using at least some of the text labels as queries, ranking the text labels, at least in part, based on a number of resources referenced by the received search results, wherein at least some of the resources each include an image matching the first image, and selecting an image label for the image from the ranked text labels, the image label being selected based on the ranking.

Abstract: Methods, systems, and apparatus, including computer program products, for ranking images are disclosed. An image search subsystem generates an adjustment factor representative of a quality measure of an image relative to a search query. The quality represents a relevance of the image to the query. The adjustment factor can be computed based on relevance data for the image to the query and image similarity data representing a relative similarity between the image and other images relevant to the query. The relevance data can be based on user actions in response to the image being included in search results for the query. The adjustment factor can be scaled based on whether the relevance data and the image similarity data both indicate that the image is relevant to the search query. A relevance score is computed based on the adjustment factor (e.g., a product of the adjustment factor and relevance score).

Abstract: Methods, systems and apparatus for identifying result images for a query image. One or more labels that are associated with the query image are obtained. Candidate images matching the query labels are identified. Visual similarity scores are generated for the candidate images. Each visual similarity score represents the visual similarity of a respective candidate image to the query image. Relevance scores are generated for each of the candidate images based on the visual similarity scores. Each relevance score represents a measure of relevance of the respective candidate images to the query image. The candidate images are ranked based on the relevance scores, a highest ranking subset of the candidate images being identified as result images and referenced by image search results. The result images can be candidate images that satisfy a similarity condition relative to the query image and other result images.

Abstract: In general, in one aspect, a method includes receiving, by the one or more processors, from a client device, data specifying a query image, requesting, by the one or more processors, text labels responsive to the query image from a database of text labels associated with images, in response to no zero text labels that meet a threshold being returned for the request, sending, by the one or more processors, request data to the client device that causes the client device to display a prompt to enter a textual search query, receiving, in response to the request data, a textual search query from the client device, and in the database of text labels associated with images, associating, by the one or more processors, the textual search query with the query image. This and other embodiments of this aspect can include corresponding systems, apparatus, and computer program products.

Abstract: In general, in one aspect, a method includes receiving, by one or more processors, a query image, obtaining, by the one or more processors, search results responsive to the query image, each search result identifying a corresponding web page containing an image that is determined to be responsive to the query image, the search results being ordered in a first arrangement, obtaining, by the one or more processors, text labels associated with the query image, and generating, by the one or more processors a second arrangement of the search results by ordering the search results based on a presence in the respective corresponding web pages of one or more of the text labels associated with the query image. This and other implementations of this aspect can include corresponding systems, apparatus, and computer program products.

Abstract: In general, in one aspect, a method includes receiving a query image, obtaining a set of text labels associated with the query image, each text label associated with a confidence score that the text label accurately describes a topic of the query image, selecting a first text label associated with the query image, the first text label having a highest confidence score among the set of text labels, selecting a subset of the set of text labels that have a confidence score greater than a confidence score threshold, obtaining search results responsive to the query image, and in response to at least one of the search results identifying a web page containing the first text label, sending data that causes a client device to display the subset of text labels with the search results.