Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a machine-learned object tracking policy. One of the methods includes receiving a current video frame by a user device having a plurality of installed object trackers, wherein each object tracker is configured to perform a different object tracking procedure on the current video frame rent video frame. The current video frame and one or more object tracks previously generated by the one or more object trackers are provided as input to a trained policy engine that implements a reinforcement learning model to generate a particular object tracking plan. A particular object tracking plan is selected based on the output of the reinforcement learning model, and the selected object tracking plan is performed on the current video frame to generate one or more updated object tracks for the current video frame.

Abstract: Methods and systems for performing black box optimization to identify an output that optimizes an objective.

Abstract: A computing system comprising one or more computing devices can obtain one or more images of an animal. The computing system can determine, using at least one of one or more machine-learned models, a plurality of joint positions associated with the animal based on the one or more images. The computing system can determine a body model for the animal. The computing system can estimate a body pose for the animal based on the one or more images, the plurality of joint positions, and the determined body model.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for semantically-consistent image style transfer. One of the methods includes: receiving an input source domain image; processing the source domain image using one or more source domain low-level encoder neural network layers to generate a low-level representation; processing the low-level representation using one more high-level encoder neural network layers to generate an embedding of the input source domain image; processing the embedding using one or more high-level decoder neural network layers to generate a high-level feature representation of features of the input source domain image; and processing the high-level feature representation of the features of the input source domain image using one or more target domain low-level decoder neural network layers to generate an output target domain image that is from the target domain but that has similar semantics to the input source domain image.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing visual tracking. In one aspect, a method comprises receiving: (i) one or more reference video frames, (ii) respective reference labels for each of a plurality of reference pixels in the reference video frames, and (iii) a target video frame. The reference video frames and the target video frame are processed using a colorization machine learning model to generate respective pixel similarity measures between each of (i) a plurality of target pixels in the target video frame, and (ii) the reference pixels in the reference video frames. A respective target label is determined for each target pixel in the target video frame, comprising: combining (i) the reference labels for the reference pixels in the reference video frames, and (ii) the pixel similarity measures.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing action localization in images and videos. In one aspect, a system comprises a data processing apparatus; a memory in data communication with the data processing apparatus and storing instructions that cause the data processing apparatus to perform image processing and video processing operations comprising: receiving an input comprising an image depicting a person; identifying a plurality of context positions from the image; determining respective feature representations of each of the context positions; providing a feature representation of the person and the feature representations of each of the context positions to a context neural network to obtain relational features, wherein the relational features represent relationships between the person and the context positions; and determining an action performed by the person using the feature representation of the person and the relational features.

Abstract: A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described.

Abstract: Systems and methods for transforming grayscale images into color images using deep neural networks are described. One of the systems include one or more computers and one or more storage devices storing instructions that, when executed by one or more computers, cause the one or more computers to implement a coloring neural network, a refinement neural network, and a subsystem. The coloring neural network is configured to receive a first grayscale image having a first resolution and to process the first grayscale image to generate a first color image having a second resolution lower than the first resolution. The subsystem processes the first color image to generate a set of intermediate image outputs. The refinement neural network is configured to receive the set intermediate image outputs, and to process the set of intermediate image outputs to generate a second color image having a third resolution higher than the second resolution.

Abstract: A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing action localization. In one aspect, a system comprises a data processing apparatus; a memory in data communication with the data processing apparatus and storing instructions that cause the data processing apparatus to perform operations comprising: receiving an input comprising an image depicting a person; identifying a plurality of context positions from the image; determining respective feature representations of each of the context positions; providing a feature representation of the person and the feature representations of each of the context positions to a context neural network to obtain relational features, wherein the relational features represent relationships between the person and the context positions; and determining an action performed by the person using the feature representation of the person and the relational features.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for implementing a machine-learned object tracking policy. One of the methods includes receiving a current video frame by a user device having a plurality of installed object trackers, wherein each object tracker is configured to perform a different object tracking procedure on the current video frame rent video frame. The current video frame and one or more object tracks previously generated by the one or more object trackers are provided as input to a trained policy engine that implements a reinforcement learning model to generate a particular object tracking plan. A particular object tracking plan is selected based on the output of the reinforcement learning model, and the selected object tracking plan is performed on the current video frame to generate one or more updated object tracks for the current video frame.

Abstract: Systems and methods provide distantly supervised wrapper induction for semi-structured documents, including automatically generating and annotating training documents for the wrapper. Training of the wrapper may occur in two phases using the training documents. An example method includes identifying a training set of semi-structured web pages having a subject entity that exists in a knowledge base and, for each training page, identifying target objects, identifying predicates in the knowledge base that connect the subject entity to a target objects identified in the training page, and annotating the training page. Annotating a training page includes generating a feature set for a mention of the target object, generating predicate-target object pairs for the mention, and labeling each predicate-target object pair with a corresponding example type and weight. The annotated training pages are used to train the wrapper to extract new subject entities and new facts from the set of semi-structured web pages.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing visual tracking. In one aspect, a method comprises receiving: (i) one or more reference video frames, (ii) respective reference labels for each of a plurality of reference pixels in the reference video frames, and (iii) a target video frame. The reference video frames and the target video frame are processed using a colorization machine learning model to generate respective pixel similarity measures between each of (i) a plurality of target pixels in the target video frame, and (ii) the reference pixels in the reference video frames. A respective target label is determined for each target pixel in the target video frame, comprising: combining (i) the reference labels for the reference pixels in the reference video frames, and (ii) the pixel similarity measures.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for semantically-consistent image style transfer. One of the methods includes: receiving an input source domain image; processing the source domain image using one or more source domain low-level encoder neural network layers to generate a low-level representation; processing the low-level representation using one more high-level encoder neural network layers to generate an embedding of the input source domain image; processing the embedding using one or more high-level decoder neural network layers to generate a high-level feature representation of features of the input source domain image; and processing the high-level feature representation of the features of the input source domain image using one or more target domain low-level decoder neural network layers to generate an output target domain image that is from the target domain but that has similar semantics to the input source domain image.

Abstract: A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described.

Abstract: Systems and methods for transforming grayscale images into color images using deep neural networks are described. One of the systems include one or more computers and one or more storage devices storing instructions that, when executed by one or more computers, cause the one or more computers to implement a coloring neural network, a refinement neural network, and a subsystem. The coloring neural network is configured to receive a first grayscale image having a first resolution and to process the first grayscale image to generate a first color image having a second resolution lower than the first resolution. The subsystem processes the first color image to generate a set of intermediate image outputs. The refinement neural network is configured to receive the set intermediate image outputs, and to process the set of intermediate image outputs to generate a second color image having a third resolution higher than the second resolution.

Abstract: Mechanisms are provided that: identify topics associated with a plurality of pieces of media content presented in a session; calculate a distance metric for pairs of topics, wherein each of the pairs of topics includes a first topic associated with a first piece of media content and a second topic associated with a second piece of media content, and wherein the second piece of media content was presented within a given span of the presentation of the first piece of media content; for each first topic of the pairs of topics, generate a rank-ordered list for all corresponding second topics; for each of the plurality of pieces of media content, generate a single rank-ordered list of all second topics; and for each of the plurality of pieces of media content, identify one or more other pieces of media content as recommended media content based on the single rank-ordered list.

Abstract: Mechanisms are provided that: identify topics associated with a plurality of pieces of media content presented in a session; calculate a distance metric for pairs of topics, wherein each of the pairs of topics includes a first topic associated with a first piece of media content and a second topic associated with a second piece of media content, and wherein the second piece of media content was presented within a given span of the presentation of the first piece of media content; for each first topic of the pairs of topics, generate a rank-ordered list for all corresponding second topics; for each of the plurality of pieces of media content, generate a single rank-ordered list of all second topics; and for each of the plurality of pieces of media content, identify one or more other pieces of media content as recommended media content based on the single rank-ordered list.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for combining authentication and application shortcut. An example method includes responsive to a user request identifying an entity: identifying a first time period associated with the entity based at least on a type of the entity; determining, within the first time period, a plurality of first candidate entities associated with the first entity; selecting first entities in the plurality of first candidate entities according to one or more selection criteria; and providing, for presentation to the user, first user-selectable graphical elements on a first graphical user-interactive timeline. Each first user-selectable graphical element identifies a corresponding first entity in the first entities.

Abstract: Mechanisms are provided that: identify topics associated with a plurality of pieces of media content presented in a session; calculate a distance metric for pairs of topics, wherein each of the pairs of topics includes a first topic associated with a first piece of media content and a second topic associated with a second piece of media content, and wherein the second piece of media content was presented within a given span of the presentation of the first piece of media content; for each first topic of the pairs of topics, generate a rank-ordered list for all corresponding second topics; for each of the plurality of pieces of media content, generate a single rank-ordered list of all second topics; and for each of the plurality of pieces of media content, identify one or more other pieces of media content as recommended media content based on the single rank-ordered list.