Abstract: Systems and methods for attribute recognition can include obtaining an image and a text string. The text string can be processed with a language model to generate a set of candidate attributes based on sequence based prediction. The image and the candidate attributes can be processed with an image-text model to determine a likelihood that the respective candidate attribute is depicted in the image. The likelihood determination can then be utilized to determine a predicted attribute for the object of interest.

Abstract: The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

Abstract: Methods, systems, and media for determining perceptual quality indicators of video content items are provided.

Abstract: Systems, devices, methods, and computer readable medium for evaluating visual quality of digital content are disclosed. Methods can include identifying content assets including one or more images that are combined to create different digital components distributed to one or more client devices. A quality of each of the one or more images is evaluated using one or more machine learning models trained to evaluate one or more visual aspects that are deemed indicative of visual quality. An aggregate quality for the content assets is determined based, at least in part, on an output of the one or more machine learning models indicating the visual quality of each of the one or more images. A graphical user interface of a first computing device is updated to present a visual indication of the aggregate quality of the content assets.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing a network input using a neural network to generate a network output. The neural network includes a normalization block that is between a first neural network layer and a second neural network layer in the neural network. Processing the network input using the neural network comprises: receiving a first layer output from the first neural network layer; processing data derived from the first layer output using standardization neural network layers of the normalization block to generate one or more adaptive standardization values; standardizing the first layer output using the adaptive standardization values to generate a standardized first layer output; generating a normalization block output from the standardized first layer output; and providing the normalization block output as an input to the second neural network layer.

Abstract: The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

Abstract: The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

Abstract: Methods, systems, and media for determining perceptual quality indicators of video content items are provided.

Abstract: Systems and methods of the present disclosure are directed to a method for generating a machine-learned multitask model configured to perform tasks. The method can include obtaining a machine-learned multitask search model comprising candidate nodes. The method can include obtaining tasks and machine-learned task controller models associated with the tasks. As an example, for a task, the method can include using the task controller model to route a subset of the candidate nodes in a machine-learned task submodel for the corresponding task. The method can include inputting task input data to the task submodel to obtain a task output. The method can include generating, using the task output, a feedback value based on an objective function. The method can include adjusting parameters of the task controller model based on the feedback value.

Abstract: The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

Abstract: Systems, devices, methods, and computer readable medium for evaluating visual quality of digital content are disclosed. Methods can include identifying content assets including one or more images that are combined to create different digital components distributed to one or more client devices. A quality of each of the one or more images is evaluated using one or more machine learning models trained to evaluate one or more visual aspects that are deemed indicative of visual quality. An aggregate quality for the content assets is determined based, at least in part, on an output of the one or more machine learning models indicating the visual quality of each of the one or more images. A graphical user interface of a first computing device is updated to present a visual indication of the aggregate quality of the content assets.

Abstract: Systems, devices, methods, and computer readable medium for evaluating visual quality of digital content are disclosed. Methods can include training machine learning models on images. A request is received to evaluate quality of an image included in a current version of a digital component generated by the computing device. The machine learning models are deployed on the image to generate a score for each quality characteristic of the image. A weight is assigned to each score to generate weighted scores. The weighted scores are combined to generate a combined score for the image. The combined score is compared to one or more thresholds to generate a quality of the image.

Abstract: The technology relates to navigating imagery that is organized into clusters based on common patterns exhibited when imagery is captured. For example, a set of captured images which satisfy a predetermined pattern may be determined. The images in the set of set of captured images may be grouped into one or more clusters according to the predetermined pattern. A request to display a first cluster of the one or more clusters may be received and, in response, a first captured image from the requested first cluster may be selected. The selected first captured image may then be displayed.

Abstract: The technology relates to navigating imagery that is organized into clusters based on common patterns exhibited when imagery is captured. For example, a set of captured images which satisfy a predetermined pattern may be determined. The images in the set of set of captured images may be grouped into one or more clusters according to the predetermined pattern. A request to display a first cluster of the one or more clusters may be received and, in response, a first captured image from the requested first cluster may be selected. The selected first captured image may then be displayed.

Abstract: The technology relates to navigating imagery that is organized into clusters based on common patterns exhibited when imagery is captured. For example, a set of captured images which satisfy a predetermined pattern may be determined. The images in the set of set of captured images may be grouped into one or more clusters according to the predetermined pattern. A request to display a first cluster of the one or more clusters may be received and, in response, a first captured image from the requested first cluster may be selected. The selected first captured image may then be displayed.

Abstract: The technology relates to navigating imagery that is organized into clusters based on common patterns exhibited when imagery is captured. For example, a set of captured images which satisfy a predetermined pattern may be determined. The images in the set of set of captured images may be grouped into one or more clusters according to the predetermined pattern. A request to display a first cluster of the one or more clusters may be received and, in response, a first captured image from the requested first cluster may be selected. The selected first captured image may then be displayed.