Abstract: Subsets of training data are selected for iterations of a statistical model through a training process. The selection can reduce the amount of data to be processed by selecting the training data that will likely have significant training value for the pass. This can include using a metric such as the loss or certainty to sample the data, such that easy to classify instances are used for training less frequently than harder to classify instances. A cutoff value or threshold can also, or alternatively, be used such that harder to classify instances are not selected for training until later in the process when the model may be more likely to benefit from training on those instances. Sampling can vary between passes for variety, and the cutoff value might also change such that all data instances are eligible for training selection by at least the last iteration.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.

Abstract: Techniques for performing learnt image compression and object detection using compressed image data are described. A system may perform image compression using an image compression model that includes an encoder, an entropy model, and a decoder. The encoder, the entropy model, and the decoder may be jointly trained using machine learning based on training data. After training, the encoder and the decoder may be separated to encode image data to generate compressed image data or to decode compressed image data to generate reconstructed image data. In addition, the system may perform object detection using a compressed object detection model that processes compressed image data generated by the image compression model. For example, the compressed object detection model may perform partial decoding using a single layer of the decoder and perform compressed object detection on the partially decoded image data.

Abstract: The present application relates to a multi-stage encoder/decoder system that provides image compression using hierarchical auto-regressive models and saliency-based masks. The multi-stage encoder/decoder system includes a first stage and a second stage of a trained image compression network, such that the second stage, based on the image compression performed by the first stage, identify certain redundancies that can be removed from the bit string to reduce the storage and bandwidth requirements. Additionally, by using saliency-based masks, distortions in different sections of the image can be weighted differently to further improve the image compression performance.

Abstract: Techniques for learned lossy image compression are described. A system may perform image compression using an image compression model that includes an encoder to compress an image and a decoder to reconstruct the image. The encoder and the decoder are trained using machine learning techniques. After training, the encoder can encode image data to generate compressed image data and the decoder can decode compressed image data to generate reconstructed image data.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.

Abstract: Various embodiments of a framework which allow, as an alternative to resource-taxing decompression, efficient computation of feature maps using a compressed content data subset, such as video, by exploiting the motion information, such as a motion vector, present in the compressed video. This framework allows frame-specific object recognition and action detection algorithms to be applied to compressed video and other media files by executing only on I-frames in a Group of Pictures and linearly interpolating the results. Training and machine learning increases recognition accuracy. Yielding significant computational gains, this approach accelerates frame-wise feature extraction I-frame/P-frame/P-frame videos as well as I-frame/P-frame/B-frame videos. The present techniques may also be used for segmentation to identify and label respective regions for objects in a video.

Abstract: Images may be analyzed to determine a visually cohesive color palette, for example by comparing a subset of the colors most frequently appearing in the image to a plurality of color schemes (e.g., complementary, analogous, etc.), and potentially modifying one or more of the subset of colors to more accurately fit the selected color scheme. Various regions of the image are selected and portions of the regions having one or more colors of the color palette are extracted and classified to generate and compare feature vectors of the patches to previously-determined feature vectors of items to identify visually similar items. The visually similar items are selected for presentation in various ways, such as by choosing an outfit of visually-similar apparel items based on the locations of the corresponding colors in the image, etc.

Abstract: Approaches provide for identifying text represented in image data as well as determining a location or region of the image data that includes the text represented in the image data. For example, a camera of a computing device can be used to capture a live camera view of one or more items. The live camera view can be presented to the user on a display screen of the computing device. An application executing on the computing device or at least in communication with the computing device can analyze the image data of the live camera view to identify text represented in the image data as well as determine locations or regions of the image that include the representations.