Abstract: An encoder provides a bitstream having a coded current picture including a coded coding tree unit and signaling information. The signaling information indicating that geometric partitioning is enabled for the bitstream and further including partition index information enabling the decoder to determine endpoints for first and second straight line partition boundaries in the coding tree unit. The bitstream enables a compliant decoder to reconstruct the coding tree unit, which is partitioned into three non-rectangular regions by the first and second straight line partition boundaries.

Abstract: An encoder configured to generate an encoded bitstream for decoding by a compatible decoder, wherein the decoder receives the encoded bitstream with a current picture including a current block having a first partition boundary partitioning the block into first and second non-rectangular regions and a second partition boundary, intersecting the first partition boundary, and partitioning the second non-rectangular region into three portions. Using the encoded bitstream, the decoder determines a first predictor for use within the first non-rectangular region using a first motion vector selected from a first list of motion vector candidates, determines a second predictor for use within the second non-rectangular region using a second motion vector selected from a second list of motion vector candidates; and decodes the current block using the first predictor and the second predictor. Decoding further includes smoothing the first predictor and the second predictor across the first partition boundary.

Abstract: An encoder configured to encode a video bitstream for decoding by a compliant decoder configured to receive the bitstream, the encoded bitstream including a reference picture and a current picture including a coded current block, the bitstream also including an index useful for determining a scaling factor. The decoder determines that a mode when a reference picture and a current picture have different resolutions is enabled. Utilizing the mode, the decoder reconstructs the coded current block encoded by the encoder by determining a scaling factor using information in the bitstream including the index, determining a scaled prediction block by scaling a block of the reference picture using the scaling factor and by applying a pixel interpolation filter determined specifically for the prediction block and combining the scaled prediction block with a residual.

Abstract: A system and method for video and feature coding of neural-network structures used for multi-task machine learning includes an encoder, decoder and a decoder-compliant bitstream. A task-specific video decoder includes a first decoder receiving a bitstream having at least one feature and a description of a neural network backbone used to generate the bitstream, and a task-specific neural network head. The neural network head recreates the neural network backbone from the description, receives a feature from the bitstream a generates a task-specific output.

Abstract: Systems and methods for video coding of visual features that uses picture structures such as subpictures to independently encode individual features or groups of features are disclosed. An encoding method includes extracting a plurality of features from an image, representing each of the image features as a two-dimensional feature unit, grouping the feature units into at least one subpicture of the frame, and encoding the video frame into a bitstream. A compatible decoder for reconstructing the sequence of features is also provided.

Abstract: An image transmission system with privacy protection includes a privacy request processor which receives a request to secure at least one object type in an image prior to transmission of the image to a receiving system. An object detector receives the image data from a camera and processes the image data to identify at least one predetermined object type in the image data. An object scrambler receives a set of protected objects from the privacy request processor and receives the image data with identified objects of the predetermined object type and operates to obscure the image data for any identified objects identified as protected objects prior to encoding and transmission to a remote receiver site.

Abstract: A video surveillance system includes a plurality of cameras including an action recognition engine that classifies the video content to at least one of a plurality of predetermined actions in real time. The action recognition engine of a first camera communicates with at least one other action recognition engine, whereby detected actions and tasks related to the detected action can be exchanged. The cameras include a feature encoder operatively coupled to the action recognition engine that generates an encoded feature substream and a video encoder that receives the video content and provides an encoded video substream. A multiplexor receives the encoded feature substream and encoded video substream and generates an encoded camera bitstream including encoded video content and detected action content, which can be decoded at a receiver site for human and machine consumption.

Abstract: A method includes receiving a bit stream, decoding a first frame using the bit stream, determining a scaled first frame using the first frame and a scaling constant, storing the first frame in a first picture buffer at a first index location, and storing the scaled first frame in a second picture buffer at the first index location. Related apparatus, systems, techniques and articles are also described.

Abstract: A decoder is configured to receive a bitstream that has a reference picture, a current picture having a resolution different from the reference picture and including a first block and a second block that is an intra-coded block, and information useful for determining a scaling constant. The decoder is further configured to determine a scaling constant using the information in the bitstream, determine a scaled prediction block from the reference picture by determining a location of the scaled prediction block by scaling a motion vector component using the scaling constant, scaling the resolution of a block of the reference picture using the scaling constant and applying a filter determined specifically for the prediction block and not determined for the whole reference picture, reconstruct the first block using the scaled prediction block, and reconstruct the intra-coded second block without using a scaled prediction block.

Abstract: A method includes receiving a bit stream; determining, using the bit stream, whether an adaptive resolution management mode is enabled; determining, using the bit stream, a scaling factor including a vertical scaling constant component and a horizontal scaling constant component; and reconstructing pixel data of a current block in adaptive resolution management mode and using the determined scaling factor, the reconstructing including scaling the current block horizontally by the horizontal scaling constant component and scaling the current block vertically by the vertical scaling constant component. Related apparatus, systems, techniques and articles are also described.

Abstract: A decoder is configured to receive a bit stream including a current frame and a picture header associated with the entire current frame, determine, as a function of the picture header, that one global motion mode is enabled for the entire current frame, the enabled global motion mode being selected from a group including translational motion, 4-parameter affine motion, and 6-parameter affine motion, detect, based on the enabled global motion mode, a plurality of parameters applicable to the entire frame, and decode the current frame using the detected parameters.

Abstract: An encoder for video encoding and decoding with long term reference picture retention. The encoder generates an encoded bitream for a compliant decoder. At the decoder, coded pictures are decoded using a list of references pictures, of which at least one is a long term reference picture. A first picture is decoded using the long term reference picture. A second picture is decoded without using the long term reference picture which is inactive but is retained in a buffer. A subsequent picture is then decoded again using the long term reference picture.

Abstract: A system for low complexity and/or energy efficient encoding includes an encoding device, the encoding device configured to select a video application for an input video, set at least a boundary parameter, determine at least a goal parameter, generate a plurality of tool combinations, wherein each tool combination of the plurality of tool combinations has fewer tools than the full encoder toolset, tabulate measurements of each tool combination of the plurality of tool combinations, and select a tool combination of the plurality of tool combinations, wherein the selected tool combination generates optimal measurements as a function of the at least a goal parameter. An energy measurement device may be coupled to the encoding device and the selected tool combination may achieve the goal parameter at the highest energy efficiency.

Abstract: Systems and methods for motion information transfer from visual to feature domain are disclosed which provide for mapping motion information from coding units in video content to corresponding convolution unit(s) in feature content. Systems and methods are also provided for improved decoder-side motion vector refinement of video content based on characteristics of corresponding feature units.

Abstract: A decoder includes circuitry configured to receive a bitstream; determine a first region, a second region, and a third region of a current block and according to a geometric partitioning mode and decoding the current block. Related apparatus, systems, techniques and articles are also described. Decoder may determine, from a signal contained in the bitstream, a coding transformation type to decode the first region, the second region, and/or the third region, the coding transformation type characterizing at least an inverse block discrete cosine transformation and an inverse shape adaptive discrete cosine transformation, and the decoding of the current block may include using the determined transformation type for inverse transformation for each of the first region, the second region and/or the third region.

Abstract: A video encoder is provided that includes an image detection and segmentation processor receiving the video frame and generating object recognition data and at least one image segmentation mask. A mask to coding block mapping processor maps the segmentation mask to the CTUs of the video frame and partitions at least one CTU into a plurality of coding units (CUs) based on a detected object boundary in the CTU. A video encoding processor receives the video frame, the object recognition data and the partitioned CUs and encodes the CUs with at least one of a resolution or quantization parameter determined at least in part on whether the CU includes an object.

Abstract: Systems and methods for event and object detection and annotation in the video streams may include extracting a plurality of features in a picture in a video frame, grouping at least a portion of the plurality of features into at least one object, determining a region for the at least one object, assigning object identifiers to the at least one object and encoding the object identifiers into the bitstream. Feature-based rate distortion optimization may be employed for video coding including extracting a set of features from a picture in the video, generating a relevance map for the extracted features, determining a relevance score for portions of the picture using the relevance map, and encoding the portion of the picture with a bit rate determined at least in part by the relevance score.

Abstract: Aspects relate to systems and methods for optimizing a loss function for video coding for machines. An exemplary system includes a computing device including circuitry and configured to receive an input video, extract a feature map as a function of the input video and at least a feature extraction parameter, encode a feature layer as a function of the feature map, calculate a loss function as a function of the feature layer, and optimize the at least a feature extraction parameter as a function of the loss function.

Abstract: A system for predictive coding includes a computing device configured to receive an input video, determine a quality impact as a function of the input video, wherein determining the quality impact further comprises identifying a frame drop indicator as a function of the input video, and determining the quality impact as a function of the frame drop indicator, and produce an encoded video as a function of the quality impact and an encoding process.

Abstract: Systems and methods for scalable video coding for machines is provided. In an aspect, a decoder is provided which includes circuitry configured to receive a bitstream, the bitstream including at least a header, at least a base feature layer, and at least a residual visual layer. The decoder is configured to decode the at least a base feature layer, decode the at least a residual visual layer, combine the at least a decoded base feature layer with the at least a residual visual layer, and output a human-viewable video as a function of the combined at least a decoded base feature layer and the at least a residual visual layer.