Abstract: Systems and methods are provided for encoding and decoding video data for machine applications, such as Video Coding for Machines, using inference models. An encoder uses an inference selector to determine an appropriate inference model to encode a feature substream. The encoder further employs an inference metadata encoder to encode parameters of the selected inference model into an inference metadata substream, which can be multiplexed with the feature substream to generate an encoded bitstream to send to a decoder site. A decoder receiving the encoded bitstream, extracts the inference metadata, selects the appropriate inference model, and applies that inference model in decoding the feature substream and generating a decoded output signal for machine consumption.

Abstract: A video encoder is provided which generates a bitstream for decoding by a compliant decoder having an interprediction mode for generating a predictor from a reference picture for a subsequent picture having a spatial resolution different from the reference picture. The encoder provides a bitstream in which the decoder can scale the spatial resolution of a block of pixels of the reference picture using a scale factor signaled in the encoded bitstream and using a filter chosen for the particular block of the reference picture and not the reference picture as a whole to form a predictor.

Abstract: A decoder includes circuitry configured to receive a bitstream, identify a first frame, locate a first independent reference region within the first frame, extract the first independent reference region from the first frame, decode a second frame using the first independent reference region as a reference for the second frame.

Abstract: Systems and methods for encoding and decoding video for machine consumption (video coding for machines) are provided in which an autoencoder is employed. The autoencoder has an encoder portion, a bottleneck portion, and a decoder portion. The autoencoder being distributed between a VCM encoder and VCM decoder such that the VCM encoder includes the encoder portion and bottleneck portion and the VCM decoder includes the bottleneck portion and the decoder portion.

Abstract: An encoder provides a bitstream having a coded current picture including a coded coding tree unit and signaling information. The signaling information indicating that geometric partitioning is enabled for the bitstream and further including partition index information enabling the decoder to determine endpoints for first and second straight line partition boundaries in the coding tree unit. The bitstream enables a compliant decoder to reconstruct the coding tree unit, which is partitioned into three non-rectangular regions by the first and second straight line partition boundaries.

Abstract: An encoder configured to encode a video bitstream for decoding by a compliant decoder configured to receive the bitstream, the encoded bitstream including a reference picture and a current picture including a coded current block, the bitstream also including an index useful for determining a scaling factor. The decoder determines that a mode when a reference picture and a current picture have different resolutions is enabled. Utilizing the mode, the decoder reconstructs the coded current block encoded by the encoder by determining a scaling factor using information in the bitstream including the index, determining a scaled prediction block by scaling a block of the reference picture using the scaling factor and by applying a pixel interpolation filter determined specifically for the prediction block and combining the scaled prediction block with a residual.

Abstract: An encoder configured to generate an encoded bitstream for decoding by a compatible decoder, wherein the decoder receives the encoded bitstream with a current picture including a current block having a first partition boundary partitioning the block into first and second non-rectangular regions and a second partition boundary, intersecting the first partition boundary, and partitioning the second non-rectangular region into three portions. Using the encoded bitstream, the decoder determines a first predictor for use within the first non-rectangular region using a first motion vector selected from a first list of motion vector candidates, determines a second predictor for use within the second non-rectangular region using a second motion vector selected from a second list of motion vector candidates; and decodes the current block using the first predictor and the second predictor. Decoding further includes smoothing the first predictor and the second predictor across the first partition boundary.

Abstract: Systems and methods for video coding of visual features that uses picture structures such as subpictures to independently encode individual features or groups of features are disclosed. An encoding method includes extracting a plurality of features from an image, representing each of the image features as a two-dimensional feature unit, grouping the feature units into at least one subpicture of the frame, and encoding the video frame into a bitstream. A compatible decoder for reconstructing the sequence of features is also provided.

Abstract: A system and method for video and feature coding of neural-network structures used for multi-task machine learning includes an encoder, decoder and a decoder-compliant bitstream. A task-specific video decoder includes a first decoder receiving a bitstream having at least one feature and a description of a neural network backbone used to generate the bitstream, and a task-specific neural network head. The neural network head recreates the neural network backbone from the description, receives a feature from the bitstream a generates a task-specific output.

Abstract: An image transmission system with privacy protection includes a privacy request processor which receives a request to secure at least one object type in an image prior to transmission of the image to a receiving system. An object detector receives the image data from a camera and processes the image data to identify at least one predetermined object type in the image data. An object scrambler receives a set of protected objects from the privacy request processor and receives the image data with identified objects of the predetermined object type and operates to obscure the image data for any identified objects identified as protected objects prior to encoding and transmission to a remote receiver site.

Abstract: A video surveillance system includes a plurality of cameras including an action recognition engine that classifies the video content to at least one of a plurality of predetermined actions in real time. The action recognition engine of a first camera communicates with at least one other action recognition engine, whereby detected actions and tasks related to the detected action can be exchanged. The cameras include a feature encoder operatively coupled to the action recognition engine that generates an encoded feature substream and a video encoder that receives the video content and provides an encoded video substream. A multiplexor receives the encoded feature substream and encoded video substream and generates an encoded camera bitstream including encoded video content and detected action content, which can be decoded at a receiver site for human and machine consumption.

Abstract: A method includes receiving a bit stream, decoding a first frame using the bit stream, determining a scaled first frame using the first frame and a scaling constant, storing the first frame in a first picture buffer at a first index location, and storing the scaled first frame in a second picture buffer at the first index location. Related apparatus, systems, techniques and articles are also described.

Abstract: A decoder is configured to receive a bitstream that has a reference picture, a current picture having a resolution different from the reference picture and including a first block and a second block that is an intra-coded block, and information useful for determining a scaling constant. The decoder is further configured to determine a scaling constant using the information in the bitstream, determine a scaled prediction block from the reference picture by determining a location of the scaled prediction block by scaling a motion vector component using the scaling constant, scaling the resolution of a block of the reference picture using the scaling constant and applying a filter determined specifically for the prediction block and not determined for the whole reference picture, reconstruct the first block using the scaled prediction block, and reconstruct the intra-coded second block without using a scaled prediction block.

Abstract: A method includes receiving a bit stream; determining, using the bit stream, whether an adaptive resolution management mode is enabled; determining, using the bit stream, a scaling factor including a vertical scaling constant component and a horizontal scaling constant component; and reconstructing pixel data of a current block in adaptive resolution management mode and using the determined scaling factor, the reconstructing including scaling the current block horizontally by the horizontal scaling constant component and scaling the current block vertically by the vertical scaling constant component. Related apparatus, systems, techniques and articles are also described.

Abstract: A decoder is configured to receive a bit stream including a current frame and a picture header associated with the entire current frame, determine, as a function of the picture header, that one global motion mode is enabled for the entire current frame, the enabled global motion mode being selected from a group including translational motion, 4-parameter affine motion, and 6-parameter affine motion, detect, based on the enabled global motion mode, a plurality of parameters applicable to the entire frame, and decode the current frame using the detected parameters.

Abstract: An encoder for video encoding and decoding with long term reference picture retention. The encoder generates an encoded bitream for a compliant decoder. At the decoder, coded pictures are decoded using a list of references pictures, of which at least one is a long term reference picture. A first picture is decoded using the long term reference picture. A second picture is decoded without using the long term reference picture which is inactive but is retained in a buffer. A subsequent picture is then decoded again using the long term reference picture.

Abstract: A system for low complexity and/or energy efficient encoding includes an encoding device, the encoding device configured to select a video application for an input video, set at least a boundary parameter, determine at least a goal parameter, generate a plurality of tool combinations, wherein each tool combination of the plurality of tool combinations has fewer tools than the full encoder toolset, tabulate measurements of each tool combination of the plurality of tool combinations, and select a tool combination of the plurality of tool combinations, wherein the selected tool combination generates optimal measurements as a function of the at least a goal parameter. An energy measurement device may be coupled to the encoding device and the selected tool combination may achieve the goal parameter at the highest energy efficiency.

Abstract: Systems and methods for motion information transfer from visual to feature domain are disclosed which provide for mapping motion information from coding units in video content to corresponding convolution unit(s) in feature content. Systems and methods are also provided for improved decoder-side motion vector refinement of video content based on characteristics of corresponding feature units.

Abstract: A decoder includes circuitry configured to receive a bitstream; determine a first region, a second region, and a third region of a current block and according to a geometric partitioning mode and decoding the current block. Related apparatus, systems, techniques and articles are also described. Decoder may determine, from a signal contained in the bitstream, a coding transformation type to decode the first region, the second region, and/or the third region, the coding transformation type characterizing at least an inverse block discrete cosine transformation and an inverse shape adaptive discrete cosine transformation, and the decoding of the current block may include using the determined transformation type for inverse transformation for each of the first region, the second region and/or the third region.

Abstract: A video encoder is provided that includes an image detection and segmentation processor receiving the video frame and generating object recognition data and at least one image segmentation mask. A mask to coding block mapping processor maps the segmentation mask to the CTUs of the video frame and partitions at least one CTU into a plurality of coding units (CUs) based on a detected object boundary in the CTU. A video encoding processor receives the video frame, the object recognition data and the partitioned CUs and encodes the CUs with at least one of a resolution or quantization parameter determined at least in part on whether the CU includes an object.