Abstract: Data representing a 3D object to be rendered is obtained at a user device, the data comprising: a mesh structure defining 3D co-ordinates for each of a plurality of object vertices; and a feature vector encoding visual characteristics for an object surface defined by the plurality of object vertices. An artificial neural network, ANN, is selected from a plurality of ANNs stored on the user device, each of the plurality of ANNs being configured to output pixel colour values for the object surface based on at least some of the visual characteristics encoded in the feature vector, wherein different ANNs have different numbers of layers and/or different numbers of parameters, wherein the selection is based on a resource characteristic of the user device. The mesh structure is processed using the selected ANN to generate a rendered image of the three-dimensional object.

Abstract: A method for encoding video data, comprising: receiving, at a machine learning model, encoding statistics derived from an encoding, performed by an external encoder, of at least one frame of a first video scene; processing the received encoding statistics using the machine learning model to determine one or more encoder settings for the external encoder; and outputting, from the machine learning model, the determined one or more encoder settings for use by the external encoder to encode a second video scene. The machine learning model is trained to predict, using encoding statistics input into the machine learning model, encoder settings which, when used by the external encoder to encode video data, optimise a data rate and/or video quality associated with the encoded video data.

Abstract: A computer-implemented method of transmitting video data. A sequence of video frames is received. A warp operation for a first frame and a reference frame of the sequence of video frames is determined, wherein the warp operation defines a transformation of the reference frame to give an approximation of the first frame. One or more regions of interest of the first frame are identified. Encoded image data from the image data of the one of more regions of interest of the first frame is generated using an image encoder. The warp operation and the encoded image data are transmitted.

Abstract: Data representing a 3D object to be rendered is obtained, the data comprising: a mesh structure defining 3D co-ordinates for each of a plurality of object vertices; and a feature vector encoding visual characteristics for at least one object surface defined by the plurality of object vertices. Motion information indicative of motion of the object in a scene is received. Based on the motion information, the mesh structure is deformed by adjusting the co-ordinates for one or more of the plurality of object vertices. The deformed mesh structure is processed using a graphics pipeline comprising a vertex shader and a fragment shader to generate a rendered image of the scene. The fragment shader comprises an artificial neural network trained to output pixel colour values for the at least one object surface on the basis of the visual characteristics encoded in the feature vector.

Abstract: A computer-implemented method for generating a rendered image of a three-dimensional object. A meta-storage component is used, that contains at least two pre-constructed grids of three-dimensional data grid points corresponding to features of three-dimensional visual representations of a plurality objects or scenes. A selection code is received, that represents the shape and appearance of the three-dimensional object. An instantiation of the three-dimensional object is constructed, using a selector component, by querying the meta storage component using the selection code to retrieve at least one combination of at least two pre-constructed grids of three-dimensional data grid points from the meta-storage component. A rendered image of the three-dimensional object is then generated using the instantiation of the three-dimensional object.

Abstract: A computer-implemented method of processing image data using a model of the human visual system. The model comprises a first artificial neural network system trained to generate the first output data using one or more differentiable functions configured to model the generation of signals from images by the human eye, and a second artificial neural network system trained to generate the second output data using one or more differentiable functions configured to model the processing of signals from the human eye by the human visual cortex. The method comprises receiving image data representing one or more images, processing the received image data using the first artificial neural network system to generate first output data, processing the first output data using a second artificial neural network system to generate second output data. Model output data is determined from the second output data, and output for use in an image processing process.

Abstract: A method of processing image data, comprising receiving, at a pre-processing artificial neural network, ANN, image data of one or more images, pre-processing the received image data at the pre-processing ANN to generate pre-processed image data of the one or more images, encoding and decoding, in accordance with an image or video codec, the pre-processed image data to generate decoded image data of the one or more images, and post-processing the decoded image data at a post-processing ANN to generate post-processed image data of the one or more images. The pre-processing ANN and the post-processing ANN are jointly trained in an end-to-end manner using a neural codec model arranged between the pre-processing ANN and the post-processing ANN, the neural codec model acting as a proxy for the image or video codec and comprising an ANN configured to emulate rate and distortion characteristics of the image or video codec.

Abstract: Image data representing one or more images at a first resolution is received at a first artificial neural network (ANN). The image data is processed using the first ANN to generate upscaled image data representing the one or more images at a second, higher resolution. The first ANN is trained to perform image upscaling and is trained using first training image data representing one or more training images at the first resolution, the first training image data being at a first level of quality. The first ANN is also trained using features of a second ANN, wherein the second ANN is trained to perform image upscaling and is trained using second training image data representing one or more training images at the first resolution, the second training image data being at a second level of quality, higher than the first level of quality.

Abstract: A method of processing image data is provided. Pixel data for a first image is preprocessed to identify a subset of the pixel data corresponding to a region of interest depicting a scene element. The subset of the pixel data is processed at a first encoder to generate a first data structure representative of the region of interest, the first data structure identifying the scene element depicted in the region of interest. The subset of pixel data is also processed at a second encoder to generate a second data structure representative of the region of interest, the second data structure comprising values for visual characteristics associated with the scene element. The first and second data structures are outputted for use by a decoder to generate a second image approximating the region of interest of the first image.

Abstract: A method is provided for generating a rendered image of a scene, the scene comprising one or more scene assets. A selected one of first scene asset data and second scene asset data is transmitted from a server to a user device, to enable the user device to generate a rendered image of the scene. The first scene asset data is useable by the user device to render the scene asset. The second scene asset data represents a rendering of the scene asset generated by the server. The selection is performed on the basis of a resource characteristic of the user device.

Abstract: A method of processing, prior to encoding using an external encoder, image data using an artificial neural network is provided. The external encoder is operable in a plurality of encoding modes. At the neural network, image data representing one or more images is received. The image data is processed using the neural network to generate output data indicative of an encoding mode selected from the plurality of encoding modes of the external encoder. The neural network trained to select using image data an encoding mode of the plurality of encoding modes of the external encoder using one or more differentiable functions configured to emulate an encoding process. The generated output data is outputted from the neural network to the external encoder to enable the external encoder to encode the image data using the selected encoding mode.

Abstract: A computer-implemented method of transmitting video data. A sequence of video frames is received. A warp operation for a first frame and a reference frame of the sequence of video frames is determined, wherein the warp operation defines a transformation of the reference frame to give an approximation of the first frame. One or more regions of interest of the first frame are identified. Encoded image data from the image data of the one of more regions of interest of the first frame is generated using an image encoder. The warp operation and the encoded image data are transmitted.

Abstract: Image data of a first image in a sequence of images is processed using an artificial neural network (ANN) to generate output image data indicative of an alignment of the first image with a second image in the sequence. The ANN is trained using outputs of an alignment pipeline configured to perform alignment of images. The alignment pipeline is configured to determine flow vectors representing optical flow between images, and perform an image transformation using the flow vectors to align the images. The ANN is trained to emulate a result derivable using the alignment pipeline.

Abstract: Image data representing one or more images at a first resolution is received at a first artificial neural network (ANN). The image data is processed using the first ANN to generate upscaled image data representing the one or more images at a second, higher resolution. The first ANN is trained to perform image upscaling and is trained using first training image data representing one or more training images at the first resolution, the first training image data being at a first level of quality. The first ANN is also trained using features of a second ANN, wherein the second ANN is trained to perform image upscaling and is trained using second training image data representing one or more training images at the first resolution, the second training image data being at a second level of quality, higher than the first level of quality.

Abstract: A method of processing a time-varying signal in a signal processing system. Data representative of one or more first time samples of the time-varying signal is received at an artificial neural network, ANN. The received data is processed using the ANN to generate predicted data representative of a second time sample of the time-varying signal, the second time sample being later than the one or more first time samples. The ANN is trained to predict data representative of time samples of time-varying signals based on data representative of earlier time samples of the time-varying signals. The signal processing system processes the predicted data representative of the second time sample in place of a third time sample of the time-varying signal, the third time sample being earlier than the second time sample.

Abstract: A computer-implemented method of processing image data using a model of the human visual system. The model comprises a first artificial neural network system trained to generate the first output data using one or more differentiable functions configured to model the generation of signals from images by the human eye, and a second artificial neural network system trained to generate the second output data using one or more differentiable functions configured to model the processing of signals from the human eye by the human visual cortex. The method comprises receiving image data representing one or more images, processing the received image data using the first artificial neural network system to generate first output data, processing the first output data using a second artificial neural network system to generate second output data. Model output data is determined from the second output data, and output for use in an image processing process.

Abstract: A method of processing image data is provided. Pixel data for a first image is preprocessed to identify a subset of the pixel data corresponding to a region of interest depicting a scene element. The subset of the pixel data is processed at a first encoder to generate a first data structure representative of the region of interest, the first data structure identifying the scene element depicted in the region of interest. The subset of pixel data is also processed at a second encoder to generate a second data structure representative of the region of interest, the second data structure comprising values for visual characteristics associated with the scene element. The first and second data structures are outputted for use by a decoder to generate a second image approximating the region of interest of the first image.

Abstract: Certain aspects of the present disclosure provide techniques for encoding image data for one or more images. In one embodiment, a method includes the steps of downscaling the one or more images, and encoding the one or more downscaled images using an image codec. Another embodiment concerns a computer-implemented method of decoding encoded image data, and a computer-implemented method of encoding and decoding image data.

Abstract: A method of processing, prior to encoding using an external encoder, image data using an artificial neural network is provided. The external encoder is operable in a plurality of encoding modes. At the neural network, image data representing one or more images is received. The image data is processed using the neural network to generate output data indicative of an encoding mode selected from the plurality of encoding modes of the external encoder. The neural network trained to select using image data an encoding mode of the plurality of encoding modes of the external encoder using one or more differentiable functions configured to emulate an encoding process. The generated output data is outputted from the neural network to the external encoder to enable the external encoder to encode the image data using the selected encoding mode.

Abstract: Certain aspects of the present disclosure provide techniques for preprocessing, prior to encoding with an external encoder, image data using a preprocessing network comprising a set of inter-connected weights is provided. At the preprocessing network, image data from one or more images is received. The image data is processed using the preprocessing network to generate an output pixel representation for encoding with the external encoder. The weights of the preprocessing network are trained to optimize a combination of at least one quality score indicative of the quality of the output pixel representation and a rate score indicative of the bits required by the external encoder to encode the output pixel representation.