Abstract: This disclosure provides methods, devices, and systems for video coding. The present implementations more specifically relate to hybrid coding techniques that combine aspects of inter-frame coding with aspects of intra-frame coding. In some aspects, a video encoder may perform inter-frame coding in a weighted manner so that the coded video frames (also referred to as “residual frames”) may include contributions from the current video frame to be transmitted over a communication channel and also may include contributions from the previous video frame transmitted over the communication channel. More specifically, any pixel value (r(n)) in the residual frame can be expressed as a weighted combination of a respective pixel value (x(n)) in the current video frame and a co-located pixel value (x(n?1)) in the previous video frame, where r(n)=x(n)??·x(n?1) and where 0???1 is a scaling factor representing the degree of contribution by the previous video frame.

Abstract: A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data is divided into a plurality of data layers of increasing depths as the distance between a given layer and the plane of the display increases. Data layers are sampled using a plenoptic sampling scheme and rendered using hybrid rendering, such as perspective and oblique rendering, to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.

Abstract: A method of video processing includes performing a conversion between a video including a picture and a bitstream of the video, wherein the bitstream conforms to a format rule, wherein the format rule specifies that an indication of whether a first flag is signalled at a beginning of a picture header associated with the picture, wherein the first flag is indicative of whether the picture is an intra random access point (IRAP) picture or a gradual decoding refresh (GDR) picture.

Abstract: Provided are a depth image processing method, a depth image processing apparatus, an electronic device and a readable storage medium. The method includes: (101) obtaining consecutive n depth image frames; (102) determining a trusted pixel and determining a smoothing factor corresponding to the trusted pixel; (103) determining a time similarity weight; (104) determining a content similarity; (105) determining a content similarity weight based on the content similarity and the smoothing factor; and (106) performing filtering processing on a depth value of the trusted pixel based on all time similarity weights and all content similarity weights.

Abstract: Video coding tools can be controlled by including syntax in a video bitstream that makes better use of video decoding resources. An encoder inserts syntax into a video bitstream to enable a decoder to parse the bitstream and easily control which tools combinations are enabled, which combinations are not permitted, and which tools are activated for various components in a multiple component bitstream, leading to potential parallelization of bitstream decoding.

Abstract: A method of decoding a coded video bitstream is provided. The method includes parsing a slice header of a current slice represented in the coded video bitstream. The slice header includes a reference picture list structure. The method also includes deriving, based on the reference picture list structure, a reference picture list of the current slice; and obtaining, based on the reference picture list, at least one reconstructed block of the current slice.

Abstract: An image signal processor includes a register and a disparity correction unit. The register stores disparity data obtained from a pattern image data that an image senor generates, and the image sensor includes a plurality of pixels, and each of the pixel includes at least a first photoelectric conversion element and a second photoelectric conversion element. The image sensor generates the pattern image data in response to a pattern image located at a first distance from the image sensor. The disparity correction unit corrects a disparity distortion of an image data based on the disparity data to generate a result image data, and the image senor generates the image data by capturing an object.

Abstract: A method and apparatus for encoding a video stream using video point cloud coding, the decoding including obtaining an input point cloud; dividing the input point cloud into a plurality of chunks, including a first chunk including a first plurality of points and a second chunk including a second plurality of points; generating a first plurality of patches based on the first plurality of points; generating a second plurality of patches based on the second plurality of points; packing the first plurality of patches and the second plurality of patches into an image; and generating the video stream based on the image.

Abstract: Disclosed here is a method of generating a dental recommendation based on image processing. Further, the method may include receiving at least one patient data comprising at least one image from at least one patient device. Further, the method may include retrieving at least one dental dataset. Further, the method may include analyzing the at least one patient data and the at least one dental dataset. Further, the method may include generating at least one landmark based on the analyzing. Further, the method may include retrieving at least one dental reference dataset. Further, the method may include processing the at least one landmark and the at least one dental reference dataset, determining at least one dental recommendation based on the processing, transmitting the at least one dental recommendation to at least one external device and storing the at least one dental recommendation.

Abstract: At least one embodiment relates to a method for padding a first depth image representative of depth values of nearer points of a point cloud frame and a second depth image representative of depth values of farther points of a point cloud frame. The method also comprises encoding a video stream including a time-interleaving of said encoded first and second images. There is also provided a method comprising decoding a video stream to provide a first depth image representative of depth values of nearer points of a point cloud frame and a second depth image representative of depth values of farther points of a point cloud frame; and filtering pixel values of the second depth image by using pixel values of the first depth image.

Abstract: During operation of a machine, an understanding of terrain features is critical. Accordingly, disclosed embodiments augment a video stream, captured by a camera mounted on the machine, with terrain information. In particular, an overlay is generated on at least one image frame. The overlay, which may comprise one or more semi-transparent bands, may cyclically recede from a foreground to a background of the image frame(s), cyclically advance from a background to a foreground of the image frame(s), or cyclically move horizontally across the image frame(s). The overlay may be colorized to illustrate the height of the terrain underneath the overlay.

Abstract: Various methods are provided for generating motion vectors in the context of 3D computer-generated images. An example method includes generating, for each pixel of one or more objects to be rendered in a current frame, a 1-phase motion vector (MV1) and a 0-phase motion vector (MV0), each MV1 and MV0 having an associated depth value, to thereby form an MV1 texture and an MV0 texture; converting the MV1 texture to a set of MV1 blocks and converting the MV0 texture to a set of MV0 blocks; and outputting the set of MV1 blocks and the set of MV0 blocks for image processing.

Abstract: Aspects of the disclosure provide a method, an apparatus, and non-transitory computer-readable storage medium for video decoding. The apparatus includes processing circuitry that is configured to decode a current picture based on a bitstream. The processing circuitry can determine, from a supplemental enhancement information (SEI) message, a first flag indicating whether a warping process is applied to the current decoded picture. Based on the first flag indicating that the warping process is applied to the current decoded picture, the processing circuitry can determine warping information of the warping process based on the SEI message. The processing circuitry can determine a warped picture from the current decoded picture based on the warping information.

Abstract: An apparatus to facilitate real-time playback of point cloud sequence data is disclosed. The apparatus comprises one or more processors to receive point cloud data of a captured scene, decompose the point cloud data into a plurality of point cloud patches, wherein each point cloud patch is associated with an object in the scene and includes contextual information regarding the point cloud patch, encode each of the point cloud patches via a deep-learning based algorithm to generate encoded point cloud patches, receive a viewpoint selection from a client, assign a priority to data chunks within each encoded point cloud patch based on the viewpoint selection and the contextual information and transmit the data chunks to the client based on the assigned priority.

Abstract: Aspects of the disclosure provide a method and an apparatus for video encoding. The apparatus includes processing circuitry configured to generate an initial feature representation from an input image to be encoded and perform an iterative update of values of a plurality of elements in the initial feature representation. The iterative update includes generate a coded representation corresponding to a final feature representation based on the final feature representation that has been updated from the initial feature representation by a number of iterations of the iterative update. A reconstructed image corresponding to the final feature representation is generated based on the coded representation. An encoded image corresponding to the final feature representation having updated values of the plurality of elements is generated. One of (i) a rate-distortion loss corresponding to the final feature representation or (ii) the number of iterations of the iterative update satisfies a pre-determined condition.

Abstract: Several techniques for video encoding and video decoding are described. An example method of processing video data includes performing a conversion between a video including one or more subpictures and a bitstream of the video according to a format rule, wherein the format rule specifies that a first syntax element in a scalable nesting supplemental enhancement information message in the bitstream is set to a particular value in response to the scalable nesting supplemental enhancement information message including one or more subpicture level information supplemental enhancement information messages, and wherein the particular value of the first syntax element indicates that the scalable nesting supplemental enhancement information message includes one or more scalable-nested supplemental enhancement information messages that apply to a specific output video layer set.

Abstract: A method of decoding an image can includes constructing an advanced motion vector predication (AMVP) candidate list using available motion vector candidates of a left motion vector candidate, an above motion vector candidate and a temporal motion vector candidate; selecting a motion vector predictor among motion vector candidates of the AMVP candidates list using an AMVP index, and generating a motion vector using the motion vector predictor and a differential motion vector, generating a prediction block using the motion vector and a reference picture index; inversely quantizing a quantized block using a quantization parameter to generate a transformed block and inversely transforming the transformed block to generate a residual block; and generating a reconstructed block using the prediction block and the residual block, in which the quantization parameter is generated per quantization unit, and a minimum size of the quantization unit is adjusted per picture.

Abstract: To enhance a mono-output-only controller such as a mobile OS to support selective mono/stereo/mixed output, a stereo controller is instantiated in communication with the mono controller. The stereo controller coordinates stereo output, but calls and adapts functions already present in the mono controller for creating surface and image buffers, rendering, compositing, and/or merging. For content designated for 2D display, left and right surfaces are rendered from a mono perspective; for content designated for 3D display, left and right surfaces are rendered from left and right stereo perspectives, respectively. Some, all, or none of available content may be delivered to a stereo display in 3D, with a remainder delivered in 2D, and with comparable content still delivered in 2D to the mono display. The stereo controller is an add-on; the mono controller need not be replaced, removed, deactivated, or modified, facilitating transparency and backward compatibility.

Abstract: The present disclosure relates to providing a computer-generated reality (CGR) platform for generating CGR environments including virtual and augmented reality environments. In some embodiments, information related to an object to be simulated and rendered in the CGR environment is provided to the CGR platform, and a three-dimensional representation of the object is displayed in the CGR environment.

Abstract: To enhance a mono-output-only controller such as a mobile OS to support selective mono/stereo/mixed output, a stereo controller is instantiated in communication with the mono controller. The stereo controller coordinates stereo output, but calls and adapts functions already present in the mono controller for creating surface and image buffers, rendering, compositing, and/or merging. For content designated for 2D display, left and right surfaces are rendered from a mono perspective; for content designated for 3D display, left and right surfaces are rendered from left and right stereo perspectives, respectively. Some, all, or none of available content may be delivered to a stereo display in 3D, with a remainder delivered in 2D, and with comparable content still delivered in 2D to the mono display. The stereo controller is an add-on; the mono controller need not be replaced, removed, deactivated, or modified, facilitating transparency and backward compatibility.

Abstract: According to some embodiments, an imaging processing method for extracting a plurality of planar surfaces from a depth map includes computing a depth change indication map (DCI) from a depth map in accordance with a smoothness threshold. The imaging processing method further includes recursively extracting a plurality of planar region from the depth map, wherein the size of each planar region is dynamically adjusted according to the DCI. The imaging processing method further includes clustering the extracted planar regions into a plurality of groups in accordance with a distance function; and growing each group to generate pixel-wise segmentation results and inlier points statistics simultaneously.

Abstract: Although wedgelet-based partitioning seems to represent a better tradeoff between side information rate on the one hand and achievable variety in partitioning possibilities on the other hand, compared to contour partitioning, the ability to alleviate the constraints of the partitioning to the extent that the partitions have to be wedgelet partitions, enables applying relatively uncomplex statistical analysis onto overlaid spatially sampled texture information in order to derive a good predictor for the bi-segmentation in a depth/disparity map. Thus, in accordance with a first aspect it is exactly the increase of the freedom which alleviates the signaling overhead provided that co-located texture information in form of a picture is present. Another aspect pertains the possibility to save side information rate involved with signaling a respective coding mode supporting irregular partitioning.

Abstract: According to the present invention, an adaptive scheme is applied to an image encoding apparatus that includes an inter-predictor, an intra-predictor, a transformer, a quantizer, an inverse quantizer, and an inverse transformer, wherein input images are classified into two or more different categories, and two or more modules from among the inter-predictor, the intra-predictor, the transformer, the quantizer, and the inverse quantizer are implemented to perform respective operations in different schemes according to the category to which an input image belongs. Thus, the invention has the advantage of efficiently encoding an image without the loss of important information as compared to a conventional image encoding apparatus which adopts a packaged scheme.

Abstract: A method of decoding may be performed by at least one processor, and may comprise: receiving an entropy coded bitstream comprising compressed video data; generating one or more dequantized blocks, determining whether at least one of a height and a width of the one or more dequantized blocks is greater than or equal to a predefined threshold, and responsive to the at least one of the height or the width of the one or more dequantized blocks being greater than or equal to the predefined threshold, transform coding a dequantized block using a tuned line graph transform (LGT) core to perform direct matrix multiplications for each of the horizontal and vertical dimensions of the one or more dequantized blocks.

Abstract: Several embodiments for video encoding and video decoding are described. An example method of processing video data includes performing a conversion between a video comprising one or more subpictures and a bitstream of the video, wherein one or more supplemental enhancement information messages that have filler payloads are processed during the conversion according to a format rule, and wherein the format rule disallows the one or more supplemental enhancement information messages having filler payloads to be in a scalable nesting supplemental enhancement information message.

Abstract: A method of setting spatial information for defining a space in the shape of dimensional data or property information including relationship information between objects installed in the space is defined, and a video monitoring apparatus is set to be linked to the defined method. Then, based on the same, a video captured by a video monitoring apparatus provided in a space may be efficiently output on the web. Therefore, various functions may be performed in which 3-dimensional spatial information and video monitoring are linked in a very easy way.

Abstract: This disclosure relates to reconstructing three-dimensional models of objects from two-dimensional images. According to a first aspect, this specification describes a computer implemented method for creating a three-dimensional reconstruction from a two-dimensional image, the method comprising: receiving a two-dimensional image; identifying an object in the image to be reconstructed and identifying a type of said object; spatially anchoring a pre-determined set of object landmarks within the image; extracting a two-dimensional image representation from each object landmark; estimating a respective three-dimensional representation for the respective two-dimensional image representations; and combining the respective three-dimensional representations resulting in a fused three-dimensional representation of the object.

Abstract: The present application discloses an image transmission device and method. The image transmission device includes a receiver configured to receive pixel data in image data from a camera in sequence and buffer the pixel data into a memory, and determine, upon reception of a line of pixel data, a line number of the line of pixel data in the image data and a frame number of the image data; and a processor configured to obtain the line of pixel data, the line number of the line of pixel data and the frame number of the image data from the receiver, package the obtained line of pixel data, line number of the line of pixel data and frame number of the image data into a data packet, and transmit the data packet to a server.

Abstract: A method for video bitstream processing includes generating, using a first video block derived from a third video block of a first component and having a first size, a prediction block for a second video block of a video related to a second component, where the first component is different from the second component, and where the second video block has a second size that is different from the first size. The method also includes performing, using the prediction block, a conversion between the second video block and a bitstream representation of the video according to a two-step cross-component prediction mode (TSCPM).

Abstract: An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Abstract: An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Abstract: Systems and methods to improve video conferencing may include various improvements and modifications to image and audio data to simulate mutual presence among users. For example, automated positioning and presentation of image data of users via presentation devices may present users at a position, size, and scale to simulate mutual presence of users within a same physical space. In addition, rendering image data of users from virtual camera positions via presentation devices may simulate mutual eye gaze and mutual presence of users. Further, blending a background, or accentuating a user, within image data that is presented via presentation devices may match visual characteristics of the environment or the user to simulate mutual presence of users. Moreover, dynamic steering of individual audio transducers of an audio transducer array, and simulating audio point sources using a combination of speakers, may simulate direct communication and mutual presence of users.

Abstract: A device may be configured to decoding capability information according to one or more of the techniques described herein.

Abstract: A method includes receiving a first depth map that includes a plurality of first pixel depths and a second depth map that includes a plurality of second pixel depths. The first depth map corresponds to a reference depth scale and the second depth map corresponds to a relative depth scale. The method includes aligning the second pixel depths with the first pixel depths. The method includes transforming the aligned region of the second pixel depths such that transformed second edge pixel depths of the aligned region are coextensive with first edge pixel depths surrounding the corresponding region of the first pixel depths. The method includes generating a third depth map. The third depth map includes a first region corresponding to the first pixel depths and a second region corresponding to the transformed and aligned region of the second pixel depths.

Abstract: A method includes receiving, at a decoder, video data and a supplemental enhancement information (SEI) message corresponding to the video data, the SEI message including information indicating whether the received video data uses neural network based tools, and performing, at the decoder, post-processing of the received video data based on the information included in the SEI message.

Abstract: A method of decoding an encoded video bitstream using at least one processor, including: obtaining from the encoded video bitstream a coded video sequence including a picture unit corresponding to a coded picture; obtaining a picture header (PH) network abstraction layer (NAL) unit included in the picture unit; obtaining at least one video coding layer (VCL) network abstraction layer (NAL) unit included in the picture unit; decoding the coded picture based on the PH NAL unit, the at least one VCL NAL unit, and an adaptation parameter set (APS) included in an APS NAL unit obtained from the coded video sequence; and outputting the decoded picture, wherein the APS NAL unit is available to the at least one processor before the at least one VCL NAL unit.

Abstract: An apparatus includes a memory and a processor. The memory receives a plurality of frames of a scene captured from a camera array. The processor selects a first frame and a second frame from the plurality of frames. The processor also rectifies and aligns the first frame and the second frame to a reference frame, where a blank region of the second frame has a greater area than a blank region of the first frame. The processor further transforms the first frame to have near-optimal superposition to the second frame. The processor inserts a patch from the transformed first frame into the blank region of the second frame.

Abstract: A viewing direction may define an angle/visual portion of a spherical video at which a viewing window is directed. A trajectory of viewing direction may include changes in viewing directions for playback of spherical video. Abrupt changes in the viewing directions may result in jerky or shaky views of the spherical video. Changes in the viewing directions may be stabilized to provide stabilized views of the spherical video. Amount of stabilization may be limited by a margin constraint.

Abstract: Generating a 3D scene reconstruction using depth fusion can include creating a high-resolution sparse depth map by mapping sensor depths from a low-resolution depth map to points corresponding to pixels of a high-resolution color image of a scene. The high-resolution sparse depth map can have the same resolution as the high-resolution color image. A fused sparse depth map can be produced by combining the high-resolution sparse depth map with sparse depths reconstructed from the high-resolution color image. The high-resolution dense depth map can be generated based on fused sparse depths of the fused sparse depth map.

Abstract: Disclosed is a hardware acceleration based automatic read control system and method for a serial peripheral interface (SPI). The automatic read control system includes an SPI module, an advanced peripheral bus (APB) module, an interrupt generation module, a direct memory access (DMA) controller, a state schedule control module, a register group module, a count signal generation module, a transmitted data buffer and a received data buffer; the state schedule control module, the register group module and the count signal generation module form a state machine system; and the state schedule control module controls automatic timed batch read of sensor data of the SPI according to configuration information of the register group module and counting and timing information of the count signal generation module.

Abstract: A self-monitoring method of a display and a display are disclosed. The self-monitoring method of a display includes: inputting a first image test signal including a predetermined image test signal; acquiring a display parameter of the predetermined image test signal; dividing at least one frame of image outputted into a plurality of areas, in which the plurality of areas of the image include a predetermined area and a remaining area; acquiring a first display parameter of the predetermined area; comparing the display parameter of the predetermined image test signal with the first display parameter, determining whether or not an image displayed by the display is matched with the first image test signal to acquire a first match result; and determining whether or not display abnormality presents in the display based on the first match result. The method can determine whether or not display abnormality presents in the display.

Abstract: Wedgelet separation lines of neighboring blocks are predicted from each other by aligning the wedgelet separation lines of such neighboring blocks to each other, thereby reducing the side information coding rate that may be used.

Abstract: Various implementations disclosed herein include devices, systems, and methods that use an object as a background for virtual content. Some implementations involve obtaining an image of a physical environment. A location of a surface of an object is detected based on the image. A virtual content location to display virtual content is determined, where the virtual content location corresponds to the location of the surface of the object. Then, a view of the physical environment and virtual content displayed at the virtual content location is provided.

Abstract: An apparatus and method provide logic for processing information. In one implementation, an apparatus includes a display unit configured to display a first stereoscopic image. The first stereoscopic image includes a first and a second content, which may be disposed at corresponding display positions in a depth direction, and at least a portion of the first content appears to overlap at least a portion of the second content. A position-changing unit is configured to modify the display positions of the first and second content, in response to the apparent overlap. A control unit is configured to generate a signal to display, a second stereoscopic image that includes the first and second content disposed at the modified display positions. The display unit is further configured to display the second stereoscopic image such that the second stereoscopic image reduces the apparent overlap between the first and second content.

Abstract: A distance measurement apparatus includes: a light projector; a sensor to receive light projected from the light projector and reflected from a target object, photoelectrically convert the received light to an electrical signal, and obtain a plurality of phase signals from the electrical signal; and an interface to output distance data indicating a distance to the target object, the distance data being obtained based on the plurality of phase signals. The light projector includes: a plurality of light emitters that are arranged two-dimensionally; and circuitry configured to cause the plurality of light emitters to emit light a plurality of times while shifting positions of the plurality of light emitters.

Abstract: An image processing apparatus that performs processing concerning the display of a stereoscopic image so as to improve the convenience of a user in viewing a stereoscopic image. A head mount display displays a stereoscopic image using image data including a plurality of images having different viewpoints. The image data is processed based on metadata attached to the image data. In a case where information indicating that the image data is associated with a file format which cannot cause the head mount display to perform display is included in the metadata, the image data is converted into a file format which can cause the head mount display to perform display.

Abstract: Systems and methods for inserting supplemental content into presentations of two-dimensional video content are disclosed. Exemplary implementations may: obtain two-dimensional video content depicting a three-dimensional space; obtain supplemental content; obtain a model of the three-dimensional space defining the one or more visible physical features within the three-dimensional space; determine the camera position of the two-dimensional video content; identify a presentation location within the two-dimensional video content; determine integration information; modify the two-dimensional video content to include the supplemental content at the identified presentation locations in accordance with the integration information and/or perform other operations.

Abstract: There is included a method and apparatus comprising computer code configured to cause a processor or processors to perform obtaining video data comprising data of a plurality of semantically independent source pictures, determining, among the video data, whether references are associated with any of a first access unit (AU) and a second AU according to at least one picture order count (POC) signal value included with the video data, and outputting a first quantity of the references set to the first AU and a second quantity of the references set to the second AU based on the at least one POC signal value.

Abstract: A region monitoring system includes: a first communication control unit that transmits monitoring camera captured-image data to a monitoring server, the monitoring camera captured-image data being captured-image data of a monitoring camera that monitors a region in which a plurality of houses are located; and a camera communication control unit that transmits in-vehicle camera captured-image data to the monitoring server, the in-vehicle camera captured-image data being captured-image data of an in-vehicle camera of a vehicle located in the region.

Abstract: A method of video processing are described. The method includes determining, for a conversion between a video region of a picture of a video and a coded representation of the video, enablement status of a coding tool used to represent the video region in the coded representation; and performing the conversion according to the determination; wherein a first flag is included in a picture header to indicate the enablement status of the coding tool for the picture.