Abstract: A system for encoding and decoding a video coding block of a multi-view video signal is provided. A decoder is configured to decode a texture-depth video coding block (t0, d0) of a first texture frame and a first depth map associated with a first view for providing a decoded texture-depth video coding block (t0, d0) and the first depth map. A synthesized predicted texture-depth video coding block (tsyn, dsyn) of a view synthesis texture frame and a view synthesis depth map associated with a second view is generated. An inpainted synthesized predicted texture-depth video coding block is generated. Based on the impainted predicted texture-depth video block, the decoder reconstructs a texture-depth video coding block (t1, d1) of a second texture frame and a second depth map associated with the second view. An encoder is configured to encode the texture-depth video coding block in a manner that complements the decoding provided by the decoder.

Abstract: Provided is an image display method for performing drawing in a display region overlapping at least with a part of an external scene in a state in which the external scene is visible. The image display method images a region including the display region, and extracts an operating object from a captured image in which this region is imaged, as a region of a color of the operating object. In accordance with a size of the region of the color thus extracted, a drawing mode of the operating object is changed, and image display in the display region is controlled in accordance with the changed drawing mode. As the control of the image display, for example, performing drawing, and moving or replicating a drawn image is assumed.

Abstract: An object identification method and an object identification device, the object identification method comprises: capturing color images and depth images that are aligned; performing motion detection on the captured color images and depth images to acquire a motion start frame and a motion end frame; comparing the motion end frame with the motion start frame in the color images to acquire a set of motion pixels; performing connected component analysis on the acquired set of motion pixels to obtain at least one connected component; and identifying an object in the at least one connected component.

Abstract: Provided are a method and apparatus for processing a video bitstream, a network device and a readable storage medium. The method includes: parsing a bitstream to be extracted, and acquiring parameter information about an extractable sub-bitstream from a data unit of the bitstream to be extracted; determining a picture type parameter value of a random access picture in the sub-bitstream according to the parameter information; and configuring, in a process of extracting the sub-bitstream from the bitstream to be extracted, a picture type parameter value in a data unit of the sub-bitstream according to the picture type parameter value of the random access picture.

Abstract: The present application provides a method and a device of encoding and decoding based on free viewpoint, and relates to the technical field of video encoding. The method includes: generating a planar splicing image and splice information based on multiple single-viewpoint videos at a server side; generating a planar splicing video based on the planar splicing image; generating camera side information of the planar splicing video based on camera side information existing in the multiple single-viewpoint videos; and encoding the planar splicing video, the splice information and the camera side information of the planar splicing video to generate a planar splicing video bit stream, and decoding a planar splicing video bit stream to acquire a virtual viewpoint according to viewpoint information of a viewer at client side.

Abstract: A recording medium includes a video stream of a standard-luminance range and a video stream of high-luminance range, which are used selectively in accordance with a playback environment. The recording medium also includes a subtitle stream of the standard-luminance range and a subtitle stream of the high-luminance range, which are used selectively in accordance with the playback environment. A playlist file includes a management region where playback control information relating to a main stream is stored, and includes an extended region. The management region stores first playback control information specifying playing of the video stream of the high-luminance range and the subtitle stream of the high-luminance range in combination. The extended region stores second playback control information specifying playing of the video stream of the standard-luminance range and the subtitle stream of the standard-luminance range in combination.

Abstract: [Problem] To provide an information processing apparatus, an information processing method, and a program. [Solution] An information processing apparatus including a display control unit that causes a display unit to display a virtual image in a three-dimensional space. The display control unit causes the display unit to display only one of a left-eye image and a right-eye image corresponding to the virtual image if it is determined that the virtual image is located outside of a predetermined range in a depth direction when viewed from a user of the display unit, and causes the display unit to display the left-eye image and the right-eye image if it is determined that the virtual image is located inside of the predetermined range.

Abstract: A method includes receiving an input video stream and scaling the input video stream into two or more spatial layers. For each spatial layer, the method also includes generating a temporal layer prediction pattern by: obtaining a temporal base layer for a corresponding spatial layer; identifying, based on the temporal base layer, a plurality of temporal layers and a plurality of temporal time slots during a temporal period; and aligning the temporal base layer for the corresponding spatial layer with one of the temporal time slots during the temporal period. Each temporal time slot is associated with one of the temporal base layer or one of the plurality of temporal layers for the corresponding spatial layer. The temporal base layer for each corresponding spatial layer is aligned with a different temporal time slot than each other temporal base layer for each other corresponding spatial layer.

Abstract: A method for transmitting data for a three-dimensional image is provided. The method comprises the steps of: generating a plurality of voxelized points corresponding to a plurality of voxels, respectively, from a plurality of points constituting a three-dimensional image; grouping the plurality of voxelized points into a plurality of segments, wherein each of the plurality of segments includes at least one of the plurality of voxelized points; for each segment, projecting the plurality of points and generating a plurality of two-dimensional images; generating a packed two-dimensional image from the generated plurality of two-dimensional images; decoding data for the packed two-dimensional image, and encapsulating same; and transmitting the decoded and encapsulated data.

Abstract: A point cloud intensity completion method and system based on semantic segmentation are provided. The point cloud intensity completion method includes: acquiring an RGB image and point cloud data of a road surface synchronously by a photographic camera and a lidar; performing spatial transformation on the point cloud data by using a conversion matrix to generate a two-dimensional reflection intensity projection map and a two-dimensional depth projection map; performing reflection intensity completion on the RGB image and the two-dimensional reflection intensity projection map to obtain a single-channel reflection intensity projection map; performing depth completion on the RGB image and the two-dimensional depth projection map to obtain a single-channel depth projection map; and performing coarse-grained completion on the RGB image, the single-channel reflection intensity projection map and the single-channel depth projection map to obtain a two-dimensional coarse-grained reflectance intensity projection map.

Abstract: A method of video decoding performed by a video decoder includes receiving a coded video bitstream containing a current picture. The method includes determining whether a current block in the current picture is coded in intra block copy (IBC) mode. The method includes in response to a determination that the current block is coded in IBC mode, determining whether a mode with a motion vector offset is enabled for the IBC encoded current block. The method further includes in response to a determination that the mode with the motion vector offset is enabled for the IBC encoded current block, decoding the current block in accordance with an offset associated with the current block. Furthermore, the fractional offsets are not permitted for the IBC encoded current block.

Abstract: Provided is an information processing apparatus, including: an estimation processing section that, on the basis of three-dimensional data and a first image feature value that are set and a second image feature value that is acquired from a captured image captured by a first imaging apparatus, estimates a position and attitude of the first imaging apparatus.

Abstract: Disclosed is an approach for displaying 3D videos in a VR and/or AR system. The 3D videos may include 3D animated objects that escape from the display screen. The 3D videos may interact with objects within the VR and/or AR environment. The 3D video may be interactive with a user such that based on user input corresponding to decisions elected by the user at certain portions of the 3D video such that a different storyline and possibly a different conclusion may result for the 3D video. The 3D video may be a 3D icon displayed within a portal of a final 3D render world.

Abstract: A method for processing 360-degree image data by a device for receiving a 360-degree video according to the present invention comprises the steps of: receiving 360-degree image data; obtaining information relating to an encoded picture and metadata from the 360-degree image data, wherein the metadata includes camera lens information; decoding a picture including a target circular area on the basis of the information relating to the encoded picture; and rendering the target circular area by processing same on the basis of the camera lens information.

Abstract: Systems and techniques for performing illumination compensation in processing video data include deriving one or more illumination compensation parameters for a block of a picture based on one or more tools which may be used for inter-prediction of the block. Illumination compensation can be selectively applied for the block based on whether bi-directional prediction is to be applied for the inter-prediction of the block. In some cases if it is determined that bi-directional prediction is to be applied for inter-prediction of the block, illumination compensation may be avoided for the block.

Abstract: Enhanced techniques for volumetric stage mapping based on a calibration object are described. An example method includes obtaining a plurality of images of a calibration object, the images being obtained from a plurality of image sensors positioned about the calibration object, and the calibration object having location information associated with a volumetric capture surface on which the calibration object is located; generating, based on the images, a reconstructed object associated with the calibration object, the reconstructed object representing a three-dimensional representation of the calibration object; determining reconstruction quality information associated with the reconstructed object, the determination being based on a ground-truth representation of the calibration object; and outputting the reconstruction quality information.

Abstract: The invention relates to a method comprising encoding, into or along a bitstream of media content, a set of indicators to indicate one or more of unequality types, wherein an unequality type defines different encoding parameters for a video stream on a first view or region and for a video stream on a second view or region. The invention also relates to an apparatus and a computer program product for implementing the method.

Abstract: A method and device for encoding an image divided into blocks. The image contains separate first and second zones). The encoding implements the following, for at least one current block of the image: determining to which of the first and second zones the current block pertains; if the current block pertains to the first zone, encoding the current block by using a first encoding method; if the current block pertains to the second zone, encoding the current block by using a second encoding method including the following: from the position of the current block in the second zone, identifying a previously encoded, then decoded block located in the first zone of the image; and reproducing the value of at least one encoding parameter associated with the identified block.

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

Abstract: Methods and apparatus for generating stereoscopic images for display with respect to a virtual display screen (VDS) include: setting a VDS position for displaying a stereoscopic image of an object in a virtual scene, the VDS having a depth profile that approximates a depth profile of the object; generating the stereoscopic image for display at the VDS within the virtual scene; and controlling a configuration of the VDS based on the detected change in the viewpoint of the stereoscopic image by the viewer.

Abstract: A method of decoding a bitstream by an electronic device is provided. A block unit is determined from an image frame according to the bitstream. One of a plurality of candidate groups is selected based on a set flag in the bitstream. A plurality of merge mode with motion vector difference (MMVD) indications of the block unit is determined according to the bitstream. A plurality of MMVD prediction parameters of the block unit is determined based on the plurality of MMVD indications and the selected candidate group. The block unit is reconstructed based on the plurality of MMVD prediction parameters.

Abstract: Sequence-level parameters are generated for an image frame sequence including sequence-level indicators for indicating metadata types present for each image frame in the sequence of image frames. Frame-present parameters are generated for a specific image frame in the sequence including frame-present indicators corresponding to the metadata types as indicated in the sequence-level parameters. The frame-present indicators identify first metadata types for which metadata parameter values are to be encoded in a coded bitstream as metadata payloads. The image frame sequence, the sequence-level parameters, the frame-present parameters and the metadata payloads are encoded in the coded bitstream. A recipient device can generate, from the specific image frame based partly on the metadata parameter values determined for the first metadata types, a target display image for a target display.

Abstract: Systems and methods for inserting supplemental content into presentations of two-dimensional video content are disclosed. Exemplary implementations may: obtain two-dimensional video content depicting a three-dimensional space; obtain supplemental content; obtain a model of the three-dimensional space defining the one or more visible physical features within the three-dimensional space; determine the camera position of the two-dimensional video content; identify a presentation location within the two-dimensional video content; determine integration information; modify the two-dimensional video content to include the supplemental content at the identified presentation locations in accordance with the integration information and/or perform other operations.

Abstract: A method of decoding may be performed by at least one processor, and may comprise: receiving an entropy coded bitstream comprising compressed video data; generating one or more dequantized blocks, determining whether at least one of a height and a width of the one or more dequantized blocks is greater than or equal to a predefined threshold, and responsive to the at least one of the height or the width of the one or more dequantized blocks being greater than or equal to the predefined threshold, transform coding a dequantized block using a tuned line graph transform (LGT) core to perform direct matrix multiplications for each of the horizontal and vertical dimensions of the one or more dequantized blocks.

Abstract: A method, computer program, and computer system is provided for video post-processing using supplemental enhancement information (SEI) messages by determining whether a structure of a neural network and one or more parameters associated with the neural network are defined for a decoder and requesting data corresponding to the structure of the neural network and the one or more parameters based on the determination using one or more SEI messages. The requested structure and parameter data is received, and post processing is performed on video data based on the received structure and parameter data.

Abstract: Embodiments of this disclosure provide an image coding method and apparatus and an image compression system. The image coding apparatus includes a memory and a processor. The processor is configured to perform feature extraction on an input image to obtain feature maps of N channels; assign a weight to a feature map of each channel among the N channels; perform down-dimension processing on weighted feature maps processed in association with the N channels, to obtain feature maps of M channels and output the feature maps of M channels, M being smaller than N. Hence, by multiplying different feature maps by a weight to obtain corresponding importance and then performing down-dimension processing on the feature maps processed according to the weighting, time for decoding may be reduced.

Abstract: Image reconstruction methods, systems, devices, and computer-readable storage media are provided.

Abstract: To enhance a mono-output-only controller such as a mobile OS to support selective mono/stereo/mixed output, a stereo controller is instantiated in communication with the mono controller. The stereo controller coordinates stereo output, but calls and adapts functions already present in the mono controller for creating surface and image buffers, rendering, compositing, and/or merging. For content designated for 2D display, left and right surfaces are rendered from a mono perspective; for content designated for 3D display, left and right surfaces are rendered from left and right stereo perspectives, respectively. Some, all, or none of available content may be delivered to a stereo display in 3D, with a remainder delivered in 2D, and with comparable content still delivered in 2D to the mono display. The stereo controller is an add-on; the mono controller need not be replaced, removed, deactivated, or modified, facilitating transparency and backward compatibility.

Abstract: There is provided a file generation apparatus and a file generation method as well as a reproduction apparatus and a reproduction method by which a file for efficiently storing quality information of a depth-related image at least including a depth image can be generated. A segment file generation unit generates a file in which quality information representative of quality of a depth-related image including at least a depth image is disposed in a form divided for each kind. The present disclosure can be applied to a file generation apparatus of an information processing system or the like by which a segment file and an MPD file of a video content are distributed, for example, in a method that complies with MPEG-DASH.

Abstract: A method for synthesizing an omni-directional parallax view includes: obtaining parallaxes between an original image data pair, wherein the parallaxes include a horizontal parallax and a vertical parallax; determining a target viewpoint based on a base line between the original image data pair; obtaining a target pixel of the target viewpoint in original image data based on the horizontal parallax and the vertical parallax; and synthesizing a target view of the target viewpoint based on the target pixel.

Abstract: A multi-mode image sensor applicable to image sensing and additional detection is provided. The multi-mode image sensor includes: a photodiode array, a hybrid type color filter array that is positioned on the photodiode array, and a mode controller. The photodiode array may include a plurality of photodiodes. The hybrid type color filter array may be arranged to perform optical filtering for the photodiode array. The mode controller may activate the photodiode array to output corresponding photo detection results in any of at least two modes. For example, in addition to an image sensing mode, the at least two modes may further include an ambient light sensor (ALS) mode and/or a proximity sensor (PS) mode.

Abstract: A target view to a 3D scene depicted by a multiview image is determined. The multiview image comprises multiple sampled views. Each sampled view comprises multiple texture images and multiple depth images in multiple image layers. The target view is used to select, from the multiple sampled views of the multiview image, sampled views. A texture image and a depth image for each sampled view in the selected sampled views are encoded into a multiview video signal to be transmitted to a downstream device.

Abstract: Disclosed herein is an immersive video formatting method and apparatus for supporting motion parallax, The immersive video formatting method includes acquiring a basic video at a basic position, acquiring a multiple view video at at least one position different from the basic position, acquiring at least one residual video plus depth (RVD) video using the basic video and the multiple view video, and generating at least one of a packed video plus depth (PVD) video or predetermined metadata using the acquired basic video and the at least one RVD video.

Abstract: To enhance a mono-output-only controller such as a mobile OS to support selective mono/stereo/mixed output, a stereo controller is instantiated in communication with the mono controller. The stereo controller coordinates stereo output, but calls and adapts functions already present in the mono controller for creating surface and image buffers, rendering, compositing, and/or merging. For content designated for 2D display, left and right surfaces are rendered from a mono perspective; for content designated for 3D display, left and right surfaces are rendered from left and right stereo perspectives, respectively. Some, all, or none of available content may be delivered to a stereo display in 3D, with a remainder delivered in 2D, and with comparable content still delivered in 2D to the mono display. The stereo controller is an add-on; the mono controller need not be replaced, removed, deactivated, or modified, facilitating transparency and backward compatibility.

Abstract: An example method of coding video data includes selecting, by one or more processors a sub-set of a plurality of neighboring samples of a current block in a current picture, wherein the plurality of neighboring samples includes a row of samples adjacent to a top row of the current block in the current picture and a column of samples adjacent to a left column of the current block in the current picture; deriving, by the one or more processors and based on the sub-set of the plurality of neighboring samples in the current picture, local illumination compensation (LIC) parameters for the current block; and performing, by the one or more processors and based on the LIC parameters, LIC on samples of the current block to generate compensated samples of the current block.

Abstract: A method for encoding a picture of a video sequence in a bit stream that constrains tile processing overhead is provided. The method includes computing a maximum tile rate for the video sequence, computing a maximum number of tiles for the picture based on the maximum tile rate, and encoding the picture wherein a number of tiles used to encode the picture is enforced to be no more than the maximum number of tiles.

Abstract: A system includes a sensor including a set of cameras configured to capture a set of images of an enclosed space. The system further includes a processor and a non-transitory computer-readable medium containing instructions which, when executed on the processor, cause the processor to perform operations including: (a) receiving the set of images captured by the set of cameras; (b) pre-processing the set of images to increase quality and reduce effect of ambient lighting; (c) building a disparity map for the set of images based at least in part on a relationship between the set of cameras; and (d) determining an occupancy of the enclosed space based at least in part on the disparity map. The system can be used to determine occupancy in a trailer with irregularly shaped cargo using low cost sensors.

Abstract: A providing apparatus configured to provide three-dimensional geometric data to be used to generate a virtual viewpoint image receives a data request from a communication apparatus, decides which of a plurality of pieces of three-dimensional geometric data including first three-dimensional geometric data and second three-dimensional geometric data with a different quality than the first three-dimensional geometric data is to be provided to the communication apparatus from which the received data request was transmitted, and provides the three-dimensional geometric data decided on from among the plurality of pieces of three-dimensional geometric data, to the communication apparatus as a response to the received data request.

Abstract: The image generation apparatus is an image generation apparatus having: a first acquisition unit configured to acquire three-dimensional shape data of a background; a second acquisition unit configured to acquire a two-dimensional map image; a third acquisition unit configured to acquire a first image and a second image; and a generation unit configured to generate a first color information image indicating color information on an element configuring the three-dimensional shape data of the background and generate a second color information image indicating color information on an element configuring the three-dimensional shape data of the background.

Abstract: A stereoscopic VR HMD has left and right display portions in which one of the display portions is arranged with a transformation to improve the perceived virtual image resolution and therefore the perceived virtual image quality. The transformation can include offsetting sub-pixels of one of the display portions in the horizontal and if desired vertical dimensions relative to the other display portion. By applying very small (sub-millimeter) offsets along with appropriately adjusted image data, two slightly different virtual images form a stereoscopic image with a perceived increased of resolution in the binocular overlap zone established by the display portions.

Abstract: Stereoscopic images are subsampled and placed in a “checkerboard” pattern in an image. The image is encoded in a monoscopic video format. The monoscopic video is transmitted to a device where the “checkerboard” is decoded. Portions of the checkerboard (e.g., “black” portions) are used to reconstruct one of the stereoscopic images and the other portion of the checkerboard (e.g., “white” portions) are used to reconstruct the other image. The subsamples are, for example, taken from the image in a location coincident to the checkerboard position in which the subsamples are encoded.

Abstract: Methods of modeling a three-dimensional (3D) object are provided. A method of modeling a 3D object includes generating a 3D model of at least a portion of the 3D object based on data from a plurality of two-dimensional (2D) images. Moreover, the method includes scaling the 3D model by estimating a distance to the 3D object. Related devices and computer program products are also provided.

Abstract: Techniques for calculating a predicted head pose time for a display device are described herein. A request to start a frame is generated. A target finish time stamp associated with rendering the frame is calculated. A number of VSync periods from a last VSync to a target VSync is determined based on the target finish time stamp. A target VSync time stamp is calculated based on the number of VSync periods from the last VSync to the target VSync. The target VSync time stamp is compared to the target finish time stamp. The predicted head pose time is calculated based on the target VSync time stamp and a fixed platform offset.

Abstract: Methods, processes, and systems are presented for inter-layer scaling and mapping to perform dynamic-range scalability in video coding using single-loop video decoders. Two approaches are proposed: a) A mapping function is applied to each of the color components, b) a mapping function is applied to the luma component and luma-based scaling is applied to the chroma components. Both schemes may be combined with existing schemes for in-loop reshaping, such as luma mapping and chroma residual scaling employed in the versatile video codec (VVC).

Abstract: A video encoding apparatus includes a data buffer and a video encoding circuit. Encoding of a first frame includes: deriving reference pixels of a reference frame from reconstructed pixels of the first frame, respectively, and storing reference pixel data into the data buffer for inter prediction, wherein the reference pixel data include information of pixel values of the reference pixels. Encoding of a second frame includes performing prediction upon a coding unit in the second frame to determine a target predictor for the coding unit. The prediction performed upon the coding unit includes: determining the target predictor for the coding unit according to whether a search range on the reference frame for finding a predictor of the coding unit under an inter prediction mode includes at least one reference pixel of the reference frame that is not accessible to the video encoding circuit.

Abstract: Methods and systems for image encoding and decoding are disclosed. According to some embodiments, scene metadata and input images associated with a scene are received. A first encoding operation is performed on the scene metadata and input images to generate reference images and reference disparity information. A second encoding operation is performed on the reference images and reference disparity information to output encoded data. The encoded data includes encoded reference images and encoded reference disparity information. The encoded data is transmitted.

Abstract: In one embodiment, a system includes at least one projector configured to project a plurality of projected patterns, where a projected lighting characteristic of each of the projected patterns varies over a time period in accordance with an associated predetermined temporal lighting-characteristic pattern, a camera configured to capture images of detected patterns, and one or more processors configured to: determine, for each detected pattern, a detected temporal lighting-characteristic pattern based on variations in a detected lighting characteristic of the detected pattern, identify a detected pattern that corresponds to one of the projected patterns by comparing at least one of the detected temporal lighting-characteristic patterns to at least one of the temporal lighting-characteristic patterns, and compute a depth associated with the detected patterns based on the one or more of the projected patterns, the detected pattern, and a relative position between the camera and the projector.

Abstract: In general, techniques are described by which to obtain combined in-loop filters for video coding. A device for coding video data comprising a memory and one or more processors may be configured to perform the techniques. The memory may store the video data. The one or more processors, which may be implemented in circuitry, are configured to implement a combined sample adaptive offset and bilateral filter. The combined sample adaptive offset and bilateral filter may be configured to obtain reconstructed samples of a current block of the video data, and perform filtering with respect to the reconstructed samples of the current block of the video data to obtain filtered reconstructed samples of the current block of the video data.

Abstract: A method, computer program, and computer system is provided for coding video data. Video data including a reference view and a current view is received. A co-located block in the reference view is identified for a current block in the current view. A predicted block vector is calculated based on an offset vector between the current block and the co-located block, and a disparity vector between the co-located block and the reference block in the reference view. The video data is encoded/decoded based on the calculated predicted block vector.

Abstract: The method of determination of a depth map of a scene comprises generation of a distance map of the scene obtained by time of flight measurements, acquisition of two images of the scene from two different viewpoints, and stereoscopic processing of the two images taking into account the distance map. The generation of the distance map includes generation of distance histograms acquisition zone by acquisition zone of the scene, and the stereoscopic processing includes, for each region of the depth map corresponding to an acquisition zone, elementary processing taking into account the corresponding histogram.