Abstract: Although wedgelet-based partitioning seems to represent a better tradeoff between side information rate on the one hand and achievable variety in partitioning possibilities on the other hand, compared to contour partitioning, the ability to alleviate the constraints of the partitioning to the extent that the partitions have to be wedgelet partitions, enables applying relatively uncomplex statistical analysis onto overlaid spatially sampled texture information in order to derive a good predictor for the bi-segmentation in a depth/disparity map. Thus, in accordance with a first aspect it is exactly the increase of the freedom which alleviates the signaling overhead provided that co-located texture information in form of a picture is present. Another aspect pertains the possibility to save side information rate involved with signaling a respective coding mode supporting irregular partitioning.

Abstract: Disclosed are a texture synthesis method and a device using the same. The texture synthesis method adopts a source labeled diagram and a target labeled diagram in combination to guide the texture synthesis process, so that the texture synthesis is in a controlled state, thus effectively improving the accuracy and efficiency of the computer in processing complex texture information such as textures composed of multiple materials or textures involving non-uniform gradients. Meanwhile, determination of the accuracy of the texture features of the labeled diagram is introduced to the process of producing the labeled diagram, and the labeled diagram with low accuracy is re-abstracted and -segmented so that the classification of the texture features would be more accurate. This interactive iterative method improves the accuracy of the labeled diagram generation process. The device using this method also provides the same technical effects.

Abstract: A moving image encoding/decoding apparatus that performs encoding/decoding while predicting a multiview moving image including moving images of a plurality of different views includes: a corresponding region setting unit that sets a corresponding region on a depth map for an encoding/decoding target region; a region dividing unit that sets a prediction region that is one of regions obtained by dividing the encoding/decoding target region; a disparity vector generation unit that generates, for the prediction region, a disparity vector for a reference view using depth information for a region within the corresponding region that corresponds to the prediction region; a motion information generation unit that generates motion information in the prediction region from the reference view motion information based on the disparity vector for the reference view; and a prediction image generation unit that generates a prediction image for the prediction region using the motion information in the prediction region.

Abstract: An MPEG media transport (MMT) apparatus and method for processing stereoscopic video data are provided. The MMT apparatus includes an asset file generator configured to generate a single asset file that contains an entire or part of the stereoscopic video data; and a signaling message generator configured to generate a signaling message for delivery or consumption of the stereoscopic video data. At least one of the generated single asset file and the generated signaling message contains stereoscopic video information related to the stereoscopic video data.

Abstract: Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.

Abstract: Systems and methods may provide for occlusion detection in frame rate conversion. Detecting the occlusion allows frame rate conversion to be more accurately performed. In some embodiments, one or more stereoscopic depth cameras may be used to determine the depth of a moving object to more accurately determine the occlusion. In some embodiments, the compression ratio may be adjusted to balance the frame rate and power to help ensure compliance with a power budget. In at least some embodiments, the motion of a camera may be passed from a 3D render pipe to an encoder to avoid motion calculation and thereby saving power.

Abstract: A metal mask substrate includes a metal obverse surface configured such that a resist is placed on the obverse surface. The obverse surface has a three-dimensional surface roughness Sa of less than or equal to 0.11 ?m. The obverse surface also has a three-dimensional surface roughness Sz of less than or equal to 3.17 ?m.

Abstract: There is provided a file generation apparatus and a file generation method as well as a reproduction apparatus and a reproduction method by which a file for efficiently storing quality information of a depth-related image at least including a depth image can be generated. A segment file generation unit generates a file in which quality information representative of quality of a depth-related image including at least a depth image is disposed in a form divided for each kind. The present disclosure can be applied to a file generation apparatus of an information processing system or the like by which a segment file and an MPD file of a video content are distributed, for example, in a method that complies with MPEG-DASH.

Abstract: A 360-degree video data processing method performed by an apparatus for transmitting a 360-degree video according to the present invention comprises the steps of: obtaining 360-degree video data captured by at least one camera; obtaining a 2D-based picture by processing the 360-degree video data; generating metadata associated with the 360-degree video data; encoding the picture; and performing processing for storing or transmitting the encoded picture and the metadata, wherein the metadata includes information on the central point of the picture, and the information on the central point of the picture indicates whether spherical coordinates in a spherical plane corresponding to the central point of the picture have a yaw value of 0 and a pitch value of 0, respectively.

Abstract: An image processing device includes a frame buffer compressor, which is configured to: (i) compress source data into compressed data having CRC bits appended thereto, and (ii) decompress the compressed data into output data and use the CRC bits to check for errors in the output data. A multimedia device is provided, which is configured to generate the source data in response to raw data. A memory device is provided, which is configured to store the compressed data. The frame buffer compressor may include an encoder configured to compress the source data into the compressed data with the CRC bits appended thereto, and a decoder configured to decompress the compressed data into the output data. The encoder may include a prediction module, which is configured to generate prediction data including reference data and residual data, from the source data.

Abstract: An exemplary virtual reality system determines a first confidence metric representing an objective degree to which a pre-modeled object is recognized within a first captured image depicting a scene from a first vantage point. The virtual reality system also determines a second confidence metric representing an objective degree to which the pre-modeled object is recognized within a second captured image depicting the scene from a second vantage point distinct from the first vantage point. The virtual reality system then generates a rendered image that includes a depiction of the pre-modeled object from a third vantage point distinct from the first and second vantage points. The depiction of the pre-modeled object is generated based on data from the first and second captured images, the data prioritized for the depiction of the pre-modeled object according to relative magnitudes of the first and second confidence metrics. Corresponding methods and systems are also disclosed.

Abstract: A method of planning deployment of a radio communication network in at least a portion of a geographic area is provided. The method comprises: providing a three-dimensional model of the at least a portion of the geographic area; providing a model of at least one radio base station, the model comprising at least an indication of a position of the radio base station within the geographic area; and evaluating a propagation channel within the at least a portion of the geographic area associated with electromagnetic radiation, radiated by the at least one radio base station, at predetermined positions in the three-dimensional model of the geographic area. The method further comprises planning the deployment of the radio communication network based on the evaluation of the propagation channel.

Abstract: In one embodiment, a computing system may access a first image and a second image of at least a common portion of an environment while a light emission with a predetermined emission pattern is projected by a projector. The first and second images are respectively captured by a first and a second detector that are respectively separated from the projector by a first and a second distance. The system may determine that a first portion of the first image corresponds to a second portion of the second image. The system may compute, using triangulation, a first depth value associated with the first portion and a second depth value associated with the second portion. The system may determine that the first and second depth values match in accordance with one or more predetermined criteria, and generate a depth map of the environment based on at least one of the depth values.

Abstract: The techniques described herein relate to methods, apparatus, and computer readable media configured to process (including encoding and decoding) composition layouts. The video data includes a plurality of encoded two-dimensional sub-picture tracks associated with a viewport and a composition operation to compose the tracks. The composition operation comprises data indicative of a composition to perform on the plurality of two-dimensional sub-picture tracks to compose the tracks into a canvas associated with the viewport, and a composition layout operation to adjust the composition if the canvas comprises a composition layout created by two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas.

Abstract: An image encoding method includes: determining whether a current picture is a TSA picture or a trailing picture; and encoding a plurality of pictures according to whether the current picture is the TSA picture or a trailing picture. The determining includes: determining the current picture to be the TSA picture when the current picture does not belong to the highest layer and corresponds to a first field, and determining the current picture to be the trailing picture when the current picture belongs to the highest layer or corresponds to a second field.

Abstract: A head-mounted display image delivery system in combination with a head-mounted display (HMD) device, the image delivery system and the HMD device being local to one another. The HMD device includes: one or more displays; one or more sensors to determine movement, position and attitude of the display; and one or more image inputs configured to receive image data for direct display on the one or more displays without image processing by the HMD from the image delivery system. The image delivery system is configured to communicate a stereoscopic image to the HMD system to convey image information to a wearer.

Abstract: Embodiments relate to fusion processing between two images captured with two different exposure times to generate a fused image with a higher dynamic range. An unscaled single color version of a first image is blended with another unscaled single color version of a second image to generate an unscaled single color version of the fused image. A downscaled multi-color version of the first image is blended with a downscaled multi-color version of the second image to generate a downscaled multi-color version of the fused image of a plurality of downscaled versions of the fused image. A first downscaled multi-color version of the fused image is generated by upscaling and accumulating the plurality of downscaled versions of the fused image. The first downscaled multi-color version of the fused image has a pixel resolution lower than a pixel resolution of the unscaled single color version of the fused image.

Abstract: An object detection device includes a map generator configured to generate, based on a distance image that includes distance information for each coordinate, a first map in which the coordinate in a horizontal direction of the distance image is associated with the distance information, and a second map in which the coordinate in the horizontal direction of the distance image is associated with the distance information, the second map having a lower resolution than the first map; and an object detector configured to detect an object based on the first map when a distance represented by the distance information is within a first range, and to detect an object based on the second map when a distance represented by the distance information is within a second range.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: A motion estimation method and apparatus, and a non-transitory computer-readable storage medium are provided. In the method, for a predicting unit (PU) in a to-be-coded image, a candidate motion vector (MV) list for the PU is constructed based on advanced motion vector prediction (AMVP). A rate distortion (RD) cost value of each MV in the candidate MV list is calculated. A target MV of the AMVP corresponding to the smallest RD cost value of the AMVP is obtained. Integer motion estimation (IME) is performed on the PU based on a mapping point of the target MV of the AMVP in a reference frame. A target MV of the IME is obtained. The target MV of the IME is converted to quarter pixel precision, to obtain a reference target MV of quarter motion estimation (QME). Further, as a final result of a motion estimation process is determined.

Abstract: A picture prediction method and a related apparatus are disclosed. The picture prediction method includes: determining motion vector predictors of K pixel samples in a current picture block, where K is an integer greater than 1, the K pixel samples include a first vertex angle pixel sample in the current picture block, a motion vector predictor of the first vertex angle pixel sample is obtained based on a motion vector of a preset first spatially adjacent picture block of the current picture block, and the first spatially adjacent picture block is spatially adjacent to the first vertex angle pixel sample; and performing, based on a non-translational motion model and the motion vector predictors of the K pixel samples, pixel value prediction on the current picture block. Solutions in the embodiments of the present application are helpful in reducing calculation complexity of picture prediction based on a non-translational motion model.

Abstract: The subject disclosure is directed towards active depth sensing based upon moving a projector or projector component to project a moving light pattern into a scene. Via the moving light pattern captured over a set of frames, e.g., by a stereo camera system, and estimating light intensity at sub-pixel locations in each stereo frame, higher resolution depth information at a sub-pixel level may be computed than is captured by the native camera resolution.

Abstract: An apparatus may include a control unit to determine whether a display state notification indicating a dimensional display process exists. The control unit may set an output form of display data to be displayed on image data to an output form corresponding to a display state indicated by the display state notification.

Abstract: An artificial intelligence (AI) decoding apparatus includes a memory storing one or more instructions, and a processor configured to execute the stored one or more instructions, to obtain image data corresponding to a first image that is encoded, obtain a second image corresponding to the first image by decoding the obtained image data, determine whether to perform AI up-scaling of the obtained second image, based on the AI up-scaling of the obtained second image being determined to be performed, obtain a third image by performing the AI up-scaling of the obtained second image through an up-scaling deep neural network (DNN), and output the obtained third image, and based on the AI up-scaling of the obtained second image being determined to be not performed, output the obtained second image.

Abstract: A motion vector decoding method includes: determining a prediction motion vector of a to-be-decoded unit based on a motion vector of a prediction unit of the to-be-decoded unit; when a face image in which the to-be-decoded unit is located and at least one of a face image in which a first reference unit is located and a face image in which the prediction unit is located are not face images in a same orientation, performing a first update on the prediction motion vector, where the first update is used to determine a mapping vector that is of the prediction motion vector and that is in a plane of the face image in which the to-be-decoded unit is located; and obtaining a motion vector of the to-be-decoded unit based on the prediction motion vector obtained after the first update.

Abstract: A remote control system includes an object detection unit, an object determination unit, and a static gesture processing unit. The object detection unit detects an object corresponding to an operator according to a depth image including the operator and a face detection result corresponding to the operator. The object determination unit utilizes a combination of a gesture database, a color image of the object, and a two-dimensional image corresponding to the depth map to determine a gesture formed by the object when the operator moves the object to a predetermined position. The operator moves the object to the predetermined position and pulls the object after the operator moves the object to the predetermined position within a first predetermined period. The static gesture processing unit generates a first control command to control an electronic device according to at least one static gesture determined by the object determination unit.

Abstract: A video processing apparatus, a video processing method thereof and a non-transitory computer readable medium are provided. In the method, at least two original video files are obtained, where each original video file is recorded in different shoot direction. Each video frame of the original video file is stitched, to generate multiple stitched video frame. In response to generating each stitched video frame, each stitched video frame is provided for use of playback directly without encoding those stitched video frame into a video file. Accordingly, a real-time and smooth playback effect is achieved.

Abstract: Machine learning is applied to both 2D images from an infrared imager imaging laser reflections from an object and to the 3D depth map of the object that is generated using the 2D images and time of flight (TOF) information. In this way, the 3D depth map accuracy can be improved without increasing laser power or using high resolution imagers.

Abstract: In accordance with a first aspect, the intra prediction direction of a neighboring, intra-predicted block is used in order to predict the extension direction of the wedgelet separation line of a current block, thereby reducing the side information rate necessitated in order to convey the partitioning information. In accordance with a second aspect, the idea is that previously reconstructed samples, i.e. reconstructed values of blocks preceding the current block in accordance with the coding/decoding order allow for at least a prediction of a correct placement of a starting point of the wedgelet separation line, namely by placing the starting point of the wedgelet separation line at a position of a maximum change between consecutive ones of a sequence of reconstructed values of samples of a line of samples extending adjacent to the current block along a circumference thereof. Both aspects may be used individually or in combination.

Abstract: A method for encoding a picture of a video sequence in a bit stream that constrains tile processing overhead is provided. The method includes computing a maximum tile rate for the video sequence, computing a maximum number of tiles for the picture based on the maximum tile rate, and encoding the picture wherein a number of tiles used to encode the picture is enforced to be no more than the maximum number of tiles.

Abstract: Innovations in unified intra block copy (“BC”) and inter prediction modes are presented. In some example implementations, bitstream syntax, semantics of syntax elements and many coding/decoding processes for inter prediction mode are reused or slightly modified to enable intra BC prediction for blocks of a frame. For example, to provide intra BC prediction for a current block of a current picture, a motion compensation process applies a motion vector that indicates a displacement within the current picture, with the current picture being used as a reference picture for the motion compensation process. With this unification of syntax, semantics and coding/decoding processes, various coding/decoding tools designed for inter prediction mode, such as advanced motion vector prediction, merge mode and skip mode, can also be applied when intra BC prediction is used, which simplifies implementation of intra BC prediction.

Abstract: A video, such as a spherical video, may include motion due to motion of one or more image capture devices during capture of the video. Motion of the image capture devices during the capture of the video may cause the playback of the video to appear jerky/shaky. The video may be stabilized by using both a horizontal feature and a fixed feature captured within the video.

Abstract: The techniques described herein relate to methods, apparatus, and computer readable media configured to derive a composite track. Three-dimensional video data includes a plurality of two-dimensional sub-picture tracks associated with a viewport. A composite track derivation for composing the plurality of two-dimensional sub-picture tracks for the viewport includes data indicative of the plurality of two-dimensional sub-picture tracks belonging to a same group, placement information to compose sample images from the plurality of two-dimensional tracks into a canvas associated with the viewport, and a composition layout operation to adjust the composition if the canvas comprises a composition layout created by two or more of the plurality of two-dimensional sub-picture tracks composed on the canvas. The composite track derivation can be encoded and/or used to decode the three-dimensional video data.

Abstract: An image stitching method using viewpoint transformation and a system therefor are provided. The method includes obtaining images captured by a plurality of cameras included in the camera system, performing viewpoint transformation for each of the images using a depth map for the images, and stitching the images, the viewpoint transformation of which is performed.

Abstract: The way of predicting a current block by assigning constant partition values to the partitions of a bi-partitioning of a block is quite effective, especially in case of coding sample arrays such as depth/disparity maps where the content of these sample arrays is mostly composed of plateaus or simple connected regions of similar value separated from each other by steep edges. The transmission of such constant partition values would, however, still need a considerable amount of side information which should be avoided. This side information rate may be further reduced if mean values of values of neighboring samples associated or adjoining the respective partitions are used as predictors for the constant partition values.

Abstract: Although wedgelet-based partitioning seems to represent a better tradeoff between side information rate on the one hand and achievable variety in partitioning possibilities on the other hand, compared to contour partitioning, the ability to alleviate the constraints of the partitioning to the extent that the partitions have to be wedgelet partitions, enables applying relatively uncomplex statistical analysis onto overlaid spatially sampled texture information in order to derive a good predictor for the bi-segmentation in a depth/disparity map. Thus, in accordance with a first aspect it is exactly the increase of the freedom which alleviates the signaling overhead provided that co-located texture information in form of a picture is present. Another aspect pertains the possibility to save side information rate involved with signaling a respective coding mode supporting irregular partitioning.

Abstract: A method, an image processing device, and a system for generating a depth map are proposed. The method includes the following steps. A first original image and a second original image are obtained, and first edge blocks corresponding to the first original image and second edge blocks corresponding to the second original image are obtained. Depth information of edge blocks is generated according to the first edge blocks and the second edge blocks, and depth information of non-edge blocks is set according to the depth information of the edge blocks. The depth map is generated by using the depth information of the edge blocks and the depth information of the non-edge blocks.

Abstract: Techniques related to generating a virtual view from multi-view images for presentation to a viewer are discussed. Such techniques include determining, based on a viewer position relative to a display region, first and second crop positions of planar image and cropping the planar image to a cropped planar image to fill the display region using the first and second crop positions such that the first and second crop positions define an asymmetric frustum between the cropped planar image and a virtual window corresponding to the display region.

Abstract: Examples of the present disclosure describe systems and methods for capturing data to acquire indoor and outdoor geometry. In aspects, a data capture system may be configured to acquire texture data, geometry data, navigation data and/or orientation data to support geolocation and georeferencing within indoor and outdoor environments. The data capture system may further be configured to acquire seamless texture data from a 360° horizontal and vertical perspective to support panoramic video and images.

Abstract: A system for producing 3D video using first and second cameras on first and second axes. The second camera has a field of view (FOV) overlapping with the first camera's FOV. The second axis is at a convergence angle relative to the first axis. A control computer changes the inter-camera distance by effectively moving the second camera laterally, and changes convergence angle by effectively rotating the second camera. The control computer automatically calculates the inter-camera distance and convergence angle based on the distance of a user to the display screen, working distance of the cameras, zoom settings, and size of the screen, and effectively moves the second camera accordingly. A keystone correction is performed to account for the camera projections, the frames are rotationally aligned, and the corrected/aligned frames are combined to produce a 3D image frame that is displayed on a 3D display screen.

Abstract: Convenience is achieved in performing processing depending on decoding capability in a reception side. High-frame-rate ultra-high-definition image data is processed to obtain first image data for acquisition of a base-frame-rate high-definition image, second image data for acquisition of a high-frame-rate high-definition image by being used with the first image data, third image data for acquisition of a base-frame-rate ultra-high-definition image by being used with the first image data, and fourth image data for acquisition of a high-frame-rate ultra-high-definition image by being used with the first to third image data. A container is transmitted including a predetermined number of video streams including encoded image data of the first to fourth image data. Information is inserted into the container, the information corresponding to information that is inserted into each of the predetermined number of video streams and associated with image data included in the video streams.

Abstract: Augmented reality (AR) telepresence systems and methods are disclosed for obtaining a 3D model of a physical location from a 3D-capture system comprising one or more 3D depth sensors disposed throughout the physical location, generating a truncated 3D model of the physical location, the truncated 3D model corresponding to the intersection of the generated 3D model and a field of view of a user terminal camera at the physical location, and transmitting the truncated 3D model to a remote location.

Abstract: To make it possible to obtain a natural virtual viewpoint image in which a structure or the like existing within an image capturing scene is represented three-dimensionally so as to be the same as a real one while suppressing a network load at the time of transmission of multi-viewpoint image data. The generation device according to the present invention generates a virtual viewpoint image based on three-dimensional shape data corresponding to an object, three-dimensional shape data corresponding to a structure, background data corresponding to a background different at least from the object and the structure, and information indicating a virtual viewpoint.

Abstract: A terminal device includes a memory configured to store computer-readable instructions and a processor configured to perform the computer-readable instructions. The processor is configured to: cause a real space camera in a real space to capture a real space image including a real player; cause a virtual space camera in a virtual space to capture a virtual space image including a virtual object, the real player performing an instruction input to the virtual object; create a composite image that is formed by composing part of the virtual space image stored in the memory and a player image in the real space image stored in the memory; and output the composite image to a display so that the display is configured to display the composite image.

Abstract: Methods and apparatus for using a display in a manner which results in a user perceiving a higher resolution than would be perceived if a user viewed the display from a head on position are described. In some embodiments one or more displays are mounted at an angle, e.g., sometimes in range a range from an angle above 0 degrees to 45 relative to a user's face and thus eyes. The user sees more pixels, e.g., dots corresponding to light emitting elements, per square inch of eye area than the user would see if the user were viewing the display head on due to the angle at which the display or displays are mounted. The methods and display mounting arrangement are well suited for use in head mounted displays, e.g., Virtual Reality (VR) displays for stereoscopic viewing (e.g., 3D) and/or non-stereoscopic viewing of displayed images.

Abstract: Disclosed is technology associated with video encoding and decoding having a structure including one or more layers (quality, spatial, and view) and technology associated with a method that predicts an higher layer signal by using one or more reference layers in encoding and decoding an higher layer. In more detail, an interlayer prediction is capable of being performed by considering a characteristic of each layer by separating a spatial and quality reference layer list constituted by spatial and quality layers to be referred at the same view as a target layer and a view reference layer list constituted by the same spatial and quality layers as a target layer in encoding and decoding encoding and decoding pictures of an higher layer in encoding and decoding encoded and decoded pictures of the higher layer to improve encoding and decoding efficiency.

Abstract: A three dimensional system including rendering with variable displacement.

Abstract: Stereoscopic images are subsampled and placed in a “checkerboard” pattern in an image. The image is encoded in a monoscopic video format. The monoscopic video is transmitted to a device where the “checkerboard” is decoded. Portions of the checkerboard (e.g., “black” portions) are used to reconstruct one of the stereoscopic images and the other portion of the checkerboard (e.g., “white” portions) are used to reconstruct the other image. The subsamples are, for example, taken from the image in a location coincident to the checkerboard position in which the subsamples are encoded.

Abstract: Motion compensation requires a significant amount of memory bandwidth, especially for smaller prediction unit sizes. The worst case bandwidth requirements can occur when bi-predicted 4×8 or 8×4 PUs are used. To reduce the memory bandwidth requirements for such smaller PUs, methods are provided for restricting inter-coded PUs of small block sizes to be coded only in a uni-predictive mode, i.e., forward prediction or backward prediction. More specifically, PUs of specified restricted sizes in bi-predicted slices (B slices) are forced to be uni-predicted.

Abstract: A three dimensional system including a marker mode.