Abstract: This disclosure relates to reconstructing three-dimensional models of objects from two-dimensional images. According to a first aspect, this specification describes a computer implemented method for creating a three-dimensional reconstruction from a two-dimensional image, the method comprising: receiving a two-dimensional image; identifying an object in the image to be reconstructed and identifying a type of said object; spatially anchoring a pre-determined set of object landmarks within the image; extracting a two-dimensional image representation from each object landmark; estimating a respective three-dimensional representation for the respective two-dimensional image representations; and combining the respective three-dimensional representations resulting in a fused three-dimensional representation of the object.

Abstract: A method for video bitstream processing includes generating, using a first video block derived from a third video block of a first component and having a first size, a prediction block for a second video block of a video related to a second component, where the first component is different from the second component, and where the second video block has a second size that is different from the first size. The method also includes performing, using the prediction block, a conversion between the second video block and a bitstream representation of the video according to a two-step cross-component prediction mode (TSCPM).

Abstract: The present application discloses an image transmission device and method. The image transmission device includes a receiver configured to receive pixel data in image data from a camera in sequence and buffer the pixel data into a memory, and determine, upon reception of a line of pixel data, a line number of the line of pixel data in the image data and a frame number of the image data; and a processor configured to obtain the line of pixel data, the line number of the line of pixel data and the frame number of the image data from the receiver, package the obtained line of pixel data, line number of the line of pixel data and frame number of the image data into a data packet, and transmit the data packet to a server.

Abstract: An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Abstract: An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Abstract: Systems and methods to improve video conferencing may include various improvements and modifications to image and audio data to simulate mutual presence among users. For example, automated positioning and presentation of image data of users via presentation devices may present users at a position, size, and scale to simulate mutual presence of users within a same physical space. In addition, rendering image data of users from virtual camera positions via presentation devices may simulate mutual eye gaze and mutual presence of users. Further, blending a background, or accentuating a user, within image data that is presented via presentation devices may match visual characteristics of the environment or the user to simulate mutual presence of users. Moreover, dynamic steering of individual audio transducers of an audio transducer array, and simulating audio point sources using a combination of speakers, may simulate direct communication and mutual presence of users.

Abstract: A device may be configured to decoding capability information according to one or more of the techniques described herein.

Abstract: A method includes receiving a first depth map that includes a plurality of first pixel depths and a second depth map that includes a plurality of second pixel depths. The first depth map corresponds to a reference depth scale and the second depth map corresponds to a relative depth scale. The method includes aligning the second pixel depths with the first pixel depths. The method includes transforming the aligned region of the second pixel depths such that transformed second edge pixel depths of the aligned region are coextensive with first edge pixel depths surrounding the corresponding region of the first pixel depths. The method includes generating a third depth map. The third depth map includes a first region corresponding to the first pixel depths and a second region corresponding to the transformed and aligned region of the second pixel depths.

Abstract: A method of decoding an encoded video bitstream using at least one processor, including: obtaining from the encoded video bitstream a coded video sequence including a picture unit corresponding to a coded picture; obtaining a picture header (PH) network abstraction layer (NAL) unit included in the picture unit; obtaining at least one video coding layer (VCL) network abstraction layer (NAL) unit included in the picture unit; decoding the coded picture based on the PH NAL unit, the at least one VCL NAL unit, and an adaptation parameter set (APS) included in an APS NAL unit obtained from the coded video sequence; and outputting the decoded picture, wherein the APS NAL unit is available to the at least one processor before the at least one VCL NAL unit.

Abstract: A method includes receiving, at a decoder, video data and a supplemental enhancement information (SEI) message corresponding to the video data, the SEI message including information indicating whether the received video data uses neural network based tools, and performing, at the decoder, post-processing of the received video data based on the information included in the SEI message.

Abstract: An apparatus includes a memory and a processor. The memory receives a plurality of frames of a scene captured from a camera array. The processor selects a first frame and a second frame from the plurality of frames. The processor also rectifies and aligns the first frame and the second frame to a reference frame, where a blank region of the second frame has a greater area than a blank region of the first frame. The processor further transforms the first frame to have near-optimal superposition to the second frame. The processor inserts a patch from the transformed first frame into the blank region of the second frame.

Abstract: A viewing direction may define an angle/visual portion of a spherical video at which a viewing window is directed. A trajectory of viewing direction may include changes in viewing directions for playback of spherical video. Abrupt changes in the viewing directions may result in jerky or shaky views of the spherical video. Changes in the viewing directions may be stabilized to provide stabilized views of the spherical video. Amount of stabilization may be limited by a margin constraint.

Abstract: Generating a 3D scene reconstruction using depth fusion can include creating a high-resolution sparse depth map by mapping sensor depths from a low-resolution depth map to points corresponding to pixels of a high-resolution color image of a scene. The high-resolution sparse depth map can have the same resolution as the high-resolution color image. A fused sparse depth map can be produced by combining the high-resolution sparse depth map with sparse depths reconstructed from the high-resolution color image. The high-resolution dense depth map can be generated based on fused sparse depths of the fused sparse depth map.

Abstract: Disclosed is a hardware acceleration based automatic read control system and method for a serial peripheral interface (SPI). The automatic read control system includes an SPI module, an advanced peripheral bus (APB) module, an interrupt generation module, a direct memory access (DMA) controller, a state schedule control module, a register group module, a count signal generation module, a transmitted data buffer and a received data buffer; the state schedule control module, the register group module and the count signal generation module form a state machine system; and the state schedule control module controls automatic timed batch read of sensor data of the SPI according to configuration information of the register group module and counting and timing information of the count signal generation module.

Abstract: A self-monitoring method of a display and a display are disclosed. The self-monitoring method of a display includes: inputting a first image test signal including a predetermined image test signal; acquiring a display parameter of the predetermined image test signal; dividing at least one frame of image outputted into a plurality of areas, in which the plurality of areas of the image include a predetermined area and a remaining area; acquiring a first display parameter of the predetermined area; comparing the display parameter of the predetermined image test signal with the first display parameter, determining whether or not an image displayed by the display is matched with the first image test signal to acquire a first match result; and determining whether or not display abnormality presents in the display based on the first match result. The method can determine whether or not display abnormality presents in the display.

Abstract: Wedgelet separation lines of neighboring blocks are predicted from each other by aligning the wedgelet separation lines of such neighboring blocks to each other, thereby reducing the side information coding rate that may be used.

Abstract: Various implementations disclosed herein include devices, systems, and methods that use an object as a background for virtual content. Some implementations involve obtaining an image of a physical environment. A location of a surface of an object is detected based on the image. A virtual content location to display virtual content is determined, where the virtual content location corresponds to the location of the surface of the object. Then, a view of the physical environment and virtual content displayed at the virtual content location is provided.

Abstract: An apparatus and method provide logic for processing information. In one implementation, an apparatus includes a display unit configured to display a first stereoscopic image. The first stereoscopic image includes a first and a second content, which may be disposed at corresponding display positions in a depth direction, and at least a portion of the first content appears to overlap at least a portion of the second content. A position-changing unit is configured to modify the display positions of the first and second content, in response to the apparent overlap. A control unit is configured to generate a signal to display, a second stereoscopic image that includes the first and second content disposed at the modified display positions. The display unit is further configured to display the second stereoscopic image such that the second stereoscopic image reduces the apparent overlap between the first and second content.

Abstract: Systems and methods for inserting supplemental content into presentations of two-dimensional video content are disclosed. Exemplary implementations may: obtain two-dimensional video content depicting a three-dimensional space; obtain supplemental content; obtain a model of the three-dimensional space defining the one or more visible physical features within the three-dimensional space; determine the camera position of the two-dimensional video content; identify a presentation location within the two-dimensional video content; determine integration information; modify the two-dimensional video content to include the supplemental content at the identified presentation locations in accordance with the integration information and/or perform other operations.

Abstract: A distance measurement apparatus includes: a light projector; a sensor to receive light projected from the light projector and reflected from a target object, photoelectrically convert the received light to an electrical signal, and obtain a plurality of phase signals from the electrical signal; and an interface to output distance data indicating a distance to the target object, the distance data being obtained based on the plurality of phase signals. The light projector includes: a plurality of light emitters that are arranged two-dimensionally; and circuitry configured to cause the plurality of light emitters to emit light a plurality of times while shifting positions of the plurality of light emitters.

Abstract: An image processing apparatus that performs processing concerning the display of a stereoscopic image so as to improve the convenience of a user in viewing a stereoscopic image. A head mount display displays a stereoscopic image using image data including a plurality of images having different viewpoints. The image data is processed based on metadata attached to the image data. In a case where information indicating that the image data is associated with a file format which cannot cause the head mount display to perform display is included in the metadata, the image data is converted into a file format which can cause the head mount display to perform display.

Abstract: There is included a method and apparatus comprising computer code configured to cause a processor or processors to perform obtaining video data comprising data of a plurality of semantically independent source pictures, determining, among the video data, whether references are associated with any of a first access unit (AU) and a second AU according to at least one picture order count (POC) signal value included with the video data, and outputting a first quantity of the references set to the first AU and a second quantity of the references set to the second AU based on the at least one POC signal value.

Abstract: A region monitoring system includes: a first communication control unit that transmits monitoring camera captured-image data to a monitoring server, the monitoring camera captured-image data being captured-image data of a monitoring camera that monitors a region in which a plurality of houses are located; and a camera communication control unit that transmits in-vehicle camera captured-image data to the monitoring server, the in-vehicle camera captured-image data being captured-image data of an in-vehicle camera of a vehicle located in the region.

Abstract: A method of video processing are described. The method includes determining, for a conversion between a video region of a picture of a video and a coded representation of the video, enablement status of a coding tool used to represent the video region in the coded representation; and performing the conversion according to the determination; wherein a first flag is included in a picture header to indicate the enablement status of the coding tool for the picture.

Abstract: A method and device for coding attribute information of a point cloud may include obtaining the point cloud including a set of points in a three-dimensional space. The attribute information of the point cloud is coded using a YCoCg-R transform.

Abstract: Systems, methods, and non-transitory computer-readable medium storing instructions that, when executed, causes a processor to perform operations to display a three-dimensional (3D) space. The methods may include, with an imaging device, capturing a first series of frames as the imaging device travels from a first location to a second location within a space, and capturing a second series of frames as the imaging device travels from the second location to the first location. The method may also include determining a first segment in the first series of frames that matches a second segment in the second series of frames to create a segmentation dataset, generating video clip data based on the segmentation dataset, the video clip data defining a series of video clips, and displaying the series of video clips.

Abstract: In various examples there is an apparatus for aligning three-dimensional, 3D, representations of people. The apparatus comprises at least one processor and a memory storing instructions that, when executed by the at least one processor, perform a method comprising accessing a first 3D representation which is an instance of a parametric model of a person; accessing a second 3D representation which is a photoreal representation of the person; computing an alignment of the first and second 3D representations; and computing and storing a hologram from the aligned first and second 3D representations such that the hologram depicts parts of the person which are observed in only one of the first and second 3D representations; or controlling an avatar representing the person where the avatar depicts parts of the person which are observed in only one of the first and second 3D representations.

Abstract: An apparatus for transmitting a video comprises an inter-view redundancy remover configured to remove redundant information of pictures for viewing positions, where redundant pixel information between the pictures in adjacent viewing positions is removed, a packing unit configured to pack the pictures and generate packing information, and an encoder configured to encode the pictures. The apparatus is further configured to perform multi-view packing the pictures into a packed picture, where each view for the picture includes different types of a texture and a depth map, and a residual of texture and a depth map are generated for a subsidiary view based on redundancy between each view. The apparatus is further configured to generate signaling information for the inter-view redundancy remover, the packing unit, or the multi-view packing.

Abstract: An electronic device used in a method for expanding image depth obtains first images by a first sensor, the first images comprising depth information. The electronic device obtains second images by a second sensor, the second images comprising gradient information, and the first images correspond to the second images. The electronic device determines the pixels in the first images which contain expandable content according to the gradient information of the second images, applies expansion accordingly to the pixels in the first images to generate third images, and generate target depth maps according to the gradient information of the second images and the depth information of the third images.

Abstract: A system comprises an encoder configured to compress attribute information and/or spatial information for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. In some embodiments, a bit stream structure may be used to communicate compressed point cloud data. The bit stream structure may include point cloud compression network abstraction layer (PCCNAL) units that enable use of groups of frames (GOFs), frame, and sub-frame signaling of patch information. Such a bit stream structure may permit low delay streaming and random access reconstruction of point clouds amongst other applications.

Abstract: A method for high-precision true color three dimensional reconstruction of mechanical component. Firstly performs image acquisition: the left and right high-resolution grayscale cameras are fixed at same height and spaced at certain distance, an optical transmitter fixed between the two grayscale cameras, and low-resolution color camera fixed above optical transmitter, thus images of measured high-precision mechanical component are shot. Then performs image processing: all images are transmitted to a computer, which uses image processing to record surface information of measured high-precision mechanical component in the point cloud by high-precision true color three-dimensional reconstruction, which reflects color texture information of the surface, so as to realize the non-contact high-precision true color three dimensional reconstruction of high-precision mechanical component.

Abstract: Embodiments of the present application provide a data processing method and an electronic device. The data processing method includes: determining whether a current collection scene satisfies a condition for enabling a high-dynamic range (HDR) collection function; automatically enabling the HDR collection function in response to the current collection scene satisfying the condition for enabling the HDR collection function; and collecting at least two two-dimensional images with different exposures within a collection time of one frame of three-dimensional video data based on the HDR collection function; wherein the at least two two-dimensional images are configured to enable a mobile edge computing (MEC) server to build a three-dimensional video.

Abstract: Systems and methods for generating a three-dimensional reconstruction from a two-dimensional image of an asset are described. In some aspects, at least one computer hardware processor is used to perform: receiving a two-dimensional input image of an asset; encoding, using a first statistical model, the input image into a latent vector; and generating, using a second statistical model, at least one depth image from the latent vector, wherein pixel values in the at least one depth image correspond to coordinates of a point cloud.

Abstract: A method of generating a composite image includes capturing a video image of a physical scene with a camera, identifying a green-screen region within the video image, identifying a viewpoint and a position and/or orientation of the green-screen region relative to the viewpoint, and generating a modified video image rendered from the viewpoint onto a display surface in which the green-screen region is replaced with an image of a virtual object. The image of the virtual object is generated by projection rendering of a model of the virtual object based on the position and/or orientation of the green-screen region relative to the viewpoint such that the virtual object is constrained within the green-screen region.

Abstract: A system and method for creating an improved three-dimensional image includes several steps. One step includes providing one or more adjacent viewing zones, where each of the adjacent viewing zones includes several views of content, and where the adjacent viewing zones include central subset zones that include centrally located views within the adjacent viewing zones, and transition subset views that include views at edges of the adjacent viewing zones. Another step includes inserting at least one of the views from the central subset views into the transition zone to create an expanded transition zone. A further step includes removing at least one transition subset view from the adjacent viewing zone and replacing the removed at least one transition subset view with the inserted at least one of the views from the central subset views.

Abstract: Reconstructing a second component signal relating to a second component of a multi-component picture from a spatially corresponding portion of a reconstructed first component signal and a correction signal derived from a data stream for the second component promises increased coding efficiency over a broader range of multi-component picture content. By including the spatially corresponding portion of the reconstructed first component signal into the reconstruction of the second component signal, any remaining inter-component redundancies/correlations present such as still present despite a possibly a priori performed component space transformation, or present because of having been introduced by such a priori performed component space transformation, for example, may readily be removed by way of the inter-component redundancy/correlation reduction of the second component signal.

Abstract: An image processing apparatus having a processor configured to: select a reference frame from a frame group including a plurality of images; acquire a reference correct answer frame representing a region of interest in the selected reference frame; generate a complementary correct answer frame corresponding to a frame other than the reference frame included in the frame group based on at least one reference correct answer frame; and generate a correct answer image group for machine learning from the reference correct answer frame and the complementary correct answer frame.

Abstract: A 360 image data processing method performed by a 360 video receiving device, according to the present invention, comprises the steps of: receiving 360 image data; acquiring information and metadata on an encoded picture from the 360 image data; decoding the picture on the basis of the information on the encoded picture; and rendering the decoded picture and an overlay on the basis of the metadata, wherein the metadata includes overlay-related metadata, the overlay is rendered on the basis of the overlay-related metadata, and the overlay-related metadata includes information on a region of the overlay.

Abstract: An image coding method according to the present document comprises the steps of: deriving transform coefficients for a current block on the basis of residual-related information; and generating residual samples of the current block on the basis of the transform coefficients. The residual-related information comprises low frequency non-separable transform (LFNST) index information, showing information relating to non-separable transform for low-frequency transform coefficients of the current block, on the basis of a matrix-based intra-prediction (MIP) flag showing whether or not MIP is applied to the current block. The residual samples are generated from the transform coefficients on the basis of the LFNST index information.

Abstract: Example systems and methods for virtual visualization of a three-dimensional (3D) model of an object in a two-dimensional (2D) environment. The method may include superimposing a first 3D model of an object onto the 2D environment, and replacing the first 3D model of the object with a second 3D model of an object. Further, the method may include superimposing a smart 3D model of an object onto the 2D environment. Additionally, the method may include a code or mark to identify images of the 2D environment to which 3D models of objects have been superimposed.

Abstract: According to an embodiment, an image projection device outputting an image to a screen comprises an input unit receiving an image from an outside, a sensor unit sensing a viewer's position and gaze direction, a controller correcting the received image into a preset model based on the viewer's position and gaze direction sensed by the sensor unit, and an output unit outputting the preset model.

Abstract: Various concepts for media content streaming are described. Some allow for streaming spatial scene content in a spatially unequal manner so that the visible quality for the user is increased, or the processing complexity or used bandwidth at the streaming retrieval site is decreased. Others allow for streaming spatial scene content in a manner enlarging the applicability to further application scenarios.

Abstract: A video transmission method that includes predicting, from a texture picture or a depth picture of an anchor viewing position, a picture for a target viewing position on the basis of target viewing position information and processing a prediction error with respect to the predicted picture on the basis of a source picture of the target viewing position. An error-prone region map is generated on the basis of the predicted picture and the source picture. The video transmission method also includes patch packing the prediction error-processed picture on the basis of the error-prone region map and encoding the packed patch on the basis of the texture picture or the depth picture of the anchor viewing position.

Abstract: An apparatus for stitching together multiple camera images to form a blended image having an output projection format. The apparatus is configured to convert each of the multiple camera images into the output projection format. It is configured to stitch together the converted images to form a single image. It is also configured to output the single image as the blended image having the output projection format.

Abstract: The present disclosure relates to an information processing apparatus and an information processing method that enable processing to be performed simply, and a program. By converting a point cloud representing a three-dimensional structure into two dimensions, a geometry image and a texture image, and three-dimensional information metadata required for constructing the geometry image and the texture image in three dimensions are obtained. Then, one PC sample included in a Point Cloud displayed at a specific time is generated by storing the geometry image, the texture image, and the three-dimensional information metadata in accordance with a playback order required at a time of reproducing and playing back the geometry image and the texture image in three dimensions on the basis of the three-dimensional information metadata. It is possible to apply to a data generation device that generates data for distribution of a Point Cloud.

Abstract: Techniques are described enabling users to transfer the association of a cloud-managed video encoding or decoding device from one region of a cloud provider network to another. A video processing service of a cloud provider network enables users to purchase video encoding or decoding devices that can be used to encode and reliably transfer video content to the video processing service from locations of the users' choice. The video processing service receives a request to transfer the association of a video encoding or decoding device from one region of the cloud provider to another and the video processing service executes workflows that enable the association of a video encoding or decoding device to be transferred to one region from another.

Abstract: Various concepts for media content streaming are described. Some allow for streaming spatial scene content in a spatially unequal manner so that the visible quality for the user is increased, or the processing complexity or used bandwidth at the streaming retrieval site is decreased. Others allow for streaming spatial scene content in a manner enlarging the applicability to further application scenarios.

Abstract: Various concepts for media content streaming are described. Some allow for streaming spatial scene content in a spatially unequal manner so that the visible quality for the user is increased, or the processing complexity or used bandwidth at the streaming retrieval site is decreased. Others allow for streaming spatial scene content in a manner enlarging the applicability to further application scenarios.

Abstract: At lease a computer program product is provided for efficiently encoding or decoding a video frame. The computer program product when executed by one or more processors configures the one or more processors to compress or encode the different video subsections within the video frame with different compression methods. The visual artifact is reduced between the different video subsections encoded or compressed with the different compression methods within the video frame.

Abstract: Provided is a scalable video-decoding method based on multiple layers. The scalable video-decoding method according to the present invention comprises: a step of predicting first filter information of a video to be filtered using the information contained in an object layer and/or information contained in another layer, and generating second filter information in accordance with the prediction; and a step of filtering the video to be filtered using the second filter information. According to the present invention, the amount of information being transmitted is reduced, and video compression performance is improved.