Abstract: In a safety boundary generation method, an image of a surrounding environment captured by a camera in a head-mounted display device is received. Three-dimensional reconstruction is performed based on the image of the surrounding environment to obtain a three-dimensional environment model. The three-dimensional environment model indicates a three-dimensional representation of the surrounding environment of a user that wears the head-mounted display device. Ground detection on the three-dimensional environment model is performed to obtain a plurality of ground boundary points. Each of the plurality of ground boundary points is an intersection point between a ground plane and an obstacle in the surrounding environment. A safety boundary is determined based on the plurality of ground boundary points. A region enclosed by the safety boundary is a safety region for the user to interact with the head-mounted display device.

Abstract: In a virtual object generation method, object description information of a three-dimensional virtual object to be generated is obtained. First features of each of a plurality of first spatial points in a target three-dimensional space are obtained based on the object description information. Each of the first features indicates a color and a positional relationship between the respective first spatial point and the three-dimensional virtual object. The plurality of first spatial points is distributed in the target three-dimensional space. Each of the first features is processed through a rendering model to obtain a color and a directed distance of each of the plurality of first spatial points. The directed distance indicates a distance between the respective first spatial point and a surface of the three-dimensional virtual object. The three-dimensional virtual object is generated based on the colors and the directed distances of the plurality of first spatial points.

Abstract: Method for generating a three-dimensional (3D) model includes: obtaining noise adding feature representations corresponding to noise data, the noise adding feature representations being configured to denoise at viewing angles, to obtain viewing angle images corresponding to an entity element; determining input feature representations of denoising network layers corresponding to the viewing angles when the denoising network layers denoise the noise adding feature representations; obtaining 3D shared information shared between 3D transformation matrices corresponding to the input feature representations, a 3D transformation matrix being obtained through dimension transformation of the input feature representations; adjusting the input feature representations based on the 3D shared information to obtain adjusted feature representations with a correspondence established between the input feature representations and the adjusted feature representations; and generating the viewing angle images based on the adjusted

Abstract: A three-dimensional (3D) entity model reconstruction method performed by a computer device includes obtaining 3D spatial information corresponding to a 3D entity model, performing voxel partitioning on the 3D entity model based on the 3D spatial information to determine voxels corresponding to the 3D entity model and connected in a tree structure, and constructing a 3D mesh corresponding to the 3D entity model based on the voxels. The voxels are distributed on a surface of the 3D entity model and represent a geometric shape of the surface of the 3D entity model.

Abstract: In a method for adjusting spatial elements, an environmental picture is displayed. The environmental picture includes a picture obtained by image acquisition of a scene region by an extended reality (XR) device. A first spatial element generated by three-dimensional layout recognition of the scene region is displayed based on the environmental picture. The first spatial element represents a three-dimensional simulation structure of a scene object in the scene region. An adjusted spatial layout element is displayed based on an adjustment operation on the first spatial element. The adjustment operation adjusts the three-dimensional simulation structure of the scene object. The adjusted spatial layout element is used to generate a three-dimensional virtual environment corresponding to the scene region in an XR application.

Abstract: In a safety boundary generation method, an image of a surrounding environment captured by a camera in a head-mounted display device is received. Three-dimensional reconstruction is performed based on the image of the surrounding environment to obtain a three-dimensional environment model. The three-dimensional environment model indicates a three-dimensional representation of the surrounding environment of a user that wears the head-mounted display device. Ground detection on the three-dimensional environment model is performed to obtain a plurality of ground boundary points. Each of the plurality of ground boundary points is an intersection point between a ground plane and an obstacle in the surrounding environment. A safety boundary is determined based on the plurality of ground boundary points. A region enclosed by the safety boundary is a safety region for the user to interact with the head-mounted display device.

Abstract: This application provides an image generation method performed by a computer device. The method includes: obtaining a target depth image including a target object in a real scene, and each pixel point in the target depth image having a depth value; segmenting the target depth image based on the depth value corresponding to each pixel point in the target depth image, to obtain a target object template image including a plurality of pixel points corresponding to the target object in the target depth image; obtaining M background images corresponding to a target scene, the target scene being a scene set associated with an image processing model, and M being an integer greater than or equal to 1; and superimposing the target object template image on the M background images, to generate M target scene images, the target scene images being configured for training the image processing model.

Abstract: This application relates to a method and an apparatus for determining three-dimensional layout information. The foregoing method includes: obtaining a first image and a second image obtained by photographing a same 3D region by a first camera and a second camera simultaneously; and generating three-dimensional layout information of the 3D region based on a relative position between the first camera and the second camera, a photographing parameter of each of the first camera and the second camera, the first image, and the second image, the three-dimensional layout information being configured for characterizing a three-dimensional spatial layout of at least one real object in the 3D region. The foregoing method can improve accuracy of layout information of the 3D region.

Abstract: A panoramic image formed by projecting a target place to a virtual environment can be generated based on a plurality of environment images obtained by observing the target place from different angles of view. A machine can automatically recognize and intelligently extract layout information of the target place based on the panoramic image, and build, by using the layout information, a target virtual environment for simulating the target place.

Abstract: A method and an apparatus for distortion coefficient calibration for an extended reality device is described.

Abstract: A method provided for 3D object localization predicts pairs of 2D bounding boxes. Each pair corresponds to a detected object in each of the two consecutive input monocular images. The method generates, for each detected object, a relative motion estimation specifying a relative motion between the two images. The method constructs an object cost volume by aggregating temporal features from the two images using the pairs of 2D bounding boxes and the relative motion estimation to predict a range of object depth candidates and a confidence score for each object depth candidate and an object depth from the object depth candidates. The method updates the relative motion estimation based on the object cost volume and the object depth to provide a refined object motion and a refined object depth. The method reconstructs a 3D bounding box for each detected object based on the refined object motion and refined object depth.

Abstract: A method for jointly removing rolling shutter (RS) distortions and blur artifacts in a single input RS and blurred image is presented. The method includes generating a plurality of RS blurred images from a camera, synthesizing RS blurred images from a set of GS sharp images, corresponding GS sharp depth maps, and synthesized RS camera motions by employing a structure-and-motion-aware RS distortion and blur rendering module to generate training data to train a single-view joint RS correction and deblurring convolutional neural network (CNN), and predicting an RS rectified and deblurred image from the single input RS and blurred image by employing the single-view joint RS correction and deblurring CNN.

Abstract: A method for improving geometry-based monocular structure from motion (SfM) by exploiting depth maps predicted by convolutional neural networks (CNNs) is presented. The method includes capturing a sequence of RGB images from an unlabeled monocular video stream obtained by a monocular camera, feeding the RGB images into a depth estimation/refinement module, outputting depth maps, feeding the depth maps and the RGB images to a pose estimation/refinement module, the depths maps and the RGB images collectively defining pseudo RGB-D images, outputting camera poses and point clouds, and constructing a 3D map of a surrounding environment displayed on a visualization device.

Abstract: A method is provided in an Advanced Driver-Assistance System (ADAS). The method extracts, from an input video stream including a plurality of images using a multi-task Convolutional Neural Network (CNN), shared features across different perception tasks. The perception tasks include object detection and other perception tasks. The method concurrently solves, using the multi-task CNN, the different perception tasks in a single pass by concurrently processing corresponding ones of the shared features by respective different branches of the multi-task CNN to provide a plurality of different perception task outputs. Each respective different branch corresponds to a respective one of the different perception tasks. The method forms a parametric representation of a driving scene as at least one top-view map responsive to the plurality of different perception task outputs.

Abstract: A computer-implemented method for implementing a self-supervised visual odometry framework using long-term modeling includes, within a pose network of the self-supervised visual odometry framework including a plurality of pose encoders, a convolution long short-term memory (ConvLSTM) module having a first-layer ConvLSTM and a second-layer ConvLSTM, and a pose prediction layer, performing a first stage of training over a first image sequence using photometric loss, depth smoothness loss and pose cycle consistency loss, and performing a second stage of training to finetune the second-layer ConvLSTM over a second image sequence longer than the first image sequence.

Abstract: Methods and systems for determining a path include detecting objects within a perspective image that shows a scene. Depth is predicted within the perspective image. Semantic segmentation is performed on the perspective image. An attention map is generated using the detected objects and the predicted depth. A refined top-down view of the scene is generated using the predicted depth and the semantic segmentation. A parametric top-down representation of the scene is determined using a relational graph model. A path through the scene is determined using the parametric top-down representation.

Abstract: A method provided for 3D object localization predicts pairs of 2D bounding boxes. Each pair corresponds to a detected object in each of the two consecutive input monocular images. The method generates, for each detected object, a relative motion estimation specifying a relative motion between the two images. The method constructs an object cost volume by aggregating temporal features from the two images using the pairs of 2D bounding boxes and the relative motion estimation to predict a range of object depth candidates and a confidence score for each object depth candidate and an object depth from the object depth candidates. The method updates the relative motion estimation based on the object cost volume and the object depth to provide a refined object motion and a refined object depth. The method reconstructs a 3D bounding box for each detected object based on the refined object motion and refined object depth.

Abstract: A method for correcting blur effects is presented. The method includes generating a plurality of images from a camera, synthesizing blurred images from sharp image counterparts to generate training data to train a structure-and-motion-aware convolutional neural network (CNN), and predicting a camera motion and a depth map from a single blurred image by employing the structure-and-motion-aware CNN to remove blurring from the single blurred image.

Abstract: A method for correcting rolling shutter (RS) effects is presented. The method includes generating a plurality of images from a camera, synthesizing RS images from global shutter (GS) counterparts to generate training data to train the structure-and-motion-aware convolutional neural network (CNN), and predicting an RS camera motion and an RS depth map from a single RS image by employing a structure-and-motion-aware CNN to remove RS distortions from the single RS image.

Abstract: A method for jointly removing rolling shutter (RS) distortions and blur artifacts in a single input RS and blurred image is presented. The method includes generating a plurality of RS blurred images from a camera, synthesizing RS blurred images from a set of GS sharp images, corresponding GS sharp depth maps, and synthesized RS camera motions by employing a structure-and-motion-aware RS distortion and blur rendering module to generate training data to train a single-view joint RS correction and deblurring convolutional neural network (CNN), and predicting an RS rectified and deblurred image from the single input RS and blurred image by employing the single-view joint RS correction and deblurring CNN.