Abstract: A method includes obtaining an image that includes a plurality of pixels. The method includes identifying a first subset of the plurality of pixels that defines a particular geometry that a user of the electronic device is currently focusing on. The method includes warping the first subset of the plurality of pixels according to a first warping function that is based on a movement of the electronic device. The method includes warping a second subset of the plurality of pixels, that is different from the first subset, according to a second warping function that is different from the first warping function.

Abstract: An example of an information processing system generates a display mesh based on voxel data. The information processing system generates a voxel update range in a virtual space, based on game processing, and updates at least one of a density and a material ID, in the voxel data, for each of voxels corresponding to the voxel update range. The information processing system updates the display mesh corresponding to the updated voxel data. The information processing system, based on collision determination between a determination mesh and a determination shape corresponding to a determination target, generates an in-game behavior according to a material of the determination mesh at a collision position. The information processing system renders a virtual space including the display mesh, based on a texture corresponding to the material of the display mesh.

Abstract: An apparatus includes a communication interface configured to receive a compressed bitstream including a base mesh sub-bitstream and a processor operably coupled to the communication interface. The processor is configured to decode a plurality of submeshes from the base mesh sub-bitstream. The processor is also configured to subdivide a submesh of the plurality of submeshes according to a subdivision iteration count to generate at least one subdivided submesh, including determine a number of vertex positions for the at least one subdivided submesh by use of a number of vertices associated with an original submesh and by use of distortion information between the original submesh and a base mesh at each subdivision iteration associated with the subdivision iteration count. The processor is also configured to reconstruct at least a portion of a mesh-frame using the vertex positions corresponding to the at least one subdivided submesh.

Abstract: A system may perform motion capture using motion capture targets with concave reflector structures. For example, the motion capture target include a target body and a plurality of tracking markers located on respective portions of the surface of the target body. At least one tracking marker of the plurality of tracking markers may be a concave reflector structure including a tapered hole in the surface of the target body and at least a portion of a surface of the tapered hole may be reflective.

Abstract: Input images are obtained to include color data and depth data of a target. An image graph is generated based on the color data of the input images such that the image graph includes correlation values between the input images. A registration order is determined based on the image graph. Pose information of each input image is sequentially generated based on the registration order and the depth data of the input images. The input images are sequentially registered as registered images such that each registered image includes the pose information. A three-dimensional model of the target is reconstructed based on the registered images. The pose information may be generated and the elaborate three-dimensional model may be reconstructed by determining the registration order based on the color data and sequentially generating the pose information of the input images according to the registration order using the registered images.

Abstract: A system for video-based tracking of objects in one or more regions of interest. The system includes one or more cameras to capture image streams of the one or more regions of interest, the imaging streams including zero or more objects. A plurality of image processors of the system receive the captured image streams from the one or more cameras, and detect the one or more objects or object parts in the captured image streams and generate geometric and tracking data for detected objects. A fusion processor of the system receives the captured image streams and geometric and tracking data for detected objects from the plurality of image processors, and generates fused 3D referenced data using the detection and tracking data detected objects.

Abstract: A method includes obtaining a 2D image captured using an imaging sensor. The 2D image is associated with an imaging sensor pose. The method also includes providing the 2D image, the imaging sensor pose, and one or more additional imaging sensor poses to at least one machine learning model that is trained to generate a texture map and a depth map for the imaging sensor pose and each additional imaging sensor pose. The method further includes generating a stereo image pair based on the texture maps and the depth maps. The stereo image pair represents a 2.5D view of the 2D image. The 2.5D view includes a pair of images each including multiple collections of pixels and, for each collection of pixels, a common depth associated with the pixels in the collection of pixels. In addition, the method includes initiating display of the stereo image pair on an XR device.

Abstract: A deep learning network can perform three-dimensional (3D) image reconstruction of a scene from multi-view calibrated two-dimensional (2D) images. The network can include a convolutional neural network that performs feature extraction to generate feature pyramids corresponding to features at different levels of resolution. The feature pyramids can be used to compute a respective cost volume for each feature pyramid at each level of resolution, with the cost volume incorporating a learnable parameter that corresponds to a weight allocated to the “reference” feature pyramid relative to other feature pyramids. A depth map for each input image can be generated based at least in part on the cost volume.

Abstract: A method and a device of designing a metasurface, and a storage medium are provided. The method includes determining a phase distribution of an initial metasurface according to a preset image and a light source function of a first light source; the preset image includes a plurality of projection points in an array; performing a simulation based on the phase distribution of the initial metasurface and a second light source, so as to obtain a speckle image; if the speckle image does not meet a preset standard, optimizing the phase distribution of the initial metasurface through adjusting the projection points on the preset image, so as to determine a target phase distribution; generating the metasurface according to the target phase distribution.

Abstract: Systems and techniques are provided for generating a three-dimensional (3D) facial model. For example, a process can include obtaining at least one input image associated with a face. In some aspects, the process can include obtaining a pose for a 3D facial model associated with the face. In some examples, the process can include generating, by a machine learning model, the 3D facial model associated with the face. In some cases, one or more parameters associated with a shape component of the 3D facial model are conditioned on the pose. In some implementations, the 3D facial model is configured to vary in shape based on the pose for the 3D facial model associated with the face.

Abstract: An apparatus for use with an image display device configured for head-worn by a user, includes: a screen; and a processing unit configured to assign a first area of the screen to sense finger-action of the user; wherein the processing unit is configured to generate an electronic signal to cause a change in a content displayed by the display device based on the finger-action of the user sensed by the assigned first area of the screen of the apparatus.

Abstract: In some embodiments, the dynamic animation generation system can provide a deep learning framework to produce a large variety of martial arts movements in a controllable manner from unstructured motion capture data. The system can imitate animation layering using neural networks with the aim to overcome challenges when mixing, blending and editing movements from unaligned motion sources. The system can synthesize movements from given reference motions and simple user controls, and generate unseen sequences of locomotion, but also reconstruct signature motions of different fighters. For achieving this task, the dynamic animation generation system can adopt a modular framework that is composed of the motion generator, that maps the trajectories of a number of key joints and root trajectory to the full body motion, and a set of different control modules that map the user inputs to such trajectories.

Abstract: Rendering an avatar for a user in a communication session includes obtaining one or more identity textures for the user for the session and obtaining, throughout the session, shading latents for the user. The shading latents are derived from lighting and other information and are utilized as input into a decoder to obtain one or more neural maps. A target texture is generated by warping the identity textures based on the neural maps. The target texture is used for rendering the avatar.

Abstract: A method for controlling XR content in an XR CAVE environment comprising projecting XR content into the XR CAVE, detecting a plurality of pedestrians and calculating the average movement speed and direction using a camera disposed within the XR CAVE, correcting the projection position and speed of the XR content based on the calculated average speed and direction, and projecting the corrected XR content onto the wall(s) of the XR CAVE.

Abstract: Systems and methods generate a hybrid lighting model for rendering objects within an image. The hybrid lighting model includes lighting effects attributed to a first source, such as the sun, and to a second source, such as spatially-varying effects of objects within the image. The hybrid lighting model may be generated for an input image and then one or more virtual objects may be rendered to appear as if part of the input image, where the hybrid lighting model is used to apply one or more lighting effects to the one or more virtual objects.

Abstract: Disclosed are a method and device for generating a three-dimensional (3D) model through RGB-D camera tracking. A 3D model generation method may comprise the steps of: identifying RGB images and depth images; determining an initial camera pose of an RGB-D camera for each original frame of the depth images and the RGB images; determining a final camera pose for each of the original frames by updating the initial camera pose by using differences between an original frame, transformed according to the initial camera pose, and remaining original frame among two consecutive original frames of the depth images and the RGB images; selecting key vertexes capable of covering all original vertexes among original vertexes; and generating a 3D model by using key-frames, corresponding to the key vertexes in the original frames of the depth images and the RGB images, and final camera poses corresponding to the key-frames.

Abstract: Computer implemented measurement space deformation method and apparatus. A method embodiment comprises defining a measurement space as a multi-dimensional space, populating the measurement space with a set of pre-computed example avatars, searching the measurement space for a subset of example avatars, and interpolating between the example avatars of the subset.

Abstract: Method, apparatus, and system for generating temporally correlated UV atlases are provided. The process may include generating plurality of consistent UV charts based on partition information associated with one or more previous frames; removing one or more non-manifold vertices from the plurality of consistent UV charts based on assigning one or more incident faces associated with the one or more non-manifold vertices to corresponding UV charts; merging more than one of the plurality of consistent UV charts based on a similarity between the plurality of consistent UV charts and one or more reference charts associated with the one or more previous frames; and generating temporally correlated UV atlases for the current frame based on aligning the plurality of consistent UV charts with the one or more reference charts associated with the one or more previous frames.

Abstract: A method for vision-based tool localization (VTL) in a robotic assembly system including one or more calibrated cameras, the method comprising capturing a plurality of images of the tool contact area from a plurality of different vantage points, determining an estimated position of the tool contact area based on an image, and refining the estimated position based on another image from another vantage point. The method further comprises providing the refined position to the robotic assembly system to enable accurate control of the tool by the robotic assembly system.

Abstract: A method for acquiring a texture of a three-dimensional (3D) model includes: acquiring at least two 3D networks generated by a target object based on a plurality of angles, the at least two 3D networks including a first correspondence between point cloud information and color information of the target object, and first camera poses of the target object; acquiring an offset between 3D points used for recording the same position of the target object in the at least two 3D networks according to the first camera poses respectively included in the at least two 3D networks; updating the first correspondence according to the offset, to acquire a second correspondence between the point cloud information and the color information of the target object; and acquiring a surface color texture of a 3D model of the target object according to the second correspondence.