Abstract: Implementations described herein relate to methods, systems, and computer-readable media to relight a video. In some implementations, a computer-implemented method includes receiving a plurality of frames of a video. Each video frame includes depth data and color data for a plurality of pixels. The method further includes segmenting each frame based on the depth data to classify each pixel as a foreground pixel or a background pixel. The method further includes setting depth value of each background pixel to a fixed depth value and applying a Gaussian filter to smooth depth value for the plurality of pixels. The method further includes calculating surface normals based on the depth values of the plurality of pixels. The method further includes rendering a relighted frame by adding a virtual light based on the surface normals and the color data.

Abstract: An information processing apparatus obtains constraint information representing constraints in placing a plurality of image capturing apparatuses that capture images for generating a virtual viewpoint image, obtains placement information based on the obtained constraint information and relating to placement of the plurality of image capturing apparatuses, and outputs the obtained placement information of the plurality of image capturing apparatuses.

Abstract: A graphics processing architecture in one example performs vertex manipulation operations and pixel manipulation operations by transmitting vertex data to a general purpose register block, and performing vertex operations on the vertex data by a processor unless the general purpose register block does not have enough available space therein to store incoming vertex data; and continues pixel calculation operations that are to be or are currently being performed by the processor based on instructions maintained in an instruction store until enough registers within the general purpose register block become available.

Abstract: A method for structure inspection includes the steps of: selecting a structure; providing a vehicle having an imaging device; moving the vehicle in proximity of the structure; capturing two-dimensional images of surfaces of the structure with the imaging device; storing the two-dimensional images, wherein each image includes associated position information related to the surface; producing a three-dimensional virtual model from the stored two-dimensional images; displaying the three-dimensional virtual model on a user device; selecting a location on the display of the three-dimensional model; using the selected location to identify a corresponding one of the stored position information associated with at least one of the two-dimensional images; and displaying the at least one two-dimensional image on the user device.

Abstract: A method for virtual fitting, includes providing a customer avatar and a garment model of a selected garment; registering the garment model to the customer avatar to obtain a dressed garment; capturing a motion image of the customer; performing simulation to the customer avatar and the dressed garment based on the motion image; rendering the simulated customer avatar-garment; and displaying the rendered avatar-garment on a displaying device. Rigging is performed to the dressed garment to obtain rigged garment, and cloth simulation is performed to the dressed garment model to obtain simulated cloth. Each vertex of the simulated cloth has a distance to nearest body mesh vertices on the simulated customer avatar, and the simulated garment is the rigged garment when the distance is less than 0 or the distance is greater than a threshold distance. Air mesh between the avatar and the garment is used for collision detection.

Abstract: In various example embodiments, techniques are provided for representing uncertainty when displaying a rendered view of a multi-dimensional mesh (e.g., created by SfM photogrammetry) in a user interface by applying a real-time, obfuscation filter that modifies the rendered view based on uncertainty in screen space. Where the multi-dimensional mesh is within a limit of data accuracy, the rendered view is shown without modification (i.e. as normal), and a user may trust the information displayed. Where the multi-dimensional mesh is beyond the limit of data accuracy, the obfuscation filter obfuscates detail (e.g., by blurring, pixilating, edge enforcing, etc.) in the rendered view so that a user may visually perceive the uncertainty. The amount of obfuscation may be weighted based on uncertainty to allow the user to visually quantify uncertainty.

Abstract: A method for displaying an image is applied to an electronic device including a first camera and a second camera. The method includes: acquiring a first real-time image collected by the first camera and a second real-time image collected by the second camera, the first real-time image being different from the second real-time image; determining a target object in the first real-time image; generating a real-time virtual image corresponding to the target object; and displaying the real-time virtual image on the second real-time image.

Abstract: One or more disclosed techniques comprise generating an extended reality (XR) template at a first location, where the XR template represents a virtual space in an XR environment that is associated with a real-world object at a second location, and generating a template object that represents the real-world object, wherein the template object is included in the XR template. The techniques further comprise modifying the template object to include a marker that corresponds to a real-world marker associated with the real-world object, generating a first template XR object that corresponds a first XR object to be displayed in the XR environment, where the first template XR object is positioned within the XR template relative to the marker included on the template object, and assigning the XR template to the real-world marker.

Abstract: In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.

Abstract: A video reproduction apparatus determines a position of a joint of a performer based on a sensing result by a sensor. The video reproduction apparatus calculates an angle formed by a joint used for scoring an element of an athletic event by using calculation information regarding a calculation formula for calculating the angle and the determined position of the joint of the performer. The video reproduction apparatus displays a performance image of the performer acquired from a camera and a scoring image having the angle displayed on a 3D model image of the performer which is generated according to the sensing result so as to be compared with each other.

Abstract: Enhanced techniques for volumetric stage mapping based on a calibration object are described. An example method includes obtaining a plurality of images of a calibration object, the images being obtained from a plurality of image sensors positioned about the calibration object, and the calibration object having location information associated with a volumetric capture surface on which the calibration object is located; generating, based on the images, a reconstructed object associated with the calibration object, the reconstructed object representing a three-dimensional representation of the calibration object; determining reconstruction quality information associated with the reconstructed object, the determination being based on a ground-truth representation of the calibration object; and outputting the reconstruction quality information.

Abstract: Systems and methods described herein are provided for receiving location information for a real-world object in a user's augmented reality (AR) view, determining a direction of travel of a user, determining an activity zone shape associated with the direction of travel, responsive to a determination that the real-world object is within the activity zone, rendering on an AR display AR information associated with the real-world object at a location outside an area of the AR display used to display the activity zone. Some embodiments continue to display AR information in the same location if the user changes his or her gaze to read the AR information. Some embodiments render on an AR display subtle highlighting if an object is inside an activity zone and prominent highlighting if an object is outside an activity zone.

Abstract: A computer system detects an object in a field-of-view (FOV) using at least one sensor coupled to the computer system and determines a shape of the object using a processor of the computer system. An overlay element having the shape and a changeable characteristic is created and a value of a parameter is obtained. A representation of the overlay element is rendered with the changeable characteristic set based on a value of the parameter and presented on a display of the computer system to at least partially occlude the object.

Abstract: In some implementations, a method includes obtaining a computer graphic generated based on one or more visual elements within a first video frame. In some implementations, the first video frame is associated with a first time. In some implementations, the method includes obtaining a second video frame associated with a second time. In some implementations, the second time is different from the first time. In some implementations, the method includes applying an intensity transformation to the computer graphic in order to generate a transformed computer graphic. In some implementations, the intensity transformation is based on an intensity difference between the first video frame and the second video frame. In some implementations, the method includes rendering the transformed computer graphic based on one or more visual elements within the second video frame.

Abstract: A method is provided for determining angular relationships from a point of interest to a plurality of peripheral points of interest on a map. One or more cost functions from the point of interest to the plurality of the peripheral points of interest on the map are analyzed. A plurality of vectors emanating from the point of interest to the plurality of the peripheral points of interest on a different representation of the map are displayed. Another method is provided for identifying points of interest on a map. Regions of high interest are identified on the map. Regions of low interest are identified on the map. The regions of high interest are expanded on a different representation of the map. The regions of low interest are contracted by an amount proportional to an amount the regions of high interest are expanded on the different representation of the map.

Abstract: A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute information. Attribute values for at least one starting point are included in a compressed attribute information file and attribute correction values are included in the compressed attribute information file. Attribute values are predicted based, at least in part, on attribute values of neighboring points. The predicted attribute values are compared to attribute values of a point cloud prior to compression to determine attribute correction values. In order to improve computing efficiency and/or repeatability, fixed-point number representations are used when determining predicted attribute values and attribute correction values. A decoder follows a similar prediction process as an encoder and corrects predicted values using attribute correction values included in a compressed attribute information file using fixed-point number representations.

Abstract: According to an embodiment, a method for 3D scanning at least a part of a surface of an object is disclosed. The method includes recording, using an optical scanner comprising at least one camera, one or more test 2D images of the at least a part of the surface of the object; automatically identifying a first segment of a first level of interest within the test 2D images, the first segment imaging a region of interest on the at least a part of the surface of the object; identifying a first 3D volume comprising the region of interest of the at least a part of the surface of the object; selecting a first input defining a first resolution and/or a first coverage; and 3D scanning the at least a part of the surface of the object within the first 3D volume using the first input.

Abstract: Disclosed is a system and method for rendering point clouds via a hybrid data point and construct visualization. The system receives a point cloud of a three-dimensional (“3D”) environment, and differentiates a first set of the point cloud data points from a second set of the data points based on a position of each data point relative to a specified render position. The system generates a first visualization from values of each of the first set of data points, and a second visualization from values of a set of constructs that replace the second set of data points. Each construct has a polygonal shape and a singular set of values defined from the values of two or more of the second set of data points. The system presents a final render of the 3D environment from the render position by combining the first visualization with the second visualization.

Abstract: A system for handling 3 dimensional spatial information, the system including: a specialised application layer for the production of visual interactive applications associated with the 3 dimensional spatial information; a generic foundation client layer providing 3 dimensional spatial information interrogation routines, including a message passing interface; and a voxel server for interconnected to said generic foundation client via said message passing interface for the storage of 3 dimensional spatial information as a voxel data base.

Abstract: An information processing apparatus comprises a first acquisition unit configured to acquire information concerning an index based on a captured image including a physical object added with the index, a second acquisition unit configured to acquire a thumbnail image of the physical object, and a holding unit configured to hold, in a memory, physical object information including the information and the thumbnail image of the physical object added with the index.