Abstract: Cross-platform sharing of displayed content is provided. In one or more implementations, cross-platform sharing of display content may include displaying a first rendering of display content with a first display of a first device, and, responsive to a request to capture and/or share the displayed content, generating a second rendering of the display content, the second rendering different from the first rendering. The second rendering may be different from the first rendering in such a way that, when the second rendering is displayed by a second display of a second device having a different form factor and/or display technology, the displayed second rendering accurately represents the perceived view of the user of the first device.

Abstract: A device may include a processor configured to determine a first position of a physical camera in a physical environment, transmit the first position to a second device, obtain metadata of the physical camera, and cause a modification of a virtual camera of the second device based on the metadata. The processor is also configured to obtain, from the second device, at least one virtual image frame captured by the virtual camera placed at a second position in a computer-generated environment, and to obtain, from the physical camera, at least one image frame captured from the physical environment. The processor is also configured to generate a composite image frame including at least a portion of at least one virtual image frame composited with at least a portion of the at least one image frame, and display the composite image frame.

Abstract: Various implementations disclosed herein include devices, systems, and methods that presents playback of application content within a three-dimensional (3D) environment. An exemplary process presents a first set of views that includes application content provided by the application within a 3D environment. The first set of views are provided from a first set of viewpoints during execution of the application. The process records of the execution of the application based on recording program state information and changes to the application content that are determined based on user interactions, and presents a second set of views including a playback of the application content within the 3D environment based on the recording. The second set of views are provided from a second set of viewpoints that are different than the first set of viewpoints.

Abstract: Various implementations disclosed herein include devices, systems, and methods that incorporate enhanced metadata into spatial and/or mixed reality capture environments. For example, a process may include capturing an original video content recording that includes one or more frames depicting a first view provided by a head mounted device (HMD) at one or more instants in time. The process may further generate an adapted video content recording by adapting brightness of the one or more frames of the original video content recording to account for a difference between a first viewing condition associated with the first view and a second viewing condition associated with non-immersive viewing. The process may further generate metadata comprising information associated with the first viewing condition and the metadata may be associated with the adapted video content recording.

Abstract: Cross-platform sharing of displayed content is provided. In one or more implementations, cross-platform sharing of display content may include displaying a first rendering of display content with a first display of a first device, and, responsive to a request to capture and/or share the displayed content, generating a second rendering of the display content, the second rendering different from the first rendering. The second rendering may be different from the first rendering in such a way that, when the second rendering is displayed by a second display of a second device having a different form factor and/or display technology, the displayed second rendering accurately represents the perceived view of the user of the first device.

Abstract: The presentation of overlay content for a 3D scene is adjusted based on head tilt. Image content for a 3D scene is obtained by a head-mounted device. A target depth for the overlay content is determined based on depth information for the content of the scene corresponding to a first disparity. In response to a tilt of the head-mounted device satisfying a correction criterion, an adjusted target depth value for the overlay content is determined, and the disparity of the overlay content is reduced in accordance with the adjusted target depth value. Alternatively, an adjusted vertical disparity and an adjusted horizontal disparity are determined based on the first disparity and the orientation of the head-mounted device, and applied to the overlay content.

Abstract: Various implementations include devices, systems, and methods that reduce HMD cover glass-induced artifacts. For example, a process may obtain a reflection model for predicting image reflection locations based on an image light source location for images captured by light sources. The reflection model is generated based on camera positioning relative to the regions of the transparent structure and curvature of the regions of the transparent structure. The process identifies a light source region in a first image captured by a first camera of the HMD and predicts a reflection region in the first image based on the reflection model and the light source region. Replacement content for the first image is generated based on content from a second image captured by a second camera of the HMD and the first and second images are displayed such that the first image is provided with the replacement content replacing the reflection region.

Abstract: A device for positional synchronization of virtual and physical cameras may include a processor configured to determine a first position of a physical camera relative to another electronic device in a physical environment. The processor may be configured to initiate positioning of a virtual camera in a second position within a computer-generated environment, wherein the second position relative to a representation of the person in the computer-generated environment coincides with the first position. The processor may be configured to receive an image frame captured by the physical camera and a virtual image frame generated by the virtual camera. The processor may be configured to generate a computer-generated reality image frame that includes at least a portion of the image frame composited with at least a portion of the virtual image frame.

Abstract: In some embodiments, a computer system displays a caption for a media item at different depths depending on the depth of the portion of the media item over which the caption is displayed. In some embodiments, a computer system displays a user interface element that includes information associated with the media item at different locations relative to the media item depending on attention of the user. In some embodiments, a computer system displays a user interface element that includes information associated with the media item with different visual appearances depending on visual characteristics of the portion of the media item over which the user interface element is displayed.

Abstract: A device for positional synchronization of virtual and physical cameras may include a processor configured to determine a first position of a physical camera relative to another electronic device in a physical environment. The processor may be configured to initiate positioning of a virtual camera in a second position within a computer-generated environment, wherein the second position relative to a representation of the person in the computer-generated environment coincides with the first position. The processor may be configured to receive an image frame captured by the physical camera and a virtual image frame generated by the virtual camera. The processor may be configured to generate a computer-generated reality image frame that includes at least a portion of the image frame composited with at least a portion of the virtual image frame.

Abstract: Techniques are disclosed relating to encoding recorded content for distribution to other computing devices. In some embodiments, a first computing device creates recorded content for transmission to a second computing device configured to present the recorded content. To encode the recorded content, the first computing device detects, via a network interface of the first computing device, one or more computing nodes available to encode the recorded content in one or more formats supported by the second computing device. The first computing device offloads the recorded content via the network interface to the one or more computing nodes for encoding in the one or more formats. In some embodiments, the second computing device receives a request from a user to stream content recorded by a first computing device and requests the content in a first format being encoded by a computing node assisting the first computing device.

Abstract: Implementations of the subject technology provide systems and methods for recording an extended reality experience in a way that allows the experience to be played back at a later time from a different viewpoint or perspective. This allows computer-generated content that was rendered for display to a user during the recording, to be re-rendered during playback at the correct time and location in the recording, but from a different perspective. In order to facilitate this type of viewer-centric playback, the recording includes a computer-generated content track that references resources for re-rendering the computer-generated content at each point in time in the recording.

Abstract: Various implementations disclosed herein include devices, systems, and methods that generates a stereo view of a three-dimensional (3D) environment by rendering a first eye view and reusing pixels from the first eye view for a second eye view based on a disparity indicative characteristic (e.g., identifying regions of pixel difference/similarity in the views). For example, an example process may include rendering a first view for a first viewpoint (e.g., a left eye view) of a three-dimensional (3D) environment. The process may further include estimating differences between the first view and a second view for a second viewpoint (e.g., a right eye view) of the 3D environment. The process may further include rendering the second view for the second viewpoint based on the estimated differences.

Abstract: A set of planes is determined based on the vertices of a three-dimensional scene. The set of planes is determined independent of any particular orientation with respect to a reference viewpoint of the three-dimensional scene. The vertices are associated with the set of planes and respective textures are associated with the set of planes based on the associated vertices.

Abstract: A decoding computing device receives a bit stream for compressed volumetric visual content, such as immersive media. The bit stream includes video encoded image frames comprising packed attribute patch images, depth maps for a 3D scene represented by the volumetric visual content, and a variable transparency layer indicating boundaries of objects at different depths within the 3D scene represented by the volumetric visual content. Instead of generating a mesh having a large number of vertices, the decoder generates a lower resolution mesh and applies the variable transparency layer, such that portions of triangles or polygons of the lower resolution mesh extending beyond the boundaries of the objects at the different depths of the 3D scene are made transparent and not shown as being stretched across large depth distances.

Abstract: Aspects of the subject technology provide efficient rendering of three-dimensional scenes, including static or mostly static three-dimensional scenes. The efficient rendering may be achieved by providing a mesh atlas that helps to reduce the number of transparent pixels in a scene that are rendered, facilitate use of GPU depth testing to avoid rendering occluded pixels that are later replaced by opaque foreground pixels, and/or facilitate compact packing of image objects into an atlas. The mesh atlas can be generated in advance, and used at rendering time, for efficient rendering of a three-dimensional scene from a current point of view of a user.

Abstract: Implementations of the subject technology provides analyzing a recording of content. The subject technology generates metadata information based at least in part on the analyzing. The subject technology identifies, based at least in part on at least one of a user preference or a detected event, a region of interest or an object of interest in the recording of content. Based at least in part on the identified region of interest or object of interest, the subject technology generates a modified version of the recording of content. Further, the subject technology stores the modified version of the recording of content for subsequent playback on an electronic device.

Abstract: Aspects of the subject technology relate to encoding of video frames having content with a variable resolution that varies within the video frame. Aspects of the subject technology can provide an efficient encoding by using smaller macroblocks for lower resolution content within the video frame, and larger macroblocks for higher resolution content within the video frame. An encoder may be provided with resolution information for the content of the video frame, which can be used by the encoder to determine macroblock sizes, macroblock divisions, and/or prediction modes for the encoding of the video frame.

Abstract: A decoding computing device determines a focus area of a view of volumetric visual content, such as a three-dimensional (3D) object or scene, and indicates the focus area in a request for compressed volumetric visual content. A server provides a depth map and an attribute atlas that includes attribute and/or texture information for the focus area that is signaled at a higher resolution than other portions of the volumetric visual content. The decoding device also applies one or more mesh simplification techniques, wherein a higher resolution mesh is used for the focus area. The textures/attributes are projected onto the mesh representations, wherein the mesh representations and projected textures are used to reconstruct the volumetric visual content. Hole filling techniques are then applied, wherein a more sophisticated hole filling technique is used in the focus area.

Abstract: A decoding computing device receives a bit stream for compressed 3D volumetric content. The bit stream includes video encoded image frames comprising packed attribute patch images and depth maps for the 3D volumetric content. Instead of generating a mesh having a vertex for each depth value signaled in the depth map, the decoder performs a real-time mesh simplification process to reduce a resolution of the mesh, such that the mesh resolution is reduced without exceeding an error threshold, which may be dynamically determined. Additionally, the decoder may perform a re-meshing of particular regions of the mesh for the 3D volumetric content to avoid cracks or gaps.