Abstract: Disclosed is an improved approach for generated recordings from augmented reality systems from the perspective of a camera within the system. Instead of re-using rendered virtual content from the perspective of the user's eyes for AR recordings, additional virtual content is rendered from an additional perspective specifically for the AR recording. That additional virtual content is combined with image frames generated by a camera to form the AR recording.

Abstract: Disclosed approaches may leverage the actual spatial and reflective properties of a virtual environment—such as the size, shape, and orientation of a bidirectional reflectance distribution function (BRDF) lobe of a light path and its position relative to a reflection surface, a virtual screen, and a virtual camera—to produce, for a pixel, an anisotropic kernel filter having dimensions and weights that accurately reflect the spatial characteristics of the virtual environment as well as the reflective properties of the surface. In order to accomplish this, geometry may be computed that corresponds to a projection of a reflection of the BRDF lobe below the surface along a view vector to the pixel. Using this approach, the dimensions of the anisotropic filter kernel may correspond to the BRDF lobe to accurately reflect the spatial characteristics of the virtual environment as well as the reflective properties of the surface.

Abstract: In various examples, the actual spatial properties of a virtual environment are used to produce, for a pixel, an anisotropic filter kernel for a filter having dimensions and weights that accurately reflect the spatial characteristics of the virtual environment. Geometry of the virtual environment may be computed based at least in part on a projection of a light source onto a surface through an occluder, in order to determine a footprint that reflects a contribution of the light source to lighting conditions of the pixel associated with a point on the surface. The footprint may define a size, orientation, and/or shape of the anisotropic filter kernel and corresponding filter weights. The anisotropic filter kernel may be applied to the pixel to produce a graphically-rendered image of the virtual environment.

Abstract: The technical problem of creating an augmented reality (AR) experience that, on one hand, is accessible from a camera view user interface provided with a messaging client and that, also, can perform a modification based on a previously captured image of a user, is addressed by providing an AR component. When a user, while accessing the messaging client, engages a user selectable element representing the AR component in the camera view user interface, the messaging system loads the AR component in the messaging client. The AR component comprises a target media content object, which can be animation or live action video. The loaded AR component accesses a portrait image associated with a user and modifies the target media content using the portrait image. The resulting target media content object is displayed in the camera view user interface.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, method, and user interface to facilitate a shared control of a virtual object by two or more users. A virtual object is displayed by a first device, for example, as part of an augmented reality experience where the display of the object is overlaid on a real-world environment. User input indicative of a modification to the virtual object is received. The virtual object is modified, and a modified virtual object is displayed by a second device.

Abstract: A method includes receiving an image of a real environment using a camera worn by a user, and determining a portion of the image that comprises an object of interest. Based on the portion of the image that comprises the object of interest, a surface representing the object of interest is generated. Depth measurements of the real environment corresponding to the portion of the image comprising the object of interest are received and used to determine a depth of the surface representing the object of interest. The surface is posed in a coordinate system corresponding to the real environment based on the depth of the surface and a visibility of a virtual object is determined relative to the object of interest by comparing a model of the virtual object with the surface. The output image is generated based on the determined visibility of the virtual object.

Abstract: Apparatus and method for context-aware compression. For example, one embodiment of an apparatus comprises: ray traversal/intersection circuitry to traverse rays through a hierarchical acceleration data structure to identify intersections between rays and primitives of a graphics scene; matrix compression circuitry/logic to compress hierarchical transformation matrices to generate compressed hierarchical transformation matrices by quantizing N-bit floating point data elements associated with child transforms of the hierarchical transformation matrices to variable-bit floating point numbers or integers comprising offsets from a parent transform of the child transform; and an instance processor to generate a plurality of instances of one or more base geometric objects in accordance with the compressed hierarchical transformation matrices.

Abstract: An example method of image signal processing, comprising at least one of, receiving a set of high priority signals, receiving a set of low priority signals, reconfiguring a first portion of a pipeline to route the high priority signals through an in-line mode process and reconfiguring a second portion of the pipeline to route the low priority signals through an offline mode process.

Abstract: Provided is an automatic grading method including calculating a first strain ratio between a three-dimensional (3D) source avatar and a 3D target avatar, determining a mapping relationship between 3D source garment draped over the source avatar and a body portion of the source avatar, converting the source garment into 3D target garment draped over the target avatar, based on the first strain ratio and the mapping relationship, and outputting a two-dimensional (2D) target pattern constituting the target garment.

Abstract: A system and method for providing a weather effect in an image includes selecting at least one weather texture image indicating weather, and providing a weather effect in the image by overlapping the selected weather texture image on the image.

Abstract: Embodiments of the disclosure provide an image rendering method, an image rendering apparatus, an image processing device, and a computer-readable storage medium. In one embodiment, the method, apparatus, device, and medium involve obtaining an initial image of a current scene, and determining a first area and a second area on the initial image; rendering image data of the first area based on a first rendering rule, to obtain a first sub-image; rendering image data of the second area based on a second rendering rule, to obtain a second sub-image; and generating a target display image based on the first sub-image and the second sub-image, the first rendering rule being different from the second rendering rule.

Abstract: Methods for graphics processing are provided. One example method includes executing a plurality of kernels using a plurality of graphics processing units (GPUs), wherein responsibility for executing a corresponding kernel is divided into one or more portions each of which being assigned to a corresponding GPU. The method includes generating a plurality of dependency data at a first kernel as each of a first plurality of portions of the first kernel completes processing. The method includes checking dependency data from one or more portions of the first kernel prior to execution of a portion of a second kernel. The method includes delaying execution of the portion of the second kernel as long as the corresponding dependency data of the first kernel has not been met.

Abstract: A creation of simulation scenes from a specified view point is provided. The method consists in obtaining digital photographic images of the scene from the specified view point, detecting objects in the digital photographic images, extracting masks of the object, and associating a distance to the digital photographic image, and a lower distance to the object. The scene thus created provides a photorealistic scene wherein 3D objects can be inserted. According to the distances of the 3D objects, they can be displayed behind or beyond the masks, but always behind the digital photographic images that defines the background of the scene.

Abstract: A virtual local presence display apparatus, system and method is disclosed.

Abstract: A method of obtaining real world scale information for a scene includes obtaining at least one image of a plurality of objects in a scene; detecting at least some of the objects in the at least one image as corresponding to pre-determined objects; generating a 3D reconstruction of the scene based on the image content of the at least one image; determining a relative size of each object in the 3D reconstruction of the scene in at least one dimension, the relative size being defined in dimensions of the generated 3D reconstruction; where the relative size of each object is determined based on a distance between at least two points corresponding to that object as transformed into 3D space; obtaining a size probability distribution function for each object detected in the at least one image, each size probability distribution function defining a range of sizes in at least one dimension that a corresponding object is likely to possess in real world units; resealing the size probability distribution function for ea

Abstract: A computer system captures, via one or more cameras, information indicative of the physical environment, including respective portions of the physical environment that are in a field of view. The respective portions of the physical environment include a plurality of primary features of the physical environment and secondary features of the physical environment. After capturing the information indicative of the physical environment, the system displays a user interface, including concurrently displaying graphical representations of the plurality of primary features that are generated with a first level of fidelity to the corresponding plurality of primary features of the physical environment, and one or more graphical representations of secondary features that are generated with a second level of fidelity to the corresponding one or more secondary features of the physical environment, where the second level of fidelity is lower than the first level of fidelity.

Abstract: Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Abstract: The present invention provides an augmented reality service providing device, including: a sensing unit to photograph an object and to measure a current position of a mobile terminal; a communication unit to measure a communication distance between the mobile terminal and the object; a control unit to control to repeat a plurality of times the measurement of the current position and the communication distance while the mobile terminal is moving; a calculation unit to calculate an estimated position of the object based on the current position and the communication distance measured repeatedly a plurality of times; and a display unit to display virtual information on the object on the estimated position of the object.

Abstract: Devices, systems, and methods are provided for real-time object placement guidance in augmented reality experience. An example method may include receiving, by a device having a sensor, an indication of an object to be viewed in an physical environment of the device. The example method may also include determining a 3D model of the physical environment using data of the physical environment captured by the sensor. The example method may also include determining that a first surface in the 3D model of the environment is a first floor space, and a second surface in the 3D model of the environment is a first wall space. The example method may also include determining that a portion of the first surface is unoccupied and sized to fit the object.

Abstract: A method and an electronic apparatus of modifying a three-dimensional model are provided. In the method, multiple two-dimensional images are obtained by scanning a real environment Real objects of the real environment are identified in the two-dimensional images. A three-dimensional model is generated based on the two-dimensional images. The three-dimensional model is related to the depth information of the real objects. A modifying operation for modifying one or more virtual objects in the three-dimensional model is received. The virtual objects are corresponding to one or more real objects. The modifying operation is mapped into the two-dimensional images based on the identified result of the real objects in the two-dimensional image and the depth information of the real objects, to generate a modified image. A modified three-dimensional model is generated based on the modified image, to provide a proper modified result.