Abstract: The present disclosure provides a virtual reality-based control method, apparatus, terminal, and storage medium, wherein the virtual reality-based control method includes: displaying a virtual entity and a suspended bubble associated with the virtual entity in a virtual reality space, the suspended bubble being configured to indicate a function of the virtual entity; and controlling, in response to a movement event of the virtual entity in the virtual reality space, the suspended bubble to follow the virtual entity to move.

Abstract: In one embodiment, a method of intermingling stereoscopic and conforming virtual content to a bounded surface is performed at a device that includes one or more processors, non-transitory memory, and one or more displays. The method includes displaying a bounded surface within a native user computer-generated reality (CGR) environment, wherein the bounded surface is displayed based on a first set of world coordinates characterizing the native user CGR environment. The method further includes displaying a first stereoscopic virtual object within a perimeter of a first side of the bounded surface, wherein the first stereoscopic virtual object is displayed in accordance with a second set of world coordinates that is different from the first set of world coordinates characterizing the native user CGR environment.

Abstract: The invention relates to a computer-implemented method and an associated device for modelling a joint of a patient for real-time assistance in performing at least one bone tunnel by arthroscopy in said joint, said method comprising: receiving a stream of intraoperative monocular two-dimensional images obtained using an imaging device, each of the intraoperative monocular two-dimensional images being obtained at a time t and comprising at least a region of interest of the joint; for each intraoperative monocular two-dimensional image, calculating a depth map associated with time t by implementing a first previously trained learning model, said first learning model being configured to receive as input at least said intraoperative monocular two-dimensional image obtained at time t; calculating a current partial three-dimensional model of the patient's joint associated with a current time tc, on the basis of at least one depth map associated with the current time tc, at least one depth map associated with a previ

Abstract: Hardware tessellation units include a sub-division logic block that comprises hardware logic arranged to perform a sub-division of a patch into two (or more) sub-patches. The hardware tessellation units also include a decision logic block that is configured to determine whether a patch is to be sub-divided or not and one or more hardware elements that control the order in which tessellation occurs. In various examples, this hardware element is a patch stack that operates a first-in-last-out scheme and in other examples, there are one or more selection logic blocks that are configured to receive patch data for more than one patch or sub-patch and output the patch data for a selected one of the received patches or sub-patches.

Abstract: The subject technology receives information for a product. The subject technology generates a 3D model file of the product in a first format. The subject technology converts the 3D model file to a 3D object file in a second format. The subject technology associates the 3D object file to the product in a product catalog service. The subject technology publishes an augmented reality (AR) content generator corresponding to the product.

Abstract: A computing device is configured to obtain gridline information for a three-dimensional drawing file and generate a two-dimensional view of the three-dimensional drawing file that includes (1) at least one gridline corresponding to the obtained gridline information, (2) at least one intersection between two meshes, and (3) initial dimensioning information involving (a) the at least one gridline and (b) at least one of the two meshes. Based on a user request to adjust a perspective of the two-dimensional view, the computing device adjusts the perspective of the two-dimensional view and thereby generates an updated two-dimensional view that includes updated dimensioning information corresponding to one or more meshes displayed in the updated two-dimensional view.

Abstract: A method, system, and computer-readable medium for executing a task is disclosed. The method includes receiving input data and computing instructions, launching a workgroup including wavefronts to execute the task, wherein the launching causes the wavefronts to process the input data by sharing intermediate results and resources, and adjusting the operation based on characteristics of the wavefronts. The characteristics include data dependencies, computational load, memory usage, and execution timing requirements. The wavefronts execute the task in stages, where each stage processes portions of input data and data generated by other wavefronts.

Abstract: Three-dimensional object edit and visualization techniques and systems are described. In a first example, a content navigation control is implemented by a content editing system to aid navigation through a history of how a three-dimensional environment and a three-dimensional object included in the environment is created. In a second example, the content editing system is configured to streamline placement of a three-dimensional object within a three-dimensional environment. The content editing system, for instance, generates a manipulation visualization in support of corresponding editing operations to act as a guide, e.g., as an alignment guide or an option guide. In a third example, the content editing system implements a shadow control that is usable as part of an editing and as a visualization to control rendering of illumination within a three-dimensional environment.

Abstract: A method of facilitating a virtual conference includes determining that a first object is non-directional, the first object being presented within an original image on a first display within the virtual conference; based on determining that the first object is non-directional, reversing an appearance and a location of the first object to generate a reversed first object; determining that a second object is directional, the second object being presented within the original image on the first display within the virtual conference; based on determining that the second object is directional, transferring a location of the second object while maintaining an orientation of the second object to generate a transferred second object; and causing a second display to generate a transformed image, the transformed image including the reversed first object and the transferred second object.

Abstract: A system for rendering an augmented image frame includes a sensor, an interface, a memory, a processor, and a display. The sensor captures an environment image of an external environment of a vehicle that includes a physical object. The interface receives an augmentation class to be applied to the environment image frame. The memory stores an augmentation engine that causes the processor to receive the environment image frame and the augmentation class. The augmentation engine further causes the processor to determine an identity and location of the physical object, and retrieve a digital object associated with the physical object and the augmentation class. Subsequently, an augmented image frame is rendered including the digital object disposed at the location of the physical object in the external environment. Finally, the display depicts the augmented image frame to present the user with an aesthetically appealing synthetic view of the external environment.

Abstract: An information processing device includes a control unit that performs control to output information regarding an image of a user viewpoint in a virtual space to a user terminal, in which the control unit acquires a characteristic of at least one user related to an event performed in the virtual space and controls movement of a position of an indicator corresponding to a performer in the virtual space based on a position of an indicator corresponding to at least one user whose characteristic satisfies a condition.

Abstract: Methods and systems related to processing multiple data streams are disclosed. For example, a computing device may receive a data stream corresponding to a content item, and generate additional data streams by sampling the data stream, for example, using different sampling regions. Another data stream at a higher resolution than each of the additional data streams may then be generated by combining elements (e.g., pixels) of the additional data streams.

Abstract: Example implementations are directed to recapturing or realigning a room or scene in an artificial reality (XR) environment as a three dimensional (3D) space. To initially capture a room, a user manually annotates, using an XR system, one or more walls around the XR system. The XR system then aligns or localizes itself relative to the room. Once a room is captured and stored, and the user in the future enters the room again, the XR system may not be able to align itself in the room due to, for example, lighting conditions. Rather than require the user to manually recapture every wall captured during the initial room capture, the XR system can select one wall for recapture and, after receiving an annotation or marking of the one wall from the user, use the selection to realign the XR system relative to the room.

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that detects shadows, removes shadows, and synthesizes shadows in a joint-framework. In particular, the disclosed systems access an object mask of an object and a digital image depicting the object and a shadow of the object. Furthermore, the disclosed systems perform object-centered shadow detection and removal to generate a modified digital image without the shadow by utilizing a shadow analyzer model. Moreover, the disclosed systems receive a user interaction to manipulate an object and generate a modified shadow utilizing a shadow synthesis model where the shadow synthesis model is conditioned on a shadow mask generated by the shadow analyzer model.

Abstract: A wearable display system includes one or more emissive micro-displays, e.g., micro-LED displays. The micro-displays may be monochrome micro-displays or full-color micro-displays. The micro-displays may include arrays of light emitters. Light collimators may be utilized to narrow the angular emission profile of light emitted by the light emitters. Where a plurality of emissive micro-displays is utilized, the micro-displays may be positioned at different sides of an optical combiner, e.g., an X-cube prism which receives light rays from different micro-displays and outputs the light rays from the same face of the cube. The optical combiner directs the light to projection optics, which outputs the light to an eyepiece that relays the light to a user's eye. The eyepiece may output the light to the user's eye with different amounts of wavefront divergence, to place virtual content on different depth planes.

Abstract: A three-dimensional optical scanning system for scanning a three-dimensional dental object, the system including a first scanning station configured to receive a first three-dimensional dental object, and further configured to rotate around a first axis during the scanning; and an imaging unit including at least one camera and configured to rotate around an imaging unit axis during the scanning, wherein the optical scanning system is configured such that the imaging unit acquires a plurality of two-dimensional images of the first three-dimensional dental object corresponding to a set of predefined static relative positions between the imaging unit and the first scanning station for generating a first three-dimensional digital representation of the first three-dimensional dental object.

Abstract: A computer-implemented method for representing virtual objects outside of a display screen, comprising: receiving data defining a virtual environment that comprises a plurality of virtual objects, each virtual object having a virtual location within the virtual environment, displaying, on a display screen, a portion of the virtual environment that is within a field of view controlled by a user determining if at least one virtual object of a subset of the virtual objects is outside of the field of view but at least partially within a virtual display area adjacent the display screen. In response to determining that a virtual object of the subset of the virtual objects is at least partially within the virtual display area, the method comprises converting the virtual location of the virtual object into a real-world location, and displaying a virtual element representing the virtual object at the real-world location using a mixed-reality display device.

Abstract: Implementations described herein relate to methods, systems, and computer-readable media to render an object with a decal texture applied thereon. In some implementations, a computer-implemented method includes receiving a polygon mesh of an object. The method further includes determining a positional map and a normal map by applying a rasterization function to the polygon mesh. The method further includes receiving a target texture map associated with the object and a decal texture to be placed on the target texture map. The method further includes determining a position, an orientation, and a scale of the decal texture with respect to a local space of the polygon mesh. The method further includes determining placement of the decal texture onto the target texture map by performing a per-pixel projection operation. The method further include rendering the object with the decal texture applied thereon.

Abstract: A method includes obtaining images of a scene captured using a stereo pair of imaging sensors of an XR device and depth data associated with the images, where the scene includes multiple objects. The method also includes obtaining volume-based 3D models of the objects. The method further includes, for one or more first objects, performing depth-based reprojection of the one or more 3D models of the one or more first objects to left and right virtual views based on one or more depths of the one or more first objects. The method also includes, for one or more second objects, performing constant-depth reprojection of the one or more 3D models of the one or more second objects to the left and right virtual views based on a specified depth. In addition, the method includes rendering the left and right virtual views for presentation by the XR device.

Abstract: Cheating deterrence in electronic reality environments during a testing activity is improved over real-world anti-cheating and test monitoring by displaying and recording use of virtual scratch paper by user, such as student, that is wearing an electronic reality display device and engaged in the testing activity. A monitor of the testing activity, such as a teacher, can watch the recorded use of the virtual scratch paper by the user to help deter cheating.