Abstract: Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the preintegration term, and the wearable head device presents the virtual content based on the position of the device.

Abstract: A system aligns a 3D model of an environment with image frames of the environment and generates a visualization interface that displays a portion of the 3D model and a corresponding image frame. The system receives LIDAR data collected in the environment and generates a 3D model based on the LIDAR data. For each image frame, the system aligns the image frame with the 3D model. After aligning the image frames with the 3D model, when the system presents a portion of the 3D model in an interface, it also presents an image frame that corresponds to the portion of the 3D model.

Abstract: An extended reality environment includes one or more virtual objects. A virtual object can be selected for manipulation by orienting a pointing device towards the virtual object and performing a selection input. In some embodiments, if the selection input is a first type of selection gesture received at the pointing device, the virtual object is selected for movement operations. In some embodiments, if the selection input is a second type of selection gesture received at the pointing device, the virtual object is selected for rotation operations. While the virtual object is selected for movement or rotation operations, the virtual object moves or rotates in accordance with the movement or rotation of the user's hand, respectively. In some embodiments, the virtual object has a first visual characteristic when the pointing device is pointing towards the virtual object and has a second visual characteristic when the virtual object is selected.

Abstract: A computer system concurrently displays a view of a physical environment; and a computer-generated user interface element overlaid on the view of the physical environment. An appearance of the computer-generated user interface element is based on an appearance of the view of the physical environment on which the computer-generated user interface element is overlaid. In response to an appearance of the physical environment changing, the appearance of the computer-generated user interface element is updated at a first time based on a graphical composition of the appearance of one or more portions of the physical environment at different times prior to the first time, including: an appearance of a first portion of the physical environment at a second time that is before the first time; and an appearance of a second portion of the physical environment at a third time that is before the second time.

Abstract: Embodiments disclosed herein mitigate technological barriers in preparing, sourcing, and exporting 3D assets with practical applications for journalism. According to one embodiment, a computer-implemented method for generating a three-dimensional map is provided. The method includes displaying a two-dimensional region of a map on a graphical user interface (GUI), wherein the map is displayed along longitudinal axis and a latitudinal axis. The method includes obtaining, through the GUI, a first input from a user device, the first input comprising an indication of a selected sub-region of the two-dimensional region of the map, wherein the selected sub-region comprises a plurality of pixels having longitudinal coordinates bounded by a first longitude coordinate and a second longitude coordinate along the longitudinal axis and latitudinal coordinates bounded by a first latitude coordinate and a second latitude coordinate along the latitudinal axis.

Abstract: An example computing system is configured to (i) generate a cross-sectional view of a three-dimensional drawing file; (ii) receive a first user input indicating a selection of a first mesh, wherein the selection comprises a selection point that establishes a first end point; (iii) generate a first representation indicating an alignment of the first end point with at least one corresponding geometric feature of the first mesh and a second representation indicating a set of one or more directions; (iv) receive a second user input indicating a given direction; (v) based on receiving the second user input, generate a dynamic representation of the dimensioning information along the given direction; (vi) receive a third user input indicating that the second user input is complete; (vii) based on receiving the third user input, add the dimensioning information to the cross-sectional view between the first end point and the second end point.

Abstract: An example computing system is configured to (i) receive a request to generate a cross-sectional view of a three-dimensional drawing file, where the cross-sectional view is based on a location of a cross-section line within the three-dimensional drawing file and includes an intersection of two meshes within the three-dimensional drawing file; (ii) generate the cross-sectional view of the three-dimensional drawing file; (iii) add, to the generated cross-sectional view, dimensioning information involving at least one of the two meshes; (iv) generate one or more controls for adjusting a location of the cross-section line within the three-dimensional drawing file; and (v) based on an input indicating a selection of the one or more controls, adjust the location of the cross-section line within the three-dimensional drawing file, update the cross-sectional view based on the adjusted location of the cross-section line, and update the dimensioning information to correspond to the updated cross-sectional view.

Abstract: Apparatuses, systems, and techniques for a compiled shader program caches in a cloud computing environment.

Abstract: Systems and methods for generating an image of an automobile can include generating an artificial surface for a 3D model of the automobile, blending the artificial surface with a real world surface, and generating the image of the automobile using a blended surface. The image can have a number of different blended surfaces (e.g., a cleaner floor, shadows for the automobile, reflections for the automobile). The images can be used to create a 360 degree display of the automobile where the blended surface is displayed.

Abstract: The medical image processing device includes a processor, in which the processor acquires a medical image obtained by imaging a subject with an endoscope, identifies a tumor region and a non-tumor region from the medical image, generates a demarcation line that is a boundary between the tumor region and the non-tumor region, generates a virtual incision line at a position separated from the demarcation line by a designated distance, and performs control for superimposing the demarcation line and the virtual incision line on the medical image to be displayed.

Abstract: An augmented reality (“AR”) device applies smooth correction methods to correct the location of the virtual objects presented to a user. The AR device may apply an angular threshold to determine whether a virtual object can be moved from an original location to a target location. An angular threshold is a maximum angle by which a line from the AR device to the virtual object can change within a timestep. Similarly, the AR device may apply a motion threshold, which is a maximum on the distance that a virtual object's location can be corrected based on the motion of the virtual object. Furthermore, the AR device may apply a pixel threshold to the correction of the virtual object's location. A pixel threshold is a maximum on the distance that a pixel projection of the virtual object can change based on the virtual object's change in location.

Abstract: A method is performed at an electronic device with one or more processors, a non-transitory memory, one or more environmental sensors, and a display. The method includes determining a first positional value associated with a physical agent based on environmental data from the one or more environmental sensors. The method includes determining that a portion of computer-generated content satisfies an occlusion criterion with respect to a corresponding portion of the physical agent, based on the first positional value. The method includes, in response to determining that the occlusion criterion is satisfied and determining that the physical agent satisfies a movement criterion or a pose criterion, generating a mesh associated with the physical agent based on the first positional value, and displaying the mesh on the display.

Abstract: Imaging devices, systems, and methods for analyzing an image taken by an imaging system are described herein. An example method includes: receiving a bank image from the imaging device pursuant to one of a plurality of banks of imaging parameters; rendering the bank image in an image display; receiving an indication of a region of interest (ROI) in the image display associated with a user-selected tool; determining whether the user-selected tool is set to use the bank image, an output of a parent tool, or an output of the user-selected tool; and generating, in the image display, at least a portion of a display image in the ROI representative of: (i) the bank image, (ii) the output of the parent tool, or (iii) the output of the user-selected tool; wherein the portion of the display image is associated with the user-selected region.

Abstract: A computer system displays an alert at a first position relative to the three-dimensional environment, the alert at least partially overlapping a first object in a first view. The first position has a respective spatial relationship to the user. The computer system detects movement of the user from a first viewpoint to a second viewpoint. At the second viewpoint, the computer system, in accordance with a determination that the alert is a first type of alert, displays the alert at a second position in the three-dimensional environment, the second position having the respective spatial relationship to the user and in accordance with a determination that the alert is a second type of alert, displays the three-dimensional environment from the second viewpoint without displaying the alert with the respective spatial relationship to the user.

Abstract: In one embodiment, a method of intermingling stereoscopic and conforming virtual content to a bounded surface is performed at a device that includes one or more processors, non-transitory memory, and one or more displays. The method includes displaying a bounded surface within a native user computer-generated reality (CGR) environment, wherein the bounded surface is displayed based on a first set of world coordinates characterizing the native user CGR environment. The method further includes displaying a first stereoscopic virtual object within a perimeter of a first side of the bounded surface, wherein the first stereoscopic virtual object is displayed in accordance with a second set of world coordinates that is different from the first set of world coordinates characterizing the native user CGR environment.

Abstract: In some implementations, a method of generating a third person view of a computer-generated reality (CGR) environment is performed at a device including non-transitory memory and one or more processors coupled with the non-transitory memory. The method includes: obtaining a first viewing vector associated with a first user within a CGR environment; determining a first viewing frustum for the first user within the CGR environment based on the first viewing vector associated with the first user and one or more depth attributes; generating a representation of the first viewing frustum; and displaying, via the display device, a third person view of the CGR environment including an avatar of the first user and the representation of the first viewing frustum adjacent to the avatar of the first user.

Abstract: An augmented-reality device is used with a surveying system to guide a base station, such as a total station, during relock after loss of line of sight with a surveying instrument, such as a surveying pole with a reflective prism. A position of the surveying instrument in relation to the augmented-reality device is calculated while an object in the environment blocks line of sight from the base station to the surveying instrument. A position of the surveying instrument in relation to the environment is calculated based on the position of the surveying instrument in relation to the augmented-reality device. The position of the surveying instrument in relation to the environment is used by the base station to point toward to the surveying instrument.

Abstract: A method for placing content in an augmented reality system. A notification is received regarding availability of new content to display in the augmented reality system. A confirmation is received that indicates acceptance of the new content. Three dimensional information that describes the physical environment is provided, to an external computing device, to enable the external computing device to be used for selecting an assigned location in the physical environment for the new content. Location information is received, from the external computing device, that indicates the assigned location. A display location on a display system of the augmented reality system at which to display the new content so that the new content appears to the user to be displayed as an overlay at the assigned location in the physical environment is determined, based on the location information. The new content is displayed on the display system at the display location.

Abstract: Techniques for performing shader operations are provided. The techniques include, performing pixel shading at a shading rate defined by pixel shader variable rate shading (“VRS”) data, and updating the pixel VRS data that indicates one or more shading rates for one or more tiles based on whether the tiles of the one or more tiles include triangle edges or do not include triangle edges, to generate updated VRS data.

Abstract: An image processing system configured to obtain a mesh representation of a scene, wherein the mesh representation comprises a plurality of polygons defined by respective vertices associated with an in-plane position, the in-plane position being in a plane comprising a first dimension and a second dimension, and the vertices having an associated vertex depth value in a third dimension different from the first dimension and the second dimension. The image processing system comprises an in-plane position estimation network configured to process image data representative of an image of the scene to estimate the in-plane positions associated with respective vertices of the mesh representation. The image processing system further comprises a depth estimation engine configured to process the in-plane positions and the image data to estimate the associated vertex depth values for the respective vertices of the mesh representation.