Abstract: A wearable display system includes a mixed reality display for presenting a virtual image to a user, an outward-facing imaging system configured to image an environment of the user, and a hardware processor operably coupled to the mixed reality display and to the imaging system. The hardware processor is programmed to generate a virtual remote associated with a parent device, render the virtual remote and the virtual control element on the mixed reality display, determine when the user of the wearable system interacts with the virtual control element of the virtual remote, and perform certain functions in response to user interaction with a virtual control element of the virtual remote. These functions may include generation the virtual control element to move on the mixed reality display; and when movement of the virtual control element surpasses a threshold condition, generate a focus indicator for the virtual control element.

Abstract: A wearable display system can automatically recognize a physical remote or a device that the remote serves using computer vision techniques. The wearable system can generate a virtual remote with a virtual control panel viewable and interactable by user of the wearable system. The virtual remote can emulate the functionality of the physical remote. The user can select a virtual remote for interaction, for example, by looking or pointing at the parent device or its remote control, or by selecting from a menu of known devices. The virtual remote may include a virtual button, which is associated with a volume in the physical space. The wearable system can detect that a virtual button is actuated by determining whether a portion of the user's body (e.g., the user's finger) has penetrated the volume associated with the virtual button.

Abstract: 2D and 3D data of a scene are linked by associating points in the 3D data with corresponding points in multiple different 2D images within the 2D data. Labels assigned to points in either data can be propagated to the other data. Labels propagated to a point in the 3D data are aggregated, and the labels ranked highest are kept and propagated back to the 2D images. 3D data including labels produced in this manner allow partially obscured objects in certain views to be more accurately identified. Thus, an object can be manipulated in all 2D views of the 2D data in which the object is at least partially visible, in order to digitally remove, alter, or replace it.

Abstract: A wearable display system can automatically recognize a physical remote or a device that the remote serves using computer vision techniques. The wearable system can generate a virtual remote with a virtual control panel viewable and interactable by user of the wearable system. The virtual remote can emulate the functionality of the physical remote. The user can select a virtual remote for interaction, for example, by looking or pointing at the parent device or its remote control, or by selecting from a menu of known devices. The virtual remote may include a virtual button, which is associated with a volume in the physical space. The wearable system can detect that a virtual button is actuated by determining whether a portion of the user's body (e.g., the user's finger) has penetrated the volume associated with the virtual button.

Abstract: A wearable display system can automatically recognize a physical remote or a device that the remote serves using computer vision techniques. The wearable system can generate a virtual remote with a virtual control panel viewable and interactable by user of the wearable system. The virtual remote can emulate the functionality of the physical remote. The user can select a virtual remote for interaction, for example, by looking or pointing at the parent device or its remote control, or by selecting from a menu of known devices. The virtual remote may include a virtual button, which is associated with a volume in the physical space. The wearable system can detect that a virtual button is actuated by determining whether a portion of the user's body (e.g., the user's finger) has penetrated the volume associated with the virtual button.

Abstract: Touchscreen-enabled devices are provided to display images of either real or virtual environments, the devices having at least one user interface including a graphical overlay laid over the images for navigating the environments. The device can interpret gestures made by a single finger of the user on the touchscreen as commands to achieve intended maneuvers such as translations of the point of view and rotations of the point of view within the displayed environment. This allows for one-finger navigation of the environment. The elevational angle of the viewpoint may be tilted above or below horizontal as the viewpoint approaches topographical features within the displayed environment, and may return to horizontal after transitioning to a level surface.

Abstract: Touchscreen-enabled devices are provided to display images of either real or virtual environments, the devices having at least one user interface including a graphical overlay laid over the images for navigating the environments. The device can interpret gestures made by a single finger of the user on the touchscreen as commands to achieve intended maneuvers such as translations of the point of view and rotations of the point of view within the displayed environment. This allows for one-finger navigation of the environment. The elevational angle of the viewpoint may be tilted above or below horizontal as the viewpoint approaches topographical features within the displayed environment, and may return to horizontal after transitioning to a level surface.

Abstract: A wearable display system can automatically recognize a physical remote or a device that the remote serves using computer vision techniques. The wearable system can generate a virtual remote with a virtual control panel viewable and interactable by user of the wearable system. The virtual remote can emulate the functionality of the physical remote. The user can select a virtual remote for interaction, for example, by looking or pointing at the parent device or its remote control, or by selecting from a menu of known devices. The virtual remote may include a virtual button, which is associated with a volume in the physical space. The wearable system can detect that a virtual button is actuated by determining whether a portion of the user's body (e.g., the user's finger) has penetrated the volume associated with the virtual button.

Abstract: Methods for generating a virtual world are described. The virtual world may comprise a three-dimensional gameworld associated with a video game. The virtual world may be represented by a plurality of voxels arranged in a three-dimensional grid. Each voxel of the plurality of voxels may be associated with various attributes such as one or more color values, an opacity value, a location within the virtual world, a fill value, and a cubify value. In some embodiments, the virtual world may be generated or edited using a computer graphics editing tool that assigns one or more cubify values to one or more voxels using a voxel selection tool, such as a cubify brush. A voxel's cubify value may be used to determine how the voxel is rendered by a rendering engine, for example, whether the voxel is rendered as a rectilinear cube or as a smooth isosurface.

Abstract: Methods for generating a virtual world are described. The virtual world may comprise a three-dimensional gameworld associated with a video game. The virtual world may be represented by a plurality of voxels arranged in a three-dimensional grid. Each voxel of the plurality of voxels may be associated with various attributes such as one or more color values, an opacity value, a location within the virtual world, a fill value, and a cubify value. In some embodiments, the virtual world may be generated or edited using a computer graphics editing tool that assigns one or more cubify values to one or more voxels using a voxel selection tool, such as a cubify brush. A voxel's cubify value may be used to determine how the voxel is rendered by a rendering engine, for example, whether the voxel is rendered as a rectilinear cube or as a smooth isosurface.