Abstract: Disclosed herein are related to a system and a method for controlling a virtual reality based on a physical object. In one aspect, a shape of a hand of a user corresponding to a surface or a structure of a physical object is detected. In one aspect, according to the detected shape of the hand, an interactive feature for the surface or the structure of the physical object is generated in a virtual reality or augmented reality application. In one aspect, a user interaction with the interactive feature is detected. In one aspect, an action of the virtual reality or augmented reality application is initiated, in response to detecting the user interaction with the interactive feature.

Abstract: In some examples, a device includes a multilayer structure, a first electrode, and a second electrode, where the multilayer structure is located at least in part between the first electrode and the second electrode, and the multilayer structure includes a nanovoided polymer layer, and a solid layer. The solid layer may include a non-nanovoided layer. The nanovoided polymer layer may be an electroactive layer. The device may further include a control circuit configured to apply an electrical potential between the first electrode and the second electrode, which may induce a mechanical deformation of the multilayer.

Abstract: In one embodiment, a computing system may access a first mask associated with a first color component and a first pixel in an image, and a second mask associated with a second color component and a second pixel in the image. The system may access first component values of the first color component in a first pixel region containing the first pixel, and second component values of the second color component in a second pixel region containing the second pixel. The system may modify the first component values using the first mask, and the second component values using the second mask. The system may cause the modified first and second component values to be displayed by light-emitting elements of the first and second color components. The first and second masks may be generated based on relative positions of the first light-emitting elements and the second light-emitting elements.

Abstract: Aspects of the present disclosure are directed to an artificial reality system orchestrating interactions between virtual object “augments.” The orchestration can include linking, which can be forming two or more augments into a combination, embedding an augment within an existing combination, or triggering an action mapped to the linking of those augments. Another type of orchestration can include extracting, which can refer to taking an augment out of an existing combination, either by removing it from the combination or copying the augment to leave a version in the combination and having another version outside the combination.

Abstract: Systems and methods for rendering VR content at a VR rendering device include identifying, by a content facilitator, that that first content presented via a first device to a user includes metadata indicating availability of second content. The second content includes at least one of a three-dimensional (3D) video, 360° image, or virtual reality (VR) content. The content facilitator provides an interface element to the user at the first device responsive to the metadata indicating availability of the second content. The content facilitator receives a user selection of the interface element. The content facilitator provides access to the second content via a VR rendering device for the user responsive to the user selection. The VR rendering device is configured to render the second content.

Abstract: Methods and systems for integrating functionality of XR devices and non-XR devices are disclosed herein. An XR device can identify and obtain one or more content items being displayed on non-XR devices and create virtual objects associated with the content item in an artificial reality environment of the XR device. A user of the XR device can interact with the virtual objects, such as by moving the virtual objects to new locations within the artificial reality environment or interact with the virtual objects to view the content items. The user of the XR device can also provide one or more gestures or commands to send content items or identifiers of content items associated with the virtual objects to non-XR devices for display at the non-XR devices.

Abstract: Disclosed herein are related to a system and a method of remotely rendering artificial reality. In one approach, a console determines an expected transmission end time of a first session for wireless transmission of image data indicating an image of artificial reality to a head wearable display. In one approach, the console determines an expected process end time of generating additional data associated with the image of the artificial reality. In one approach, the console compares the expected transmission end time and the expected process end time. The console may transmit, to the head wearable display in the first session, the additional data together with the image data, in response to the expected process end time being less than the expected transmission end time.

Abstract: A method may include displaying, to a user, a first color in a first area and a second color in a second area, where (1) the second color has a longer wavelength than the first color and (2) the first and second color have an expected longitudinal chromatic aberration for a human eye. The method may also include receiving, from the user, an indication of whether the user perceives (1) the first area as being clearer than the second area or (2) the second area as being clearer than the first area. The method may further include determining, based on the indication of the user and the expected longitudinal chromatic aberration, information about a refractive error of the user's vision. Various other methods, systems, and devices are also disclosed.

Abstract: The disclosed head-mounted display assemblies may include a first eyecup and a second eyecup that are configured for respectively positioning a first lens and a second lens in front of intended locations of a user's eyes when the head-mounted display assembly is donned. The first eyecup and the second eyecup may be movable relative to each other to adjust for an interpupillary distance of the user's eyes. A single near-eye display screen may be configured for displaying an image to the user through the first and second eyecups. An enclosure over the single near-eye display screen may include a first transparent component positioned between the first lens and the single near-eye display screen and a second transparent component positioned between the second lens and the single near-eye display screen. Various other methods, devices, systems, and assemblies are also disclosed.

Abstract: In one embodiment, a method includes, by one or more computing devices, determining, based on first tracking data, a first viewpoint of the user and a first hand pose of the user. The method may include generating a virtual object in a virtual environment based on the first hand pose and a predetermined spatial relationship between the virtual object and the hand of the user. The method may include rendering a first image of the virtual object as viewed from the first viewpoint. The method may include determining, based on second tracking data, a second viewpoint of the user and a second hand pose. The method may include adjusting the first image of the virtual object based on changes from the first hand pose to the second hand pose. The method may include rendering and displaying a second image from the adjusted first image viewed from the second viewpoint.

Abstract: In one embodiment, a method includes accessing one or more surfaces of an artificial reality scene. The one or more surfaces are generated based on one or more images of the artificial reality scene rendered at a first rate and based on a first viewing position. The method includes generating subframes at a second rate higher than the first rate. Each of the subframes is generated by determining a second viewing position, determining a visibility of the one or more surfaces of the artificial reality scene from the second viewing position, generating, based on the determined visibility of the one or more surfaces, color values of the subframe corresponding to output positions of a display, and providing the color values of the subframe for display.

Abstract: Systems and methods for generating composite sets of data based on sensor data from different sensors are disclosed. Exemplary implementations may capture a color image including chromatic information; capture a depth image; generate inertial signals conveying values that are used to determine motion parameters; determine the motion parameters based on the inertial signals; generate a re-projected depth image as if the depth image had been captured at the same time as the color image, based on the interpolation of motion parameters; and generate a composite set of data based on different kinds of sensor data by combining information from the color image, the re-projected depth image, and one or more motion parameters.

Abstract: In one embodiment, a method includes receiving, from a client system associated with a first user, a first audio input. The method includes generating multiple transcriptions corresponding to the first audio input based on multiple automatic speech recognition (ASR) engines. Each ASR engine is associated with a respective domain out of multiple domains. The method includes determining, for each transcription, a combination of one or more intents and one or more slots to be associated with the transcription. The method includes selecting, by a meta-speech engine, one or more combinations of intents and slots from the multiple combinations to be associated with the first user input. The method includes generating a response to the first audio input based on the selected combinations and sending, to the client system, instructions for presenting the response to the first audio input.

Abstract: Provisioning multiple artificial reality devices can be accomplished through network communications between a device distributor, an administration system, a provisioning device, and the artificial reality devices. The device distributor can provide, to the administration system, a list of artificial reality devices corresponding to a particular organization. A provisioning device can sign into an account on the administration system and receive the list. The provisioning device can then scan for wireless connectivity with the listed artificial reality devices and can make a wireless connection with any artificial reality devices that respond and that are on the list. The provisioning device can then send these devices credentials for a local WiFi network. The artificial reality devices can automatically, upon receipt of the credentials, connect to the administration system to obtain the software update and/or the configuration data.

Abstract: Methods and systems for providing illumination for depth sensing are provided. In one example, an apparatus comprises an illuminator, an optical sensor, and a controller. The illuminator comprises a photonic integrated circuit (PIC) and a diffractive optical element (DOE). The PIC including a laser source and at least one waveguide including optical turning features. The at least one waveguide propagates light transmitted by the laser source along a first axis parallel to a top surface of the PIC. The optical turning features diverts the light to form collimated light beams to exit the at least one waveguide and the top surface along a second axis. The DOE can diffract and project the collimated light beams. The optical sensor can detect the collimated light beams reflected off an object. The controller can determine a depth of the object with respect to the apparatus based on the detection.

Abstract: A nanovoided polymer-based material may include a bulk polymer material defining a plurality of nanovoids and an interfacial film disposed at an interface between each of the plurality of nanovoids and the bulk polymer material. The interfacial film may include one or more layers of material. A method of forming a nanovoided polymer-based material may include (1) forming a bulk polymer material defining a plurality of nanovoids and (2) forming an interfacial film at an interface between each of the plurality of nanovoids and the bulk polymer material. Various other methods, systems, and materials are also disclosed.

Abstract: Disclosed are systems, methods, and non-transitory computer-readable media for reducing latency in augmented reality displays. A display controller receives, from a GPU, a stream of image pixels of a frame of virtual content to be presented on a display of a display device. The stream of image pixels is received via a high-speed bulk interface that transfers data at least as fast as can be consumed by the display. As the stream of image pixel is received, the display controller converts each respective image pixel from a data format used to transmit the stream of image pixels via the high-speed bulk interface to a data format that is compatible for display by the display. Each converted image pixel is stored in a pixel cell of the display, after which the frame is presented on the display.

Abstract: The present embodiments relate to automated memory creation and retrieval from moment content items. In some implementations, the automated memory creation and retrieval system can obtain moment content items from user-designated sources with a single user perspective or multiple user perspectives. The moment content items can be assigned tags and arranged in chronological order. The arranged moment content items can be clustered into memory content items based on clustering conditions. Once memory content items are created, they can be arranged into a memory hierarchy made up of connected nodes. In some implementations, the memory content items are also linked together based on similarity in various dimensions in a memory graph. The automated memory creation and retrieval system can receive search criteria for memories from a user interface and provide the user with memories from matched nodes in the memory hierarchy or linked memories in the memory graph.

Abstract: A disclosed system may include (1) a wearable dimensioned to be donned on a body of a user of a computing device (2) a set of sensors coupled to the wearable, wherein the set of sensors detect one or more neuromuscular signals via the body of the user, and (3) at least one processing device communicatively coupled to the set of sensors, wherein the processing device (A) determines, based at least in part on the neuromuscular signals detected by the set of sensors, an intention of the user in connection with an application running on the computing device and, in response to determining the intention of the user, (B) modifying a feature of the application running on the computing device to account for the intention of the user. Various other devices, systems, and methods are also disclosed.