Abstract: Various embodiments set forth a foveated display system and components thereof. The foveated display system includes a peripheral display module disposed in series with a foveal display module. The peripheral display module is configured to generate low-resolution, large field of view imagery for a user's peripheral vision. The foveal display module is configured to perform foveated rendering in which high-resolution imagery is focused towards a foveal region of the user's eye gaze. The peripheral display module may include a diffuser that is disposed within a pancake lens, which is a relatively compact design. The foveal display module may include a Pancharatnam-Berry Phase grating stack that increases the steering range of a beam-steering device such that a virtual image can be steered to cover an entire field of view visible to the user's eye.

Abstract: Systems on a chip (SoCs) include security logic configured to increase resistance to fault injection attacks (FIAs). The security logic includes a monitoring circuit and a cascaded series of substitution-boxes (S-Boxes) having a circuit delay that is designed to match (or most closely match) the computing device critical path length. The monitoring circuit monitors the number of iterations required for the cascaded series of S-Boxes to return to an initial value and generates an error signal if the monitored loop length is different from the expected loop length. In some examples, the error signal is received by a mitigation processor that executes one or more processes aimed at mitigating the attack.

Abstract: A virtual reality (VR) system is described that enables users to create, model, and visualize augmented reality experiences at physical, real-world locations, without needing to physically be at the location for which the augmented experience is being modeled. In one example, the VR system includes a head-mounted device (HMD), an augmented reality content development system, and a rendering engine. The HMD is configured to output VR content representing a physical environment for an augmented reality experience. The augmented reality content development system is configured to identify, based on an input, augmented reality content with which to augment the physical environment for the augmented reality experience. The rendering engine is configured to render, via the HMD, the VR content representing the physical environment, augmented with the augmented reality content.

Abstract: Various embodiments set forth high-resolution liquid crystal displays and components thereof. In some embodiments, light emitted by a high-resolution green color liquid crystal display is combined, via a combiner, with light emitted by at least one lower-resolution red and blue color liquid crystal display. The red and blue color display(s) may include a single display or two displays positioned on opposing sides of the combiner. The combiner may be a dichroic or polarization-based combiner. Combined light from the green color display and the red and blue color display(s) is passed through collimating optics, such as a pancake lens or a Fresnel lens, toward a viewer's eye.

Abstract: One embodiment of the present disclosure sets forth a method that includes receiving one or more two-dimensional images of one or more light patterns incident on an eye proximate to an eye region of a near-eye display device, and computing a gaze direction associated with the eye based on the one or more two-dimensional images.

Abstract: This disclosure describes efficient communication of surface texture data between system on a chip (SOC) integrated circuits. An example system includes a first integrated circuit, and at least one second integrated circuit communicatively coupled to the first integrated circuit by a communication interface. The first integrated circuit, upon determining that surface texture data of a frame to be rendered for display by the second SoC integrated circuit is to be updated, (a) transmits the surface texture data in one or more update packets to the second integrated circuit using the communication interface, and (b) transmits a command to the second integrated circuit indicating that the surface texture data of the frame has been updated using the communication interface.

Abstract: An artificial reality system is described that renders, presents, and controls user interface elements within an artificial reality environment, and performs actions in response to one or more detected gestures of the user. In one example, an artificial reality system comprises a head-mounted display configured to output artificial reality content; a stylus; a stylus action detector configured to detect movement of the stylus, detect a stylus selection action, and after detecting the stylus selection action, detect further movement of the stylus; a UI engine configured to generate stylus movement content in response to detecting movement of the stylus, and generate a UI input element in response to detecting the stylus selection action; and a rendering engine configured to render the stylus movement content and the UI input element as overlays to the artificial reality content, and update the stylus movement content based on the further movement of the stylus.

Abstract: An audio system includes a sound output device, a microphone, and processing circuitry. The microphone is configured to capture environmental audio. The processing circuitry is configured to analyze the environmental audio to identify one or more properties of environmental audio conditions. The processing circuitry is configured to adjust one or more sound presentation parameters based on the one or more properties of the environmental audio conditions to account for the environmental audio conditions. The processing circuitry is configured to operate the sound output device to output audio according to the one or more sound presentation parameters.

Abstract: In general, this disclosure describes a suspend mode feature for artificial reality systems and, more specifically, system configurations and techniques for generating a suspend mode environment during execution of an artificial reality application, and presenting and controlling user interface (UI) elements within the suspend mode environment. The suspend mode feature enables a user interacting with one or more other users (or one or more avatars corresponding to the one or more other users) within an artificial reality (AR) environment to suspend the AR environment from the user's perspective. While in the suspend mode environment, the user is able to view and select the other users from the AR environment. For example, the user may select another user to friend, message, mute, block, or report that user from the suspend mode environment.

Abstract: Disclosed herein are a system, a method and a device for switching between different communications protocols. A head wearable display can transmit, to a console during a first session interval, a first request to switch from a first link using a first communication protocol to a second link using a second communication protocol. The first request can include at least one characteristic of the first link. The head wearable display can receive, from the console, a first response to the first request, to perform a switch from the first link to the second link. The first response can indicate a wake up time and a second session interval to perform the switch. The head wearable display can switch from the first link to the second link in the second session interval and after the wake up time.

Abstract: In general, the disclosure describes techniques for wireless communications between multiple devices of an artificial reality system using virtual channels. In one example, a first computing device of a multi-device system, comprising: one or more first processors coupled to one or more memory devices; and a supervisory processor configured to establish a plurality of virtual channels over a physical wireless channel between the first computing device and a second computing device, wherein each of the virtual channels is associated with a different pair of processors comprising a source processor selected from one of the first processors or second processors of the second computing device and a destination processor selected from one of the first processors or the second processors, wherein, for each virtual channel and associated pair of processors, the source processor is configured to communicate application data with the destination processor using the virtual channel.

Abstract: Disclosed herein are related to a system and a method for controlling a virtual reality based on a physical object. In one aspect, a shape of a hand of a user corresponding to a surface or a structure of a physical object is detected. In one aspect, according to the detected shape of the hand, an interactive feature for the surface or the structure of the physical object is generated in a virtual reality or augmented reality application. In one aspect, a user interaction with the interactive feature is detected. In one aspect, an action of the virtual reality or augmented reality application is initiated, in response to detecting the user interaction with the interactive feature.

Abstract: Disclosed herein are related to a system and a method of remotely rendering artificial reality. In one approach, a console determines an expected transmission end time of a first session for wireless transmission of image data indicating an image of artificial reality to a head wearable display. In one approach, the console determines an expected process end time of generating additional data associated with the image of the artificial reality. In one approach, the console compares the expected transmission end time and the expected process end time. The console may transmit, to the head wearable display in the first session, the additional data together with the image data, in response to the expected process end time being less than the expected transmission end time.

Abstract: Systems and methods for generating composite sets of data based on sensor data from different sensors are disclosed. Exemplary implementations may capture a color image including chromatic information; capture a depth image; generate inertial signals conveying values that are used to determine motion parameters; determine the motion parameters based on the inertial signals; generate a re-projected depth image as if the depth image had been captured at the same time as the color image, based on the interpolation of motion parameters; and generate a composite set of data based on different kinds of sensor data by combining information from the color image, the re-projected depth image, and one or more motion parameters.

Abstract: Systems and methods for rendering VR content at a VR rendering device include identifying, by a content facilitator, that that first content presented via a first device to a user includes metadata indicating availability of second content. The second content includes at least one of a three-dimensional (3D) video, 360° image, or virtual reality (VR) content. The content facilitator provides an interface element to the user at the first device responsive to the metadata indicating availability of the second content. The content facilitator receives a user selection of the interface element. The content facilitator provides access to the second content via a VR rendering device for the user responsive to the user selection. The VR rendering device is configured to render the second content.

Abstract: Disclosed are systems, methods, and non-transitory computer-readable media for reducing latency in augmented reality displays. A display controller receives, from a GPU, a stream of image pixels of a frame of virtual content to be presented on a display of a display device. The stream of image pixels is received via a high-speed bulk interface that transfers data at least as fast as can be consumed by the display. As the stream of image pixel is received, the display controller converts each respective image pixel from a data format used to transmit the stream of image pixels via the high-speed bulk interface to a data format that is compatible for display by the display. Each converted image pixel is stored in a pixel cell of the display, after which the frame is presented on the display.

Abstract: An apparatus for creating and tuning haptic feedback is provided. The apparatus includes (A) a brace with first and second portions, (B) a vibrotactile actuator, coupled to the first portion of the brace, configured to apply haptic stimulations to a user, and (C) a linear actuator, coupled to the second portion of the brace, configured to (i) obtain a signal generated based on information about the user's body at a target location of the user and (ii) adjust a tension of the brace according to the signal. In some embodiments, the apparatus also includes one or more sensors configured to generate the information about the user's body at the target location. Furthermore, a controller may be configured to: (i) generate the signal based on the information about the user's body at the target location generated by the one or more sensors and (ii) provide the signal to the linear actuator.

Abstract: In general, this disclosure describes scalable, partitionable encryption engines. The partitionable encryption engines of this disclosure yield power savings, such as by controlling operation of partitioned sub-datapaths at reduced clock rates. An apparatus includes an interface configured to receive a block of encrypted data for decryption, and a decryption engine in communication with the interface. The decryption engine includes a plurality of decryption sub-datapaths, where each respective decryption sub-datapath has no data interdependency with any other decryption sub-datapath of the plurality of decryption sub-datapaths. The decryption engine is configured to selectively enable one or more decryption sub-datapaths of the plurality of decryption sub-datapaths to decrypt the block of encrypted data to form a decrypted block of data.

Abstract: Techniques for adjusting an optical element using an actuator assembly. The optical element is adjusted by translating the optical element along a linear axis of motion or by applying force upon a surface of the optical element. The optical element can be a liquid lens that is shaped by the application of force upon a surface of the lens. The actuator assembly includes a plurality of lead screws and a mechanical linkage that intercouples the lead screws and that is configured to simultaneously rotate the plurality of lead screws. The actuator assembly further includes a displacement element configured to act upon the optical element, through translational motion of the displacement element in response to rotation of the lead screws. Multiple optical elements can be adjusted simultaneously using respective displacement elements coupled to the lead screws.

Abstract: A device for providing a 1D line of an image is disclosed. The device includes a wavelength-tunable light source for providing image light having the angular distribution encoded in optical spectrum. The device further includes a thin slab waveguide having an out-coupler in form of a diffraction grating for out-coupling the image light at an angle dependent on wavelength. The image may be formed by scanning a collimated beam propagating in the slab waveguide when using tunable monochromatic light sources, or by forming the 1D singular distribution of brightness at a same time when using a tunable-spectrum light sources. The device may be used in a near-eye display for forming a 2D image in angular domain.