Abstract: In one embodiment, a computing system may access a video including a first frame and a second frame. The computing system may determine first sampling locations for the first frame and determine second sampling locations for the second frame by transforming the first sampling locations to the second frame according to an optical flow between the first frame and the second frame. The computing system may select a subset of the second sampling locations based on a comparison between pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the second sampling locations. The computing system may define one or more rejection areas in the second frame based on the subset of the second sampling locations to determine third sampling locations in areas outside of the rejection areas. The computing system may generate a sample of the video.

Abstract: In one embodiment, a system may access an input image of an object captured by cameras, and the input image depicts appearance information associated with an object. The system may generate a first mesh of the object based on features identified from the input image of the object. The system may generate, by processing the first mesh using a machine-learning model, a position map that defines a contour of the object. Each pixel in the position map corresponds to a three-dimensional coordinate. The system may further generate a second mesh based on the position map, wherein the second mesh has a higher resolution than the first mesh. The system may render an output image of the object based on the second mesh. The system disclosed in the present application can render a dense mesh which has a higher resolution to provide details which cannot be compensated by texture information.

Abstract: In one embodiment, a computing device may determine a virtual content to be displayed with a scene of a real-world environment. The device may generate an image depicting the virtual content. Using one or more sensors, the device may detect characteristics of the scene of the real-world environment. Based on the image and the characteristics of the scene, the device may determine that a visual enhancement is to be applied to the virtual content depicted in the image to enhance a contrast between the depicted virtual content and the scene. The device may generate a visually-enhanced image depicting the virtual content by applying the visual enhancement to the virtual content depicted in the image. The device may display the visually-enhanced image of the virtual content on a display of the computing device, wherein the scene of the real-world environment is visible through the display.

Abstract: An audio system includes a sound output device, a microphone, and processing circuitry. The microphone is configured to capture environmental audio. The processing circuitry is configured to analyze the environmental audio to identify one or more properties of environmental audio conditions. The processing circuitry is configured to adjust one or more sound presentation parameters based on the one or more properties of the environmental audio conditions to account for the environmental audio conditions. The processing circuitry is configured to operate the sound output device to output audio according to the one or more sound presentation parameters.

Abstract: In general, this disclosure describes techniques for using a random number generator to affect the lengths of clock cycles in a clock waveform that drives the timing of operations performed by processing circuitry. In one example, the processing circuitry includes a central processing unit and a clock generator. The clock generator is configured, upon receiving an indication of a boot command for the processing circuitry, generate a random number using a true random number generator and generate, based at least in part on the random number, an output clock waveform indicating at least a length of a clock cycle for the central processing unit. The central processing unit is configured to execute a boot sequence for at least the processing circuitry using the output clock waveform.

Abstract: In one embodiment, a method includes projecting a triangle primitive of an object defined in three-dimensional space onto a two-dimensional space, accessing a distortion map generated based on distortion characteristics of a display system, distorting a grid representation of a screen of the display system using the distortion map, determining a visibility of the triangle primitive relative to pixels of the screen by comparing the projected triangle primitive and the distorted grid representation of the screen, rendering an image based on the determined visibility of the triangle primitive, the rendered image being configured to be displayed by the screen of the display system having the distortion characteristics.

Abstract: An artificial reality system is described that renders, presents, and controls user interface elements within an artificial reality environment, and performs actions in response to one or more detected gestures of the user. In one example, an artificial reality system comprises a head-mounted display configured to output artificial reality content; a stylus; a stylus action detector configured to detect movement of the stylus, detect a stylus selection action, and after detecting the stylus selection action, detect further movement of the stylus; a UI engine configured to generate stylus movement content in response to detecting movement of the stylus, and generate a UI input element in response to detecting the stylus selection action; and a rendering engine configured to render the stylus movement content and the UI input element as overlays to the artificial reality content, and update the stylus movement content based on the further movement of the stylus.

Abstract: An optical device for a head-mounted display device includes a first partial reflector and a second partial reflector positioned relative to the first partial reflector so that the second partial reflector receives first light transmitted through the first partial reflector and reflects at least a portion of the first light toward the first partial reflector as second light. At least a portion of the second light is reflected by the first partial reflector as third light, and at least a portion of the third light is transmitted through the second partial reflector. At least one of the first partial reflector or the second partial reflector includes a reflective holographic element.

Abstract: The disclosed technology can perform application controls in response to recognizing particular gestures. The disclosed technology can provide a launcher with virtual objects displayed in categories (e.g., history, pinned favorites, people, and a search area). The disclosed technology can perform a clone and configure input pattern, which clones a source virtual object into one or more cloned virtual objects with alternate configuration properties. The disclosed technology can perform a page or peel input pattern, which allows users to page between grids of virtual objects and facilitates peeling items out of the grid. The disclosed technology can perform a clutter and clear input pattern, which can expand multiple elements into individual views, while clearing other virtual objects.

Abstract: In one embodiment, a method includes displaying, for one or more displays of a virtual VR device, a first output image comprising a passthrough view of a real-world environment. The method includes identifying, using one or more images captured by one or more cameras of the VR display device, a real-world object in the real-world environment. The method includes receiving a user input indicating a first dimension corresponding to the real-world object. The method includes automatically determining, based on the first dimension, a second and third dimension corresponding to the real-world object. The method includes rendering, for the one or more displays of the VR display device, a second output image of a VR environment. The VR environment includes a MR object that corresponds to the real-world object. The MR object is defined by the determined first, second, and third dimensions.

Abstract: A wearable device equipped with sensors or transducers capable of detecting a vibration propagating through two different mediums. In some embodiments, the two mediums are air and a surface contacted by a user. In such an embodiment, the sensors or transducers include a microphone and an accelerometer. In some embodiments, the device may perform some or all of the signal processing and/or logic used to determine that a contact occurred and the location of the contact using a processor programmed with a set of computer-executable instructions. In some embodiments, the device may transmit data or signals to a processor located in a different device, such as a mobile phone or artificial reality headset, for signal processing and/or the execution of logic.

Abstract: An example device may include a light source, an optical element, and an encapsulant layer. A light beam generated by the light source may be received by the optical element, and redirected into the encapsulant layer. The optical element may include a high-index material, for example, with a refractive index of at least approximately 1.5 at the wavelength of the light beam. The light source may be a semiconductor light source, such as a light emitting diode or a laser. The optical element may be embedded in the encapsulant layer, and the optical element may have a curved exit surface. Refraction at the exit surface of the optical element may redirect the light beam towards a target.

Abstract: In general, this disclosure describes a suspend mode feature for artificial reality systems and, more specifically, system configurations and techniques for generating a suspend mode environment during execution of an artificial reality application, and presenting and controlling user interface (UI) elements within the suspend mode environment. The suspend mode feature enables a user interacting with one or more other users (or one or more avatars corresponding to the one or more other users) within an artificial reality (AR) environment to suspend the AR environment from the user's perspective. While in the suspend mode environment, the user is able to view and select the other users from the AR environment. For example, the user may select another user to friend, message, mute, block, or report that user from the suspend mode environment.

Abstract: In one embodiment, one or more computing systems executes a computer program defining (1) a bound value that references a bound variable and (2) a terminating bound value that references computation instructions for deriving a runtime value from at least the bound value of the bound variable. A computation orchestration layer of a programming framework initializes, according to the computer program, a subscription tracker that tracks subscription relationships between the bound variable and the terminating bound value. The computation orchestration layer uses, in response to receiving an indication of an update to the bound value of the bound variable, the subscription tracker to determine that the terminating bound value subscribes to updates to the bound value of the bound variable. The computation orchestration layer sends an update stream associated with the update to a callback function associated with the terminating bound value.

Abstract: A method is described for utilizing NIL materials with switchable mechanical properties. The method comprises applying an imprint mask to a nano-imprint lithography (NIL) material layer. The NIL material layer is comprised of a NIL material with a modulus level below a flexibility threshold. The NIL material layer has an internal property, that when changed, causes a change in the modulus level of the NIL material. The method further comprises detaching the imprinted NIL material layer from the imprint mask, with the low modulus level of the NIL material causing a shape of the imprinted NIL material layer to remain unchanged after detachment. A modulus level of the NIL material is increased by changing an internal property of the NIL material, with the modulus level increased beyond a strength threshold to create a first imprint layer that has a structure that remains unaffected by a subsequent process.

Abstract: This disclosure describes various examples of a system which uses a multi-bank, multi-port shared memory system that may be implemented as part of a system on a chip. The shared memory system may have particular applicability in the context of an artificial reality system, and may be designed to have distributed or varied latency for one or more memory banks and/or one or more components or subsystems within the system on a chip. The described shared memory system may be logically a single entity, but physically may have multiple memory banks, each accessible by any of a number of components or subsystems. In some examples, the memory system may enable concurrent, common, and/or shared access to memory without requiring, in some situations, full locking or arbitration.

Abstract: In general, the disclosure describes techniques for wireless communications between multiple devices of an artificial reality system using virtual channels. In one example, a first computing device of a multi-device system, comprising: one or more first processors coupled to one or more memory devices; and a supervisory processor configured to establish a plurality of virtual channels over a physical wireless channel between the first computing device and a second computing device, wherein each of the virtual channels is associated with a different pair of processors comprising a source processor selected from one of the first processors or second processors of the second computing device and a destination processor selected from one of the first processors or the second processors, wherein, for each virtual channel and associated pair of processors, the source processor is configured to communicate application data with the destination processor using the virtual channel.

Abstract: In some examples, a method includes forming a material layer on a substrate, partially polymerizing a component of the material layer, to form fluid-filled droplets within a partially polymerized matrix, deforming the material layer to form anisotropic fluid-filled droplets, and further polymerizing the partially polymerized matrix to form an anisotropic voided polymer, including anisotropic voids in a polymer matrix. The anisotropic voids may include anisotropic nanovoids. Example methods may further include depositing electrodes on the anisotropic voided polymer so that at least a portion of the anisotropic voided polymer is located between the electrodes. Examples may include forming electroactive elements including an anisotropic nanovoided polymer, and devices (such as sensors and/or actuators) including electroactive elements.

Abstract: A method for using a virtual camera location to display image data to one device from another device associated with a plurality of cameras. The method includes receiving image data from a plurality of cameras associated with a second user device associated with a second user. The method further includes establishing a virtual camera location different from positions of the plurality of cameras. The method further includes providing an image using the image data on a first user display of a first user device based at least on a viewpoint of the virtual camera location.