Abstract: Systems and methods for generating composite sets of data based on sensor data from different sensors are disclosed. Exemplary implementations may capture a color image including chromatic information; capture a depth image; generate inertial signals conveying values that are used to determine motion parameters; determine the motion parameters based on the inertial signals; generate a re-projected depth image as if the depth image had been captured at the same time as the color image, based on the interpolation of motion parameters; and generate a composite set of data based on different kinds of sensor data by combining information from the color image, the re-projected depth image, and one or more motion parameters.

Abstract: In one embodiment, a method includes, by one or more computing devices, determining, based on first tracking data, a first viewpoint of the user and a first hand pose of the user. The method may include generating a virtual object in a virtual environment based on the first hand pose and a predetermined spatial relationship between the virtual object and the hand of the user. The method may include rendering a first image of the virtual object as viewed from the first viewpoint. The method may include determining, based on second tracking data, a second viewpoint of the user and a second hand pose. The method may include adjusting the first image of the virtual object based on changes from the first hand pose to the second hand pose. The method may include rendering and displaying a second image from the adjusted first image viewed from the second viewpoint.

Abstract: In one embodiment, a method includes receiving, from a client system associated with a first user, a first audio input. The method includes generating multiple transcriptions corresponding to the first audio input based on multiple automatic speech recognition (ASR) engines. Each ASR engine is associated with a respective domain out of multiple domains. The method includes determining, for each transcription, a combination of one or more intents and one or more slots to be associated with the transcription. The method includes selecting, by a meta-speech engine, one or more combinations of intents and slots from the multiple combinations to be associated with the first user input. The method includes generating a response to the first audio input based on the selected combinations and sending, to the client system, instructions for presenting the response to the first audio input.

Abstract: The disclosed head-mounted display assemblies may include a first eyecup and a second eyecup that are configured for respectively positioning a first lens and a second lens in front of intended locations of a user's eyes when the head-mounted display assembly is donned. The first eyecup and the second eyecup may be movable relative to each other to adjust for an interpupillary distance of the user's eyes. A single near-eye display screen may be configured for displaying an image to the user through the first and second eyecups. An enclosure over the single near-eye display screen may include a first transparent component positioned between the first lens and the single near-eye display screen and a second transparent component positioned between the second lens and the single near-eye display screen. Various other methods, devices, systems, and assemblies are also disclosed.

Abstract: In one embodiment, a method includes accessing one or more surfaces of an artificial reality scene. The one or more surfaces are generated based on one or more images of the artificial reality scene rendered at a first rate and based on a first viewing position. The method includes generating subframes at a second rate higher than the first rate. Each of the subframes is generated by determining a second viewing position, determining a visibility of the one or more surfaces of the artificial reality scene from the second viewing position, generating, based on the determined visibility of the one or more surfaces, color values of the subframe corresponding to output positions of a display, and providing the color values of the subframe for display.

Abstract: Methods and systems for providing illumination for depth sensing are provided. In one example, an apparatus comprises an illuminator, an optical sensor, and a controller. The illuminator comprises a photonic integrated circuit (PIC) and a diffractive optical element (DOE). The PIC including a laser source and at least one waveguide including optical turning features. The at least one waveguide propagates light transmitted by the laser source along a first axis parallel to a top surface of the PIC. The optical turning features diverts the light to form collimated light beams to exit the at least one waveguide and the top surface along a second axis. The DOE can diffract and project the collimated light beams. The optical sensor can detect the collimated light beams reflected off an object. The controller can determine a depth of the object with respect to the apparatus based on the detection.

Abstract: A method includes projecting a single color illumination light having a first color on a liquid crystal on silicon display device, thereby obtaining a single color image light having the first color from the liquid crystal on silicon display device. The method also includes receiving image light having at least a second color that is different from the first color from a display panel that is different from a liquid crystal on silicon display device and combining the single color image light and the image light for projection toward an eye.

Abstract: Provisioning multiple artificial reality devices can be accomplished through network communications between a device distributor, an administration system, a provisioning device, and the artificial reality devices. The device distributor can provide, to the administration system, a list of artificial reality devices corresponding to a particular organization. A provisioning device can sign into an account on the administration system and receive the list. The provisioning device can then scan for wireless connectivity with the listed artificial reality devices and can make a wireless connection with any artificial reality devices that respond and that are on the list. The provisioning device can then send these devices credentials for a local WiFi network. The artificial reality devices can automatically, upon receipt of the credentials, connect to the administration system to obtain the software update and/or the configuration data.

Abstract: Disclosed are systems, methods, and non-transitory computer-readable media for reducing latency in augmented reality displays. A display controller receives, from a GPU, a stream of image pixels of a frame of virtual content to be presented on a display of a display device. The stream of image pixels is received via a high-speed bulk interface that transfers data at least as fast as can be consumed by the display. As the stream of image pixel is received, the display controller converts each respective image pixel from a data format used to transmit the stream of image pixels via the high-speed bulk interface to a data format that is compatible for display by the display. Each converted image pixel is stored in a pixel cell of the display, after which the frame is presented on the display.

Abstract: A nanovoided polymer-based material may include a bulk polymer material defining a plurality of nanovoids and an interfacial film disposed at an interface between each of the plurality of nanovoids and the bulk polymer material. The interfacial film may include one or more layers of material. A method of forming a nanovoided polymer-based material may include (1) forming a bulk polymer material defining a plurality of nanovoids and (2) forming an interfacial film at an interface between each of the plurality of nanovoids and the bulk polymer material. Various other methods, systems, and materials are also disclosed.

Abstract: The present embodiments relate to automated memory creation and retrieval from moment content items. In some implementations, the automated memory creation and retrieval system can obtain moment content items from user-designated sources with a single user perspective or multiple user perspectives. The moment content items can be assigned tags and arranged in chronological order. The arranged moment content items can be clustered into memory content items based on clustering conditions. Once memory content items are created, they can be arranged into a memory hierarchy made up of connected nodes. In some implementations, the memory content items are also linked together based on similarity in various dimensions in a memory graph. The automated memory creation and retrieval system can receive search criteria for memories from a user interface and provide the user with memories from matched nodes in the memory hierarchy or linked memories in the memory graph.

Abstract: A bender beam actuator includes a first layer of piezoelectric material and a second layer of piezoelectric material overlying a portion of the first layer of piezoelectric material, where a length of the first layer of piezoelectric material is at least 2% greater than a length of the second layer of piezoelectric material.

Abstract: The disclosed computer-implemented method may include (1) presenting, using a visual display, an artificial environment to a user, (2) determining whether a virtual contact is being initiated between a virtual object in the artificial environment and a surface location of the user served by a vibrotactile stimulator coupled to the user, and (3) actuating, in response to determining that a virtual contact is being initiated between the virtual object and the surface location, the vibrotactile stimulator to generate a vibrotactile signal to simulate initiation of the virtual contact at the surface location, where the vibrotactile signal includes a magnitude-varying signal having a frequency that is constant relative to a simulated rigidity of the virtual object as presented by the visual display. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A disclosed system may include (1) a wearable dimensioned to be donned on a body of a user of a computing device (2) a set of sensors coupled to the wearable, wherein the set of sensors detect one or more neuromuscular signals via the body of the user, and (3) at least one processing device communicatively coupled to the set of sensors, wherein the processing device (A) determines, based at least in part on the neuromuscular signals detected by the set of sensors, an intention of the user in connection with an application running on the computing device and, in response to determining the intention of the user, (B) modifying a feature of the application running on the computing device to account for the intention of the user. Various other devices, systems, and methods are also disclosed.

Abstract: In one embodiment, a method includes receiving a source image and its associated parameters from each of multiple image sources, associating each of the source images with a layer in a range of layers based on the parameters associated with the source images, the range of layers specifying a composition layering order of the source images, generating a corresponding customized distortion mesh for each particular source image in the source images based on the parameters associated with the particular source image and at least a portion of the parameters associated with each of the source images that is associated with any layer preceding a layer associated with the particular source image, modifying each of the source images using the corresponding customized distortion mesh, generating a composite image using the modified source images, and displaying the composite image as a frame in a video.

Abstract: Methods and systems for image sensing are provided. In one example, an apparatus comprises a semiconductor substrate comprising a light incident surface to receive light, a first pinned photodiode, and a second pinned photodiode, the first pinned photodiode and the second pinned photodiode forming a stack structure in the semiconductor substrate along an axis perpendicular to the light incident surface, the stack structure enabling the first pinned photodiode and the second pinned photodiode to, respectively, convert a first component of the light and a second component of the light to first charge and second charge. The apparatus further comprises one or more capacitors formed in the semiconductor substrate and configured to generate a first voltage and a second voltage based on, respectively, the first charge and the second charge.

Abstract: A display assembly includes a light source, a spatial light modulator (SLM) and a grating. The light source is configured to emit illumination light, the SLM is configured to receive the illumination light and reflect at least a portion of the illumination light. The grating is positioned to redirect the illumination light output from the light source toward the SLM, receive at least a portion of the reflected light from the SLM, redirect first light having a first polarization toward the light source, and transmit through the grating second light having a second polarization that is orthogonal to the first polarization. Also disclosed are operations performed by the display assembly.

Abstract: The disclosed transparent circuit board may include a transparent substrate, a first conductive trace coupled to the transparent substrate, and an electrical component having a bottom surface facing the transparent substrate and a top surface facing away from the transparent substrate. The bottom surface may include a first electrical contact coupled to the first conductive trace, and the top surface may include a second electrical contact. The transparent circuit board may also include a second conductive trace electrically connected to the second electrical contact and a transparent encapsulation layer coupled to the transparent substrate and encapsulating the electrical component. Various other ophthalmic devices, systems, and methods are also disclosed.

Abstract: An apparatus for creating and tuning haptic feedback is provided. The apparatus includes (A) a brace with first and second portions, (B) a vibrotactile actuator, coupled to the first portion of the brace, configured to apply haptic stimulations to a user, and (C) a linear actuator, coupled to the second portion of the brace, configured to (i) obtain a signal generated based on information about the user's body at a target location of the user and (ii) adjust a tension of the brace according to the signal. In some embodiments, the apparatus also includes one or more sensors configured to generate the information about the user's body at the target location. Furthermore, a controller may be configured to: (i) generate the signal based on the information about the user's body at the target location generated by the one or more sensors and (ii) provide the signal to the linear actuator.

Abstract: Disclosed herein includes a system, a method, and a device for interacting with displayed items. A contact point can be established with a surface of a contact panel and the contact panel can be operated in a navigation mode during which a selection element, for selecting one or more items of a user interface (UI) in a display of a second device, is maintained at a determined location in the display. The UI can be moved in the display relative to the selection element as the selection element is maintained at the determined location, by moving the contact point on the surface of the contact panel in the navigation mode. At least one item of the UI that is moved within a selection threshold of the selection element can be selected by removing the contact point with the contact panel.

Abstract: In general, this disclosure describes scalable, partitionable encryption engines. The partitionable encryption engines of this disclosure yield power savings, such as by controlling operation of partitioned sub-datapaths at reduced clock rates. An apparatus includes an interface configured to receive a block of encrypted data for decryption, and a decryption engine in communication with the interface. The decryption engine includes a plurality of decryption sub-datapaths, where each respective decryption sub-datapath has no data interdependency with any other decryption sub-datapath of the plurality of decryption sub-datapaths. The decryption engine is configured to selectively enable one or more decryption sub-datapaths of the plurality of decryption sub-datapaths to decrypt the block of encrypted data to form a decrypted block of data.

Abstract: In one embodiment, a computing system may receive a video including a sequence of frames. The computing system may access a three-dimensional mask that specifies pixel-sampling locations, the three-dimensional mask having a first dimension and a second dimension corresponding to a spatial domain and a third dimension corresponding to a temporal domain. Blue noise property may be present in the pixel-sampling locations that are associated with each of a plurality of two-dimensional spatial slices of the three-dimensional mask in the spatial domain and the pixel-sampling locations that are associated with each of a plurality of one-dimensional temporal slices of the three-dimensional mask in the temporal domain. The computing system may generate a sample of the video by sampling the sequence of frames using the three-dimensional mask.

Abstract: Disclosed herein includes a system, a method, and a device for improving computation efficiency of a neural network. In one aspect, adder circuitry is configured to add input data from processing of the neural network and a first number of bits of accumulated data for the neural network to generate summation data. In one aspect, according to a carry value of the adding from the adder circuitry, a multiplexer is configured to select between i) a second number of bits of the accumulated data and ii) incremented data comprising the second number of bits of the accumulated data incremented by a predetermined value. The summation data appended with the selected one of the second number of bits of the accumulated data or the incremented data may form appended data.

Abstract: An animatronic device includes a mechanical eyeball configured to rotate about a first rotational axis and a second rotational axis that intersect at a fixed center point. A controller is configured to generate eye movement instructions that cause the animatronic device to rotate the mechanical eyeball about at least one of the first rotational axis and the second rotational axis. The eye movement instructions are generated based on an eye tracking system used to track movement of an eye of a user. The controller maps the tracked eye movement to movement of the mechanical eyeball and generates eye movement instructions based on the mapping.

Abstract: Techniques for adjusting an optical element using an actuator assembly. The optical element is adjusted by translating the optical element along a linear axis of motion or by applying force upon a surface of the optical element. The optical element can be a liquid lens that is shaped by the application of force upon a surface of the lens. The actuator assembly includes a plurality of lead screws and a mechanical linkage that intercouples the lead screws and that is configured to simultaneously rotate the plurality of lead screws. The actuator assembly further includes a displacement element configured to act upon the optical element, through translational motion of the displacement element in response to rotation of the lead screws. Multiple optical elements can be adjusted simultaneously using respective displacement elements coupled to the lead screws.

Abstract: A lens assembly includes a first optical element and a second optical element. The first optical element includes a flat partially reflective layer. The second optical element includes a curved reflective polarizer layer.

Abstract: Disclosed herein are related to a system and a method of remotely rendering an image. In one approach, a console device generates an image according to a gaze direction of a user of a head mounted display (HMD). In one aspect, the image includes a first area and a second area disposed along an axis, where the second area is located farther away from a foveated area of the image than the first area. In one aspect, the foveated area corresponds to the gaze direction of the user of the HMD. In one aspect, the console device compresses the image according to the axis, where the second area is compressed at a higher level than the first area. In one aspect, the compressed image is transmitted to the HMD. The HMD may decompress the compressed image according to the axis, and render the decompressed image.

Abstract: The invention is directed towards enhanced systems and methods for employing a pulsed photon (or EM energy) source, such as but not limited to a laser, to electrically couple, bond, and/or affix the electrical contacts of a semiconductor device to the electrical contacts of another semiconductor devices. Full or partial rows of LEDs are electrically coupled, bonded, and/or affixed to a backplane of a display device. The LEDs may be ?LEDs. The pulsed photon source is employed to irradiate the LEDs with scanning photon pulses. The EM radiation is absorbed by either the surfaces, bulk, substrate, the electrical contacts of the LED, and/or electrical contacts of the backplane to generate thermal energy that induces the bonding between the electrical contacts of the LEDs' electrical contacts and backplane's electrical contacts. The temporal and spatial profiles of the photon pulses, as well as a pulsing frequency and a scanning frequency of the photon source, are selected to control for adverse thermal effects.

Abstract: A device for providing a 1D line of an image is disclosed. The device includes a wavelength-tunable light source for providing image light having the angular distribution encoded in optical spectrum. The device further includes a thin slab waveguide having an out-coupler in form of a diffraction grating for out-coupling the image light at an angle dependent on wavelength. The image may be formed by scanning a collimated beam propagating in the slab waveguide when using tunable monochromatic light sources, or by forming the 1D singular distribution of brightness at a same time when using a tunable-spectrum light sources. The device may be used in a near-eye display for forming a 2D image in angular domain.

Abstract: A flexure guidance system may be provided for controlling movement of an optical subassembly and/or a connected combiner lens. For instance, the flexure guidance system may include a distal end piece, a proximal end piece, and multiple wire flexures that link the distal end piece to the proximal end piece. The linking wire flexures may be spaced to form an interior cavity between the distal end piece and the proximal end piece. This interior cavity may house various electronic components. One or more actuators in the system may move the electronic components according to input signals along different axes of movement provided by the wire flexures. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A display device includes a projector and a display that has a first surface and a second surface. The projector is configured to project image light toward the display. The display is configured to output diffused image light from the first surface and to transmit ambient light from the second surface to the first surface. The display device also includes an optical assembly that has a substrate with substantially uniform thickness. The optical assembly is configured to receive the diffused image light and transmit a portion of the diffused image light output from the first surface of the display at a first optical power. The optical assembly is also configured to receive the ambient light and transmit a portion of the ambient light through the optical assembly at a second optical power that is less than the first optical power.

Abstract: An electrostatic zipping actuator includes a primary electrode, a secondary electrode overlying the primary electrode, a dielectric layer located between and abutting at least a portion of the primary electrode and the secondary electrode, and a dielectric fluid disposed at least at a junction between the dielectric layer and one of the electrodes, where an average total thickness of the dielectric layer is less than approximately 10 micrometers.

Abstract: An optical waveguide is provided. The optical waveguide includes a base structure and a plurality of grating structures disposed at the base structure. The grating structures include a plurality of in-coupling grating structures configured to couple a plurality of lights into the optical waveguide. At least one of a grating period or a slant angle of a first in-coupling grating structure is different from at least one of a corresponding grating period or a corresponding slant angle of a second in-coupling grating structure. The grating structures also include a plurality of out-coupling grating structures configured to couple the lights out of the optical waveguide.

Abstract: Ghost images can interfere with eye tracking in a system that uses folded optics, such as is an artificial-reality display. Optical elements, such as waveplates and/or polarizers, can be used to attenuate or eliminate light causing the ghost images.

Abstract: Example devices include a fluidic device, such as a fluidic valve, including a body formed from a rigid body material including a fluidic source, a fluidic drain, and a fluidic gate, each of which may have a fluid connection with a chamber, or a portion thereof. The device may further include a gate transmission element, located within the chamber, that is controllable between at least a first position and a second position using a gate pressure received through the fluidic drain. Adjustment of the position of the gate transmission element may allow control of fluid flow through the device. Other devices, methods, systems, and computer-readable media are also disclosed.

Abstract: Aspects of the present disclosure are directed to virtual interaction modes for social and group communication. Additional aspects of the present disclosure are directed to automated controls for connecting a trigger performed in an artificial reality environment to an action on a personal computing device. Further aspects of the present disclosure are directed to providing versions of person virtual objects in an artificial reality environment and contextual breakpoints to switch between them. Yet further aspects of the present disclosure are directed to extending a mouse for artificial reality environment input.

Abstract: A polarization volume hologram (PVH) lens includes a PVH layer having a freeform design. The PVH layer includes a first region and a second region having different optical properties.

Abstract: A system for vibration-driven sensing may include a wearable dimensioned to be donned by a user of an artificial reality system. The system may also include a vibration sensor that is incorporated into the wearable and generates an electrical response that corresponds to a vibration detected at the wearable. The system may further include at least one processing device communicatively coupled to the vibration sensor. The processing device may determine, based at least in part on the electrical response generated by the vibration sensor, that the user has made a specific gesture with at least one body part. In response to this determination, the processing device may generate an input command for the artificial reality system based at least in part on the specific gesture made by the user. Various other systems and methods are also disclosed.

Abstract: A telecentric optical element that has a first surface, a second surface, and a layer of optically anisotropic material between the first surface and the second surface. The telecentric optical element is configured to receive light on the first surface, the light having propagated from a location within a predetermined distance range from the first surface of the optical element. The telecentric optical element is also configured to redirect a first portion of the light, having a first polarization, off the first surface of the telecentric optical element in an off-axis direction, and to transmit a second portion of light, having polarization different from the first polarization, to the second surface. An eye-tracking assembly that includes the telecentric optical element and a display device that includes the telecentric optical element are also disclosed.

Abstract: The disclosed wearable apparatus may include (1) a device dimensioned to fit about a body part of a user, (2) a tensioning mechanism that harnesses to the device in a manner that is relative to the fit of the device about the body part of the user, and (3) an actuator couples to the tensioning mechanism that, when actuated, causes substantially tangential movement of the tensioning mechanism relative to a surface of the body part to produce substantially orthogonal movement of the device relative to the surface of the body part. Various other methods, systems, and/or computer-readable media are also disclosed.

Abstract: A scanning projector display includes a plurality of light engines coupled to a MEMS scanner. Each light engine includes a light source subassembly for providing a diverging light beam optically coupled to a collimator for collimating the diverging light beam. In operation, the collimated light beams of the plurality of light engines impinge onto the tiltable reflector at different angles of incidence. A controller may be operably coupled to the light source subassembly of each light engine of the plurality of light engines and the MEMS scanner for tilting the tiltable reflector of the MEMS scanner. The controller is configured to energize the light source of each light engine in coordination with tilting the tiltable reflector for displaying the image.

Abstract: In one embodiment, a method includes receiving, from a client system associated with a first user, a first audio input. The method includes generating multiple transcriptions corresponding to the first audio input based on multiple automatic speech recognition (ASR) engines. Each ASR engine is associated with a respective domain out of multiple domains. The method includes determining, for each transcription, a combination of one or more intents and one or more slots to be associated with the transcription. The method includes selecting, by a meta-speech engine, one or more combinations of intents and slots from the multiple combinations to be associated with the first user input. The method includes generating a response to the first audio input based on the selected combinations and sending, to the client system, instructions for presenting the response to the first audio input.

Abstract: In one embodiment, a method includes, by one or more computing systems, receiving, from a client system, an input in a multi-turn message thread, parsing the input to identify a plurality of hypothesis dialog states associated with the input, generating a plurality of functions corresponding to the plurality of hypothesis dialog states, calculating a plurality of probability scores for the plurality of functions, respectively, based on a prior dialog state associated with the message thread and a current user context, wherein each probability score represents a likelihood that the respective function is associated with an intended dialog state, and storing the plurality of functions if each of the probability scores is below a threshold probability score, wherein the plurality of functions is stored as a hierarchical set of functions that is used as the prior dialog state in response to a next input in the message thread.

Abstract: An optical system includes an optical lens, a polarization volume hologram (PVH) layer arranged over the optical lens, and an IR absorbing structure arranged between the optical lens and the PVH layer. The PVH layer being configured to reflect infrared (IR) light. The IR absorbing structure includes a quarter-wave plate (QWP) arranged between the optical lens and the PVH layer and a linear absorptive polarizer arranged between the QWP and the optical lens. The linear absorptive polarizer is configured to absorb IR light.

Abstract: In one embodiment, a method includes receiving a first frame associated with a first time and one or more second frames of a video having a resolution lower than a target resolution, wherein each second frame is associated with a second time prior to the first time, generating a first feature map for the first frame and one or more second feature maps for the one or more second frames, up-sampling the first feature map and the one or more second feature maps to the target resolution, warping each of the up-sampled second feature maps according to a motion estimation between the associated second time and the first time, and generating a reconstructed frame having the target resolution corresponding to the first frame by using a machine-learning model to process the up-sampled first feature map and the one or more up-sampled and warped second feature maps.

Abstract: A system on a chip (SoC) includes a security processor configured to determine that a first channel ID describing a {source, destination} tuple for a crypto packet matches a second channel ID describing a corresponding {source, destination} tuple for a preceding crypto packet received immediately prior to the crypto packet. The SoC also includes a decryption engine configured to, responsive to the determination that the first channel ID matches the second channel ID: obtain a set of round keys applied to perform an add round key computational stage of a previous decryption datapath used to decrypt a preceding cipher text block obtained from the preceding crypto packet, and to reuse the set of round keys to perform a corresponding add round key computational stage of a current decryption datapath used to decrypt a cipher text block obtained from the crypto packet.

Abstract: In one embodiment, a method includes receiving a request to transfer a handle to an object from a first process to a second process, accessing a first security context of the handle and a second security context of the second process, identifying one or more security policies based on at least one of the first security context and the second security context, determining that the handle is allowed to be transferred to the second process by applying the one or more security policies on the first security context of the handle and the second security context of the second process, and transferring the handle to the second process in response to the request.