Abstract: A camera captures image data at a target frame rate. The camera includes a sensor and a controller. The sensor is configured to detect light from a local area and includes a plurality of augmented pixels. Each augmented pixel comprises at least a first and a second gate. The first gates are configured to store a first plurality of image frames as first image data according to a first activation pattern. The second gates are configured to store a second plurality of image frames as second image data according to a second activation pattern. The controller reads out the image data to generate a first image from the first image data and a second image from the second image data. The first and second images may be used to reconstruct a combined set of image frames at the target frame rate with a reconstruction algorithm.

Abstract: In one embodiment, a method includes accessing a distortion mesh defined in a 3D space by a plurality of mesh coordinates that are respectively associated with a plurality of predetermined points in a screen representation, where each mesh coordinate specifies a trajectory for a ray cast from the associated predetermined point into the 3D space; determining a sampling point within the screen representation; selecting a set of mesh coordinates from the plurality of mesh coordinates based on proximities between the sampling point and the predetermined points associated with the set of mesh coordinates; computing a sampling trajectory for the sampling point based on the selected set of mesh coordinates; casting a ray into the 3D space using the sampling trajectory; computing an intersection location between the cast ray and an object in the 3D space; and determining a color value for the sampling point based on the intersection location.

Abstract: The disclosed projector device may include (1) a first monochromatic emitter array having a plurality of emitters of a first color disposed in a two-dimensional configuration and (2) a second monochromatic emitter array having a plurality of emitters of a second color disposed in a two-dimensional configuration. The first and second monochromatic emitter arrays may be configured to emit images of the first and second colors into a waveguide configuration, and the first color may be different than the second color. Associated display systems and methods are also provided.

Abstract: A mechanical eyeball includes an outer housing shaped as an ocular surface configured to rotate about a first rotational axis and a second rotational axis that intersect at a fixed center point. The outer is housing is coupled to a mechanical assembly, and the mechanical assembly is contained within a volume associated with the mechanical eyeball. The mechanical assembly can include a stationary gear train and rotatable components that rotate relative to the gear train. The rotatable components are configured to cause rotation of the outer housing about one or more rotational axes. The volume may be substantially the same volume of a human eye. The mechanical assembly is coupled to one or more drivers configured to actuate rotation of the outer housing.

Abstract: A head-mounted display device for providing augmented reality contents to a wearer includes a light projector, and eye tracker, and a beam steerer. The light projector is configured to project light for rendering images based at least on the augmented reality contents. The eye tracker is configured to determine a position of a pupil of an eye of the wearer. The beam steerer is configured to change a direction of the light from the light projector based on the position of the pupil.

Abstract: In an example of the described techniques, a wireless communication system includes first memory, second memory, a direct memory access (DMA) controller, an encryption engine in-line between the DMA controller and the second memory, a first microprocessor, and a second microprocessor. The first microprocessor communicates with other systems that generate application data to be wirelessly transmitted. The application data to be wirelessly transmitted is stored in the second memory and programs the DMA controller to transfer packets of the application data to the first memory from the second memory. The encryption engine receives the packets of the application data from the DMA controller, encrypts the packets to generate encrypted application data packets, and outputs the encrypted application data packets for storage to the first memory.

Abstract: A depth camera assembly (DCA) optimizes illumination and image capture of a local area to generate depth information of the local area. The DCA determines depth information for a first portion of the local area viewable at a first pose. The DCA is moved from the first pose to a second pose, where a second portion of the local area is viewable and overlaps with the first portion. The overlapping region is not illuminated by the DCA. A non-overlapping portion of the second portion is illuminated, captured, and depth information determined.

Abstract: The disclosed touch-simulation apparatus includes (1) a set of opposing surfaces and (2) an electric actuator assembly coupled between the set of opposing surfaces, wherein (A) the electric actuator assembly includes an actuator layer and (B) a set of mechanical amplifiers coupled between opposing sides of the actuator layer and the set of opposing surfaces. Various other apparatuses, systems, and methods are also disclosed.

Abstract: A virtual reality (VR) system includes a console an imaging device, a head mounted display (HMD) and a user input device. The console includes a virtual reality (VR) engine, a tracking module and an application store. The HMD includes a display element configured to display content to a user wearing the HMD. The virtual reality engine is configured to determine a vector for a virtual object thrown at a target in response to physical input from the user, a gaze vector for the eyes of the user and a virtual object position vector for the target and modify the vector for the virtual object based on at least one of the gaze vector and the virtual object position vector.

Abstract: An artificial reality system can display notifications in various configurations including in a minimized version or a maximized version. The artificial reality system can initially display a minimized version of the notification. Upon identifying a first trigger event, such as the user's gaze resting on the minimized version of the notification or a rotation of a user's wrist, the artificial reality system can change the display of the notification from the minimized version to the maximized version. The artificial reality system can identify a second trigger event, such as the user's gaze not being directed to the maximized version of the notification or a rotation of the wrist in an opposite direction from the first wrist rotation. In response to the second trigger event, the artificial reality system can stop display of the maximized version of the notification.

Abstract: An acoustic element includes a nanovoided polymer layer having a first nanovoid topology in an unactuated state and a second nanovoid topology different than the first nanovoid topology in an actuated state. Capacitive actuation of the nanovoided polymer layer, for instance, can be used to reversibly control the size and shape of the nanovoids within the polymer layer and hence tune its sound damping characteristics or sound transduction behavior, e.g., during operation of the acoustic element. An acoustic element may be configured for passive or active sound attenuation. Various other apparatuses, systems, materials, and methods are also disclosed.

Abstract: A display system comprises a light source assembly, a display assembly, a camera assembly, and a controller. The light source assembly is configured to generate visible light and infrared light. The display assembly is configured to generate image light using the visible light and to generate tracking light using the infrared light. The tracking light and the image light pass through a beam splitter prior to being projected into a local area. The camera assembly is configured to capture one or more images of the local area illuminated with the tracking light. The images are illuminated with the tracking light that passes through the beam splitter prior to be being captured by the camera assembly. The controller is configured to calculate depth information for objects in the local area using the one or more images.

Abstract: The present technology relates to artificial reality systems. Such systems provide projections a user can create to specify object interactions. For example, when a user wishes to interact with an object outside her immediate reach, she can use a projection to select, move, or otherwise interact with the distant object. The present technology also includes object selection techniques for identifying and disambiguating between objects, allowing a user to select objects both near and distant from the user. Yet further aspects of the present technology include techniques for interpreting various bimanual (two-handed) gestures for interacting with objects. The present technology further includes a model for differentiating between global and local modes for, e.g., providing different input modalities or interpretations of user gestures.

Abstract: This disclosure describes techniques that include aligning processing of audio samples collected by multiple audio sensors or microphones. In one example, this disclosure describes a method comprising enabling a first microphone; processing, by an audio processor and using a first processing pipeline, audio data samples collected by the first microphone; enabling a second microphone a period of time after enabling the first microphone; processing, by the audio processor and using a second processing pipeline, a sample of audio data collected by the second microphone by synchronizing starting times for the first and second processing pipelines.

Abstract: A design system generates a design for an eyewear frame customized for a user. The eyewear frame provides audio content to the user. The design system captures anthropometric data of the user. Using machine learning techniques, the design system determines features of the user from the anthropometric data and generates a three dimensional (3D) geometry of the portion of the user's head. A design for the customized eyewear frame is generated based on the 3D geometry of the portion of the user's head. The design of the customized eyewear frame includes design parameters that describe a shape of a coupling element that ensures the eyewear frame is customized to the user's head.

Abstract: Methods and apparatus for substantially real-time detection of spike events in neuromuscular data. The method comprises receiving a plurality of neuromuscular signals from a plurality of neuromuscular sensors arranged on one or more wearable devices worn by a user, detecting, based on the plurality of neuromuscular signals or information derived from the plurality of neuromuscular signals, at least one spike event corresponding to firing of an action potential in at least one motor unit, determining, based on the plurality of neuromuscular signals or the information derived from the plurality of neuromuscular signals, a biological source of the detected at least one spike event, and generating at least one output based, at least in part, on the detected at least one spike event and/or the determined biological source of the detected at least one spike event.

Abstract: An optical device includes a waveguide, a projector, a reflective display, and an in-coupler. The waveguide has a first side and an opposing second side. The projector is configured to project illumination light toward the first side of the waveguide. The reflective display is configured to receive the illumination light and to output image light toward the second side of the waveguide. The in-coupler is configured to receive the image light output by the reflective display and redirect a portion of the image light so that the portion of the image light undergoes total internal reflection inside the waveguide.

Abstract: There is disclosed a method of and apparatus for predictive tracking for a head mounted display. The method comprises obtaining one or more three-dimensional angular velocity measurements from a sensor monitoring the head mounted display and setting a prediction interval based upon the one or more three-dimensional angular velocity measurements such that the prediction interval is substantially zero when the head mounted display is substantially stationary and the prediction interval increases up to a predetermined latency interval when the head mounted display is moving at an angular velocity of or above a predetermined threshold. The method further includes predicting a three-dimensional orientation for the head mounted display to create a predicted orientation at a time corresponding to the prediction interval, and generating a rendered image corresponding to the predicted orientation for presentation on the head mounted display.

Abstract: In one embodiment, a system may access a codec that encodes an appearance associated with a subject and comprise codec portions that respectively correspond to body parts of the subject. The system may generate a training codec that comprises a first subset of the codec portions (a first set of body parts) and a modified second subset of the codec portions (muted body parts). The system may decode the training codec using a machine-learning model to generate a mesh of the subject. The system may transform the mesh of the subject based on a predetermined pose. The system may update the machine-learning model based on a comparison between the transformed mesh and a target mesh of the subject having the predetermined pose. The system in the present application can train a machine-learning model to render an avatar with a pose using uncorrelated codec portions corresponding to different body parts.

Abstract: A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.