Abstract: The disclosed computer-implemented method may include (1) receiving a first 2D frame depicting an evolving 3D scene and elements in the evolving 3D scene, (2) receiving a second 2D frame depicting the evolving 3D scene and the elements, (3) deriving 2D motion vectors from the first 2D frame and the second 2D frame that each include an estimated offset from coordinates of an element in the first 2D frame to coordinates of the element in the second 2D frame, (4) receiving depth information for the evolving 3D scene, (5) using the 2D motion vectors and the depth information to extrapolate a synthetic 2D frame, and (6) displaying the synthetic 2D frame to a user. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: An audio system includes a transducer assembly, an optical sensing pathway, a laser, a detector assembly, and a controller. The transducer assembly is coupled to a user's ear and produces an acoustic pressure wave based on an audio instruction. The optical sensing pathway moves, at least in part, with a detected acoustic pressure wave. The laser emits light that is separated into a reference beam and a sensing beam that is coupled into the optical sensing pathway. The detected acoustic pressure wave interacts with the sensing beam to alter its optical path length. The detector assembly detects the reference and sensing beams from the optical sensing pathway, and measures the detected acoustic pressure wave based on changes in optical path length between the reference beam and the sensing beam. The controller adjusts the audio instruction based on the measurement of the detected acoustic pressure wave.

Abstract: A touch sensing assembly includes a touch sensing layer that is configured to couple to an interface. The interface is configured to receive contact with skin of a user on a first surface. A first touch sensing layer is coupled to interface. The first touch sensing layer includes a plurality of touch sensors and a plurality of electrodes. The touch sensors are of a first touch sensing type, configured to detect proximity of the user's skin to the interface. The electrodes provide a plurality of electrical signals corresponding to the proximity of the user's skin to the interface.

Abstract: The disclosed computer-implemented method may include (1) establishing a communication channel to indirectly convey a conversation, (2) receiving, via the communication channel, a portion of the conversation, (3) presenting the portion of the conversation to a user, (4) receiving, via the communication channel, an additional portion of the conversation, (5) detecting an additional communication channel capable of conveying the conversation, (6) determining a human-perceivable difference between how the conversation has been conveyed via the communication channel and how the conversation will be conveyed via the additional communication channel, and (7) compensating for the human-perceivable difference when presenting the additional portion of the conversation to the user in order to smoothly transition the conversation from the communication channel to the additional communication channel. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A virtual reality system including a garment worn by a user, such as a glove, includes a jamming that jams movement of a portion of the user's body by increasing a rigidity of certain portions of the garment or by preventing a certain portion of the garment from expanding past a certain length. This allows the garment to simulate the physical sensation that occurs when the user touches an object. For example, to simulate the sensation of holding a coffee mug, the jamming mechanism prevents the user's fingers from curling after the user's fingers have reached a position equivalent to making physical contact with the coffee mug.

Abstract: A system includes an eyewear device configured to present content to a user. A processor is communicatively coupled to the eyewear device. A bracelet device is communicatively coupled to the processor, and includes at least one sensor configured to determine a position signal in response to movement of a user's hand. A depth camera assembly is communicatively coupled to the processor, and configured to emit a waveform into an environment of the user, and capture a plurality of optical signals from the waveform reflected off of at least one object in the environment. The processor is configured to determine a position of the user's hand in relation to the environment based in part on the position signal and the plurality of optical signals.

Abstract: Disclosed herein includes a system, a method, and a non-transitory computer readable medium for applying an expression to an avatar. In one aspect, a class of an expression of a face can be determined according to a set of attributes indicating states of portions of the face. In one aspect, a set of blendshapes with respective weights corresponding to the expression of the face can be determined according to the class of the expression of the face. In one aspect, the set of blendshapes with respective weights can be provided as an input to train a machine learning model. In one aspect, the machine learning model can be configured, via training, to generate an output set of blendshapes with respective weights, according to an input image. An image of an avatar may be rendered according to the output set of blendshapes with respective weights.

Abstract: In one embodiment, a method includes starting a network-coordination service, a network-interface component, and an application as separate processes within a user space of an operating system of a device; receiving, by the network-coordination service, a request from the application to connect with a host; selecting, by the network-coordination service and in response to the request, the network-interface component to service the requested connection with the host, wherein the network-interface component is associated with a network interface of the device; allocating, by the network-coordination service, a shared memory region for the application and the selected network-interface component, wherein the shared memory region is associated with a reference; and sending, by the network-coordination service, the reference to the application, wherein the reference is configured to be used by the application to access the shared memory region; wherein the shared memory region is configured to be used by the appli

Abstract: Computerized systems, methods, kits, and computer-readable media storing code for implementing the methods are provided for interacting with a physical object in an augmented reality (AR) environment generated by an AR system. One such system includes: a plurality of neuromuscular sensors able to sense a plurality of neuromuscular signals from a user, and at least one computer processor. The neuromuscular sensors are arranged on one or more wearable devices worn by the user to sense the neuromuscular signals. The at least one computer processor is or are programmed to: determine, based at least in part, on the neuromuscular signals sensed by the neuromuscular sensors, information about an interaction of the user with the physical object in the AR environment generated by the AR system; and instruct the AR system to provide feedback based, at least in part, on the information about the interaction of the user with the physical object.

Abstract: A computing device, such as a headset, provides for selective sound level reduction or sound level amplification for sounds made in a local area. The computing device includes a speaker assembly and a controller. The controller receives an audio signal from a second computing device, such as a second headset, via a wireless connection with the computing device. The audio signal describes a sound made by a user of the second computing device. The audio signal arrives at the computing device prior to the sound. The controller determines an audio filter to selectively adjust an amplitude of the sound and applies the audio filter to the audio signal to generate audio content. The controller presents, via the speaker assembly, the audio content to a user such that the presented audio content combines with the sound to selectively adjust the amplitude of the sound.

Abstract: Methods and systems for performing analog-to-digital conversion are proposed. In one example, An analog-to-digital converter (ADC) comprising a quantizer, the quantizer having a first quantization resolution for a first quantization operation subrange and a second quantization resolution for a second quantization operation subrange. At least one of the first quantization resolution or the first quantization operation subrange is programmable. At least one of the second quantization resolution or the second quantization operation subrange is programmable. The quantizer is configured to: receive an input voltage; and based on whether the input voltage belongs to the first quantization operation subrange or to the second quantization operation subrange, quantize the input voltage at the first quantization resolution or at the second quantization resolution to generate a digital output.

Abstract: Methods and apparatus for authenticating a user based on neuromuscular signals. The method comprises recording, using a plurality of neuromuscular sensors arranged on one or more wearable devices, a plurality of neuromuscular signals from a user; deriving a neuromuscular signature for the user from the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals, wherein the neuromuscular signature is indicative of at least one personal characteristic of the user detected in the plurality of neuromuscular signals; and authenticating the user based on the derived neuromuscular signature.

Abstract: Embodiments disclosed herein relate to a graphics processing chip for rendering computer graphics. The graphics processing chip may include a controller configured to manage operations of the graphics processing chip in accordance with a graphics-rendering pipeline. The operations may include geometry-processing operations, rasterization operations, and shading operations. The chip may further include programmable memory components configured to store a machine-learning model configured to perform at least a portion of the shading operations. The chip may also include a plurality of processing units configured to be selectively used to perform the shading operations in accordance with the machine-learning model. The chip may also include at least one output memory configured to store image data generated using the shading operations.

Abstract: In one embodiment, a computing system may receive a number of texels organized into a two-dimensional array. The system may generate addresses for the texels based on one or more mapping rules which may map the texels from the two-dimension array into a one-dimensional array of a pre-determined size in a texel order. The system may store the texels organized in the one-dimensional array into a memory block having the pre-determined size. The system may read texels from the memory block onto a data bus including a number of data lines corresponding to different combinations of low order address bits of addresses of the texels within the two-dimension array. The texel order of the one-dimensional array may map texels having same low order address bits into same data lines. The system may load the texels directly into a number of buffer memory blocks through the data bus.

Abstract: Disclosed herein are related to a system and a method of remotely rendering an image. In one approach, a console device generates an image according to a gaze direction of a user of a head mounted display (HMD). In one aspect, the image includes a first area and a second area disposed along an axis, where the second area is located farther away from a foveated area of the image than the first area. In one aspect, the foveated area corresponds to the gaze direction of the user of the HMD. In one aspect, the console device compresses the image according to the axis, where the second area is compressed at a higher level than the first area. In one aspect, the compressed image is transmitted to the HMD. The HMD may decompress the compressed image according to the axis, and render the decompressed image.

Abstract: An audio system presented herein includes a transducer array, sensor array, and a controller. The controller control tactile content imparted to a user via actuation of at least one transducer in the transducer array while presenting audio content to the user. The transducer array presents the audio content with the tactile content to the user. The audio system can be part of a headset.

Abstract: A depth camera assembly (DCA) determines depth information within a local area. The DCA may selectively process a subset of data captured by an imaging sensor and obtained from the imaging sensor, such as pixels corresponding to a region of interest, for depth information. Alternatively, the DCA may limit retrieval of data from the imaging sensor to pixels corresponding to the region of interest from the imaging sensor for processing for depth information. The depth processing may include a semi-global match (SGM) algorithm, and the DCA adjusts a number of neighboring pixels used for determining depth information for a specific pixel based on one or more criteria. In some embodiments, the DCA performs the depth processing by analyzing images from different image sensors using left to right and right to left correspondence checks that are performed in parallel.

Abstract: In one embodiment, a computing system accesses a three-dimensional (3D) model of an environment, the 3D model comprising a virtual representation of an object in the environment. The computing system accesses an image of the object captured by a camera from a camera pose. The computing system accesses material parameters associated with a material property for the virtual representation of the object. The computing system renders, using the 3D model, pixels associated with the virtual representation of the object based on the material parameters associated with the virtual representation of the object, the pixels being rendered from a virtual perspective corresponding to the camera pose. The computing system determines updated material parameters associated with the material property for the virtual representation of the object based on a comparison of the rendered pixels to corresponding pixels located in the image of the object.

Abstract: A depth measurement assembly (DMA) includes a pulsed illuminator assembly, a depth camera assembly, and a controller. The pulsed illuminator assembly has a structured light projector that projects pulses of structured light at a pulse rate into a local area. The depth camera assembly captures images data of an object in the local area illuminated with the pulses of structured light. An exposure interval of the depth camera assembly is pulsed and synchronized to the pulses projected by the pulsed illuminator assembly. The controller controls the pulsed illuminator assembly and the depth camera assembly so that they are synchronized. The controller also determine depth and/or tracking information of the object based on the captured image data. In some embodiments, the pulsed illuminator assembly have a plurality of structured light projectors that projects pulses of structured light at different times.

Abstract: A device includes a first light source device configured to transmit a first light in a first direction and a second light source device configured to transmit a second light in a second direction. The device also includes a first set of one or more lenses configured for directing the first light from the first light source device and the second light from the second light source device toward a first eye of a user. Also disclosed is a method that includes transmitting a first light and a second light through a first set of one or more lenses and directing the first light and the second light toward a first eye of a user. Further disclosed is a method for adjusting a distance between an eye and a lens of the first set of one or more lenses.