Abstract: In various examples there is a method performed by a Head Mounted Display, HMD, comprising a high field rate display configured to display fields of rendered frames at a field rate. The method comprises receiving a stream of the rendered frames for display on the high field rate display, the stream of rendered frames having a frame rate. The process applies an early stage reprojection to the rendered frames of the stream of rendered frames at a rate which is lower than the field rate. The process applies a late stage reprojection to fields of the rendered frames at the field rate, wherein the early stage reprojection uses more computational resources than the late reprojection.

Abstract: An optical combiner in a display system of a mixed-reality head-mounted display (HMD) device comprises a lens of birefringent material and a ferroelectric liquid crystal (FLC) modulator that are adapted for use with a reflective waveguide to provide multiple different focal planes on which holograms of virtual-world objects (i.e., virtual images) are displayed. The birefringent lens has two orthogonal refractive indices, ordinary and extraordinary, depending on the polarization state of the incident light. Depending on the rotation of the polarization axis by the FLC modulator, the incoming light to the birefringent lens is focused either at a distance corresponding to the ordinary refractive index or the extraordinary refractive index. Virtual image light leaving the birefringent lens is in-coupled to a see-through reflective waveguide which is configured to form an exit pupil for the optical combiner to enable an HMD device user to view the virtual images from the source.

Abstract: A model generation system generates three-dimensional object models based on two-dimensional images of an object. The model generation system can apply an iterative gradient decent process to model parameters for part models within an object model to compute a final set of model parameter values to generate the object model. To compute the final set of model parameter values, the model generation generates a reference image of the object model and compares the reference image to a received image. The model generation system uses a differentiable error function to score the reference image based on a received image. The model generation system updates the set of model parameter values based on the score for the reference image, and iteratively repeats the process until a reference image is sufficiently similar to the received image.

Abstract: A parallel presentation system includes computer hardware including a parallel presentation platform (PPP) connected to an extended reality (XR) system associated with a user. The computer hardware is configured to perform the following operations. A lookup point associated with the user is identified by the PPP. The PPP determines that the user has shifted focus from a first data stream to a second data stream. A shift time associated with the shift along with an identification of the first data stream is stored by the PPP in association with the user. A summarization of the first data stream is generated using a summarization engine of the PPP and the summarization is based upon the shift time and the lookup point. The summarization is forwarded by the PPP to the XR system. Information regarding the summarization is configured to be parallel-presented by the XR system associated with the user.

Abstract: Systems and methods for segmenting surgical videos are disclosed. One example method includes receiving, by a processor of a computing device, surgical video, the surgical video comprising at least a sequence of video frames of a surgical procedure; in response to receiving an identification of a video frame, generating, by the processor, a bookmark based on the video frame; associating, by the processor, the bookmark with the video frame; and storing, by the processor, the bookmark in a non-transitory computer-readable medium.

Abstract: Methods, systems, and storage media for auto-generating an artificial reality environment based on access to personal user content are disclosed. Exemplary implementations may: receive consent from a user to access user content on a user device, the user content comprising digital media; generate a user profile based at least in part on the user content; determine user preferences based at least in part on the user profile; generate an artificial reality environment based at least in part on the user preferences; and share the artificial reality environment with contacts of the user.

Abstract: A method for determining a hand force and a ground reaction force for a musculoskeletal body of a subject includes obtaining video data for the musculoskeletal body during an action taken by the subject, generating, for each frame of the video data, three-dimensional pose data for the subject based on a three-dimensional skeletal model, and determining the hand force and the ground reaction force based on the three-dimensional pose data. Determining the hand force and the ground reaction force includes implementing a reconstruction of the hand force and the ground reaction force based on the three-dimensional pose data. The method additionally includes applying the three-dimensional pose data, the estimate for the ground reaction force, and the estimate of the hand force, to a neural network or other model to optimize the estimate of the hand force and the estimate of the ground reaction force.

Abstract: A gaze initiated interaction technique for facilitating interaction between wearers of head-mountable devices. Methods are shown which allow users to interact with each other following an event of interlocking gaze. Methods are also shown which allows the operator of a personal point-of-interest beacon to control and limit interaction from the public thus ensuring a greater degree of privacy. Methods are also introduced which allow a mode of interaction to be determined by a remote service after mutual authorization of each interacting user. This disclosure also introduces a system of a head-mountable device which comprises a directional beacon transceiver system using beams in the infrared light spectrum enabling the detection of a gaze interlock event.

Abstract: To provide a novel electronic device. The electronic device includes a housing and a display device. The display device includes a first layer, a second layer, and a third layer. The first layer, the second layer, and the third layer are provided in different layers. The first layer includes a driver circuit and an arithmetic circuit. The second layer includes pixel circuits and a cell array. The third layer includes light-receiving devices and light-emitting devices. The pixel circuits each have a function of controlling light emission of the light-emitting device. The driver circuit has a function of controlling the pixel circuits. The arithmetic circuit has a function of performing arithmetic processing on the basis of first data corresponding to currents output from the light-receiving devices and second data corresponding to a potential held in the cell array.

Abstract: An assisted perception (AP) module comprises an attachment mechanism to attach the AP module to a helmet, and a housing to integrate modular components of the AP module. The housing comprises a front portion and side portion, the front portion located over an eye of the user. The modular components include sensors to collect information about an environment as sensor data, and processors located in the side portion. The processors execute one or more assisted perception engines that process the sensor data from the sensors into enhanced characterization data. Output devices electronically communicate the enhanced characterization data to a user, wherein at least one of the output devices protrudes from the front portion of the housing in front of an eye of the user.

Abstract: A productivity tool for an analytical laboratory is presented. The productivity tool provides aggregated operational status of all laboratory instruments within an analytical laboratory, even without access to their respective internal data stream (i.e., legacy and third party instruments) by determining operational status of a first group of instruments by capturing and processing internal data streams comprising data indicative of an operational status. The productivity tool also determines operational status of a second group of instruments by capturing and processing external surveillance stream(s) of one or more laboratory instrument(s) of the second group.

Abstract: A method of generating pictographic representations including receiving input content comprising textual information and a video item, the textual information being associated with the video item and aligned to a video playback timeline associated with the video item, obtaining summarization information describing the video item, selecting one or more elements from the textual information, comparing the selected elements of the textual data with the obtained summarization information, determining, based on the comparison, that one or more of the selected elements satisfy a specified comparison threshold, and identifying a pictographic representation for each of the one or more of the selected elements that are determined to satisfy the specified comparison threshold, the pictographic representations being selected from a pictographic representation set comprising a plurality of pictographic representations, and selectively outputting, with the video item, the identified pictographic representations aligned to

Abstract: A system and method are disclosed to generate mixed-reality visualizations and objects to facilitate efficient interaction with large volumes of data. Embodiments include a computer comprising a processor and memory and configured to select data stored in a data table coupled to the computer. Embodiments select one or more key process indicators (KPIs), dimensions by which to display the selected data, and color schemes by which to display the selected data. Embodiments generate a first mixed-reality object displaying the selected data according to the selected one or more KPIs, dimensions, and color schemes, and a second mixed-reality object displaying the selected data as a three-dimensional model of nodes and connections. Embodiments display the first and second mixed-reality objects on a display device, and manipulate one of the first and second mixed-reality objects in response to input received by one or more sensors.

Abstract: An information processing device according to an embodiment includes a control unit that performs: control of switching a display coordinate system of display content from a first display coordinate system to a second display coordinate system, the display content displayed on a surface of a real object by a display unit, depending on a state of an input operation of changing a position or an angle of the display content; and control of changing a display position or a display angle of the display content in accordance with the second display coordinate system and causing the display unit to display input assist display corresponding to axes of the second display coordinate system in a case where the display coordinate system is switched from the first display coordinate system to the second display coordinate system.

Abstract: The user interface of an electronic eyewear device tracks a user's head movement to keep the user interface in front of the user's eyes. A position forward vector, a rotation forward vector, and an up vector are defined to enable the system to individually adjust the rates of translation and rotation of the user interface as the user's head moves through space. By disconnecting the translation and rotation of the user interface from the head forward vector, the rates of translation and rotational movement of the user interface are separated from the rate of movement of the user's head to enable a delay in user interface movement relative to head movement for a more realistic VR/AR 3D presentation. The pitch angle is monitored to maintain the user interface in front of the user's eyes in the correct orientation even at difficult pitch angles such as straight up or straight down.

Abstract: In accordance with some embodiments, an exemplary process for dynamically controlling the size of a display based on a moving of a visual object meeting a criterion in a computer-generated reality (CGR) environment is described.

Abstract: Compact occlusion-capable optical see-through head mounted displays (OCOST-HMDs) are described having a double-wrapped path and capable of rendering per-pixel mutual occlusion, and correct see-through viewing perspective or a pupil-matched viewing between the virtual and real views. An example device includes a polarizer, a polarizing beam splitter, an objective lens, a spatial light modulator (SLM), an eyepiece lens, a quarter wave plate, and a reflective optical element configured to reflect the light that is incident thereupon in a first direction, and to transit the light received from a microdisplay that is incident thereupon from a second direction. The components form a first double-pass configurations that allow the light that passes through the objective to reflect from the SLM and propagate again through the objective, and a second double-pass configuration that allows the light that passes through the eyepiece to reflect from the reflective optical element and propagate again through the eyepiece.

Abstract: Techniques and systems are provided for providing recommendations for extended reality systems. In some examples, a system determines one or more environmental features associated with a real-world environment of an extended reality system. The system determines one or more user features associated with a user of the extended reality system. The system also outputs, based on the one or more environmental features and the one or more user features, a notification associated with at least one application supported by the extended reality system.

Abstract: Methods and systems for presenting digital content on a see-through display are disclosed. In some embodiments, the digital content is presented based on a display attribute associated with personal information related to a user of a wearable head device comprising the see-through display. In some embodiments, the digital content is associated with a geo-spatial location of a physical structure, and presenting the digital content on the see-through display comprises indicating a spatial relationship between the digital content and the physical structure.

Abstract: A display device 30 displays information of variables that reflect a state of a vehicle 10. The display device 10 includes: a first display region 31 that displays a scale of a first variable; a second display region 32 that is provided to be aligned with the first display region 31 and displays a scale of a second variable; a first pointing portion 42 that points, in the first display region 31, the first variable in accordance with the state of the vehicle 10; a first strip-shaped image 43 that is displayed between a pointed position of the first pointing portion 42 and a start point S1 of the first display region 31; a second pointing portion 52 that points, in the second display region 32, the second variable in accordance with the state of the vehicle 10; and a second strip-shaped image 53 that is displayed between a pointed position of the second pointing portion 52 and a start point S2 of the second display region.