Abstract: The present invention relates to a computer-implemented medical method, a computer program and a medical system (1) for providing, in a medical environment, an AR-overlay that is projected into the field of view provided by an AR-device (4) as a function of the characteristics of a user's (5) view with respect to an observed object (6).

Abstract: In one embodiment, a method includes rendering a first output image comprising one or more augmented-reality (AR) objects for displays of an AR rendering device of an AR system associated with a first user. The method further includes accessing sensor signals associated with the first user. The one or more sensor signals may be captured by sensors of the AR system. The method further includes detecting a change in a context of the first user with respect to a real-world environment based on the sensor signals. The method further includes rendering a second output image comprising the AR objects for the displays of the AR rendering device. One or more of the AR objects may be adapted based on the detected change in the context of the first user.

Abstract: A near-eye display system including an optics module coupled with an electronics module having a controller. Wherein the optics module includes a planar waveguide operable to display virtual images, and a camera operable to capture pictures and videos. The planar waveguide coupled with the camera via the optics module, whereby a first view through the planar waveguide is oriented to be the same as a second view by the camera.

Abstract: An augmented reality (AR) gaming system: samples a first signal encoding data representative of an object's location in a game space, the first signal being, or originating from, a first sensor's signal that identifies the object; samples at least a second signal encoding data representative of the object's location in the game space, the at least a second signal being, or originating from, at least a second sensor's signal that identifies the object, where the first sensor is different from the at least a second sensor; tracks the location of the object in the game space based upon the data representative of the location of the object from both of the first, and the at least a second, signals; and causes visual graphics to be provided to an AR apparatus wearable by the player in response to the tracking of the location of the object in the game space.

Abstract: A patrol helmet and method for livestock and poultry farms is described herein. The patrol helmet is wirelessly connected to an online farm management system on a cloud platform. The patrol helmet includes a helmet body, a controller, an augmented reality (AR) display component, an infrared (IR) camera, and a visible light camera. The AR display component is secured at the bottom edge of the helmet body. Both the IR camera and the visible light camera are disposed on the helmet body. The IR camera, the visible light camera, and the AR display component each is electrically connected to the controller. The IR camera and the visible light camera are configured to photograph livestock or poultry in an area in front of a wearer to obtain a temperature and an image of the livestock or the poultry.

Abstract: This application discloses a method for controlling a plurality of virtual characters in an application with an augmented reality (AR) function running in the a terminal. The method includes: displaying a first user interface of the application, the first user interface comprising a plurality of virtual characters for user selection; receiving a first user selection operation on at least two virtual characters on the first user interface; displaying a second user interface of the application, the second user interface including a background picture of a real world and the at least two virtual characters located in the background picture; receiving a second user selection operation on the second user interface; and determining a target virtual character from the at least two virtual characters according to the second user selection operation and the rendering sequence.

Abstract: An operative target position of a leading end device supposed by an operator of a working machine is estimated with high accuracy. The controller calculates a prospective locus region on the basis of respective detection results of a posture detecting part and a manipulation detecting part. The prospective locus region is a region based on a prospective locus along which a leading end device is prospected to shift. The controller calculates a gaze point region on the basis of a detection result of a sight detecting part. The gaze point region is a region based on a gaze point of an operator. The controller sets a target position in a region where the prospective locus region and the gaze point region overlap each other.

Abstract: A virtual object display device comprises a display and a display control device configured to perform display control of the display. The display control device includes: a coordinate system calculation unit configured to detect movement and rotation of the virtual object display device in a real world, and use an inertial coordinate system in which a coordinate origin thereof follows the movement of the virtual object display device and an effective field of view of the display is rotated therein in accordance with the rotation of the virtual object display device so as to define an arrangement position of an inertial coordinate system virtual object; and a display control unit configured to, when the inertial coordinate system virtual object is included in the effective field of view of the display, display the inertial coordinate system virtual object within the effective field of view.

Abstract: A computer-based extended reality health system is adapted for electronically rendering patient sessions within an XR environment incorporating immersive scenes. The XR health system utilizes an XR device adapted for being employed by a patient to visually display the immersive scenes during a patient session within the XR environment. An input device is adapted for electronically interacting with objects, content and features visually displayed in the immersive scenes. A billing module is configured to extract claims data applicable for medical billing claims. The claims data includes billing information associated with the immersive scenes visually rendered to the patient during the patient session within the XR environment.

Abstract: A tracked device may be used in an extended reality system in coordination with a tracking device. The tracked device may be ordinarily difficult to track, for example due to changing appearances or relatively small surface areas of unchanging features, as may be the case with an electronic device with a relatively large display surrounded by a thin physical outer boundary. In these cases, the tracked device may periodically present an image to the tracking device that the tracking device stores as an indication to permit tracking of a known, unchanging feature despite the image not being presented continuously on the display of the tracked device. The image may include a static image, designated tracking data overlaid on an image frame otherwise scheduled for presentation, or extracted image features from the image frame otherwise scheduled for presentation. Additional power saving methods and known marker generation methods are also described.

Abstract: Devices, systems, and methods for augmented reality in surgical environments include an optical lens, which may be wearable by the user, and an augmented reality control system for generating an augmented reality presentation in the optical lens. The augmented reality presentation can depict the surgical area of the patient while artificially omitting one or more obstructions from the user's view point of the surgical area.

Abstract: A system for and method of projecting augmentation imagery in a head-mounted display is disclosed. A system for projecting light onto an eye includes a display to project light, a beam combiner, first and second optical systems between the display and the beam combiner along respective first and second optical paths. The first and second optical paths differ. The system also includes a switchable reflector that, in a reflective state, reflects light incident upon the reflector, and, in a non-reflective state, transmits light incident upon the reflector. The reflector is between the display and the first and second optical systems along the first and second optical paths and directs light along the first path, in the reflective state, or along the second path, in the non-reflective state, to reflect light from the beam combiner to the eye from different directions when in the different states.

Abstract: A display pose of a client computing device is computed in a hybrid manner using both the client computing device and a server. Using the client computing device, an initial pose of the client computing device at an initial time is computed. A prospective pose at a target time is predicted using the client computing device. The prospective pose is sent along with auxiliary data from the client computing device to the server and a display pose of the client computing device at a display time is computed. The display pose is computed using the prospective pose and the auxiliary data.

Abstract: A system and method is provided that provides for the transfer of the execution of content from a user device to an external device for output of the content by the external device. External devices may be detected in a physical space, and identified based on previous connection with the user device, based on a shared network or shared system of connected devices including the user device, based on image information captured by the user device and previously stored anchoring information that identifies the external devices, and the like. An external device may be selected for potential output of the content based on previously stored configuration information associated with the external device including, for example, output capabilities associated with the external device. The identified external device may output the transferred content in response to a user verification input, verifying that the content is to be output by the external device.

Abstract: Systems, computer program products, and methods are described herein for implementing continuous authentication based on object location recognition. The present invention is configured to receive, from the computing device of a user, a request for primary authorized access; receive, via the augmented reality application, a real-time visual feed, wherein the real-time visual feed comprises one or more objects; determine one or more positional parameters associated with the one or more objects; determine that the one or more objects and the one or more positional parameters associated with the one or more objects meets one or more requirements for the primary authorized access; and transmit control signals configured to cause the computing device of the user to display a notification indicating that the user has been granted the primary authorized access.

Abstract: An integrated circuit is presented. The integrated circuit has a set of sensor input interfaces and an output interface. The integrated circuit further has a memory with a first and second memory locations. The integrated circuit further has a multi-chip hub module which has a transaction buffer with both a real-time and a non-real-time buffer. The multi-chip hub module has a context mapper, a re-formatter module and an error handling module. The context mapper is configured to map data to the first or second memory location. The multi-chip hub module is configured to process data through the modules and provide processed data to the output interface.

Abstract: Examples of the disclosure describe systems and methods for generating and displaying a virtual companion. In an example method, a first input from an environment of a user is received at a first time via a first sensor on a head-wearable device. An occurrence of an event in the environment is determined based on the first input. A second input from the user is received via a second sensor on the head-wearable device, and an emotional reaction of the user is identified based on the second input. An association is determined between the emotional reaction and the event. A view of the environment is presented at a second time later than the first time via a see-through display of the head-wearable device. A stimulus is presented at the second time via a virtual companion displayed via the see-through display, wherein the stimulus is determined based on the determined association between the emotional reaction and the event.

Abstract: Apparatus, systems and methods for providing a four-part, virtual reality (VR) customer service interaction between a customer service avatar operating on a VR platform and a customer using both a mobile device and a VR device are provided. Methods may include activating a VR session between the customer service avatar and the customer. Methods may include validating the VR session between the customer service avatar and the customer. Methods may include processing a co-browsing VR session request. Methods may include initiating the VR session between the customer service avatar and the customer.

Abstract: A method for performing voice dictation with an earpiece worn by a user includes receiving as input to the earpiece voice sound information from the user at one or more microphones of the earpiece, receiving as input to the earpiece user control information from one or more sensors within the earpiece independent from the one or more microphones of the earpiece, inserting a machine-generated transcription of the voice sound information from the user into a user input area associated with an application executing on a computing device and manipulating the application executing on the computing device based on the user control information.

Abstract: Certain embodiments of the present disclosure provide techniques for eliminating an appearance of one or more objects in an environment surrounding a vehicle during operation of the vehicle. A method generally includes detecting the one or more objects in first sensor data collected from first sensors onboard the vehicle, wherein the first sensor data is representative of the environment surrounding the vehicle; receiving, from other vehicles, second sensor data collected from second sensors onboard the other vehicles, wherein the second sensor data is representative of the environment surrounding each of the one or more other vehicles; generating one or more augmented reality images depicting portions of the environment obstructed by the one or more objects in the first sensor data; displaying the one or more augmented reality images in an augmented reality display such that the augmented reality images are positioned to overlay the one or more objects.

Abstract: A head-mounted device may have a head-mounted support structure with a left side portion, a right side portion, and a transparent front portion forming a lens. The lens may extend across the front of a user's face between the left and right side portions. The lens may have a mirror coating or other optical coating that helps obscure objects on the inner side of the lens when viewed from an exterior region surrounding the head-mounted device. Left and right forward-facing cameras with overlapping fields of view may be used to capture visible-light images through the lens. The cameras may also be used for gaze tracking. Left and right light sources may provide eye illumination that reflects from the lens into left and right eye boxes. Images from the eye boxes may reflect from the lens towards the left and right forward-facing cameras.

Abstract: The present technology relates to artificial reality systems. Such systems provide projections a user can create to specify object interactions. For example, when a user wishes to interact with an object outside her immediate reach, she can use a projection to select, move, or otherwise interact with the distant object. The present technology also includes object selection techniques for identifying and disambiguating between objects, allowing a user to select objects both near and distant from the user. Yet further aspects of the present technology include techniques for interpreting various bimanual (two-handed) gestures for interacting with objects. The present technology further includes a model for differentiating between global and local modes for, e.g., providing different input modalities or interpretations of user gestures.

Abstract: A method for assessing a camera calibration. In the method, a first quality measure is ascertained, a systematic error having to be assessed using the first quality measure, the assessment being carried out with respect to errors remaining after a calibration, an overall calibration object including at least one calibration object being virtually segmented into calibration objects, a detector noise being estimated for each calibration object, which are combined to form an overall estimate, which is compared with an estimate of the detector noise of the overall calibration object.

Abstract: Data characterizing a relative arrangement of vehicle occupants with respect to one another are continuously transmitted to display devices worn on the heads of the vehicle occupants. Virtual environments are displayed as a function of these data.

Abstract: An object of the present disclosure is to enable a warning to be issued to a person who is highly likely to be put in danger at a risky point. In an information processing apparatus, a controller acquires information indicating a risky point, acquires a state of a person with a predetermined positional relationship to the risky point, and determines whether or not the state of the person is a state where there is a high possibility of being put in danger, among a plurality of states of the person assumable at the risky point. Furthermore, in a case where the state of the person is determined to be the state where there is a high possibility of being put in danger, the controller causes an output unit to output information indicating a warning.

Abstract: A method for dynamically initializing a 3 degrees of freedom (3DOF) tracking device is described. In one aspect, the method includes accessing a gyroscope signal from a gyroscope of the 3DOF tracking device, accessing an accelerometer signal from an accelerometer of the 3DOF tracking device, determining an initial state includes a combination of an initial orientation, an initial position, and an initial velocity of the 3DOF tracking device, the initial state indicating a starting condition of the 3DOF tracking device, integrating the gyroscope signal and the accelerometer signal to obtain orientation and position signals using the initial state, and refining an inclination signal of the orientation signal using the position signal.

Abstract: Disclosed herein are techniques for providing secondary content within primary content via an adaptive inset window. Adaptations of the adaptive inset window may include specifying an activation status, size, shape, and/or location of an adaptive inset window to be presented in the primary content. The adaptation may be defined based upon a salient region identified based upon object within the primary content.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving a request from a client device of a first user to engage in a shared augmented reality (AR) shopping experience with a second user; generating, for display on respective client devices of the first and second users, the shared AR shopping experience comprising a plurality of AR items that represent real-world products overlaid on a shared real-world environment of the first and second users; determining that the second user is interacting with a first AR item of the plurality of AR items; and in response to detecting that the second user is depicted in an image captured by the client device of the first user, overlaying the first AR item on the second user in the image.

Abstract: A method for displaying infrastructure information on a multi-focal plane augmented reality display of a vehicle includes receiving infrastructure data. The infrastructure data includes information about a location of at least one infrastructure along a route of the vehicle. The method further includes receiving vehicle-location data. The vehicle-location data includes information about a location of the vehicle. The method further includes determining a position of the vehicle relative to the location of the least one infrastructure using the infrastructure data and the vehicle-location data. The method further includes transmitting a command signal to the multi-focal plane augmented reality display to display a virtual image showing the infrastructure information of the infrastructure on the multi-focal plane augmented reality display.

Abstract: Interactive systems using adaptive control software and hardware from known and later developed eyepieces to later developed head-wear to lenses, including implantable, temporarily insertable and contact and related film based types of lenses including thin film transparent elements for housing cameras lenses and projector and functional equivalent processing tools. Simple controls, real-time updates and instant feedback allow implicit optimization of a universal model while managing complexity.

Abstract: A method for performing interactive operation upon a stereoscopic image and a stereoscopic image display system are provided. The stereoscopic image display system includes a stereoscopic display and a gesture sensor. In the method, the stereoscopic display displays the stereoscopic image, and the gesture sensor senses a gesture. A current gesture state is obtained. A previous state of the stereoscopic image and a previous gesture state are obtained. Stereo coordinate variations corresponding to the gesture can be calculated according to the current gesture state and the previous gesture state. New stereoscopic image data can be obtained according to the previous state of the stereoscopic image and the stereo coordinate variations corresponding to the gesture. The stereoscopic display is used to display a new stereoscopic image that is rendered from the new stereoscopic image data.

Abstract: This disclosure describes how to identify objects in an augmented reality environment. More specifically, the various systems and methods described herein describe how an augmented reality device can recognize objects within a real world environment, determine where the object is located, and also identify the various surfaces of the object in real time or substantially real time.

Abstract: A virtual image generation system comprises a planar optical waveguide having opposing first and second faces, an in-coupling (IC) element configured for optically coupling a collimated light beam from an image projection assembly into the planar optical waveguide as an in-coupled light beam, a first orthogonal pupil expansion (OPE) element associated with the first face of the planar optical waveguide for splitting the in-coupled light beam into a first set of orthogonal light beamlets, a second orthogonal pupil expansion (OPE) element associated with the second face of the planar optical waveguide for splitting the in-coupled light beam into a second set of orthogonal light beamlets, and an exit pupil expansion (EPE) element associated with the planar optical waveguide for splitting the first and second sets of orthogonal light beamlets into an array of out-coupled light beamlets that exit the planar optical waveguide.

Abstract: Usage metrics for augmented reality content may be identified and analyzed to determine measures of fitness for respective usage metrics. The measures of fitness may indicate a level of correlation with an outcome specified by an augmented reality content creator and an amount of interaction with an augmented reality content item by users of a client application. Recommendations may be provided to augmented reality content creators indicating modifications to augmented reality content items that have at least a threshold probability of increasing the level of interaction between users of the client application and the augmented reality content item.

Abstract: This disclosure enables various technologies for augmented-reality, which allow a person to obtain (e.g., electronically reserve, electronically purchase, physically receive) a memorabilia item during an event at or remote from a venue in which a performer is performing or within a preset time period after the event (e.g., within few hours after the event) at or remote from the venue. These technologies may reduce a rise in counterfeits of the memorabilia item, while minimizing a price increase of the memorabilia item.

Abstract: A head-mounted display (HMD) device is provided. The HMD device includes a transparent member, a display, at least one sensor, and at least one processor functionally connected to the display and the at least one sensor. The at least one processor is configured to obtain information on a surrounding environment of the HMD device via the at least one sensor, set a focus space generated based on a designated location, determine at least one first object positioned inside the focus space and at least one second object positioned outside the focus space, based on the information on the surrounding environment and the focus space, and display, via the display, a virtual image in an area of the transparent member, wherein the at least one second object is seen in the area.

Abstract: Provided is a display device including a display configured to display first content; a memory storing instructions; and a processor configured to execute the instructions to: obtain first content information by analyzing the first content, obtain user tracking information by tracking a user viewing the first content displayed by the display, determine whether a trigger condition is satisfied, based on at least one of the first content information or the user tracking information, and based on the trigger condition being satisfied, control a communication unit to transmit second content to a projector connected to the display device, wherein the second content is generated based on the first content.

Abstract: Embodiments include systems and methods that may include receiving situational information triggering a need to alert the driver and in response to the need to alert the driver presenting an alert to the driver based on driver state information selected based on a likelihood the driver will be more receptive to the alert modality than others of the plurality of alert modalities. The likelihood the driver will be receptive to the alert modality may be based on the received situational information, the received driver state information, and a historical behaviour record. The historical behaviour record may correlate the driver's reaction or the reaction of other drivers to previous alerts presented to the driver or other drivers with similar driver state information and similar situational information triggering presentation of previous alerts. The system may learn from the reaction of the driver and/or other drivers to better select suitable alert modalities.

Abstract: Systems, apparatus, and methods of rendering content based on a control point of a camera device are disclosed. In an example, a marker is attached to the camera device and its pose is tracked over time. Based on a camera model of the camera device, an offset between the marker and the control point is determined. The tracked pose of the marker can be translated and/or rotated according to the offset to estimate a pose of the control point. The rendering of the content is adjusted over time based on the estimated poses of the control point. Upon presentation of the content in a real-world space (e.g., on a display assembly located therein), the camera device can capture the content along with other objects of the real-world space and generate a video stream thereof.

Abstract: A camera is used to capture image data of representations of a physical environment. Planes and surfaces are determined from a representation. The planes and the surfaces are analyzed using relationships there between to obtain shapes and depth information for available spaces within the physical environment. Locations of the camera with respect to the physical environment are determined. The shapes and the depth information are analyzed using a trained neural network to determine items fitting the available spaces. A live camera view is overlaid with a selection from the items to provide an augmented reality (AR) view of the physical environment from an individual location of the locations. The AR view is enabled so that a user can port to a different location than the individual location by an input received to the AR view while the selection from the items remains anchored to the individual location.

Abstract: Augmented reality (AR) systems and methods involve an interactive head-mounted device (HMD), an external display, and a medical image computer, which is in communication with the HMD and the external display. The external display displays one or more planes of a medical image or a 3D model provided by the medical image computer. A user wearing the HMD may manipulate a medical image or 3D model displayed on the external display by focusing the user's gaze on a control object and/or a portion of a medical image or 3D model displayed on a display of the interactive HMD.

Abstract: A computer system displays a first view of a three-dimensional environment, including a first user interface object having a first surface at a first position in the three-dimensional environment. While displaying the first view of the three-dimensional environment, the computer system detects a change in biometric data of a first user, and in response, changes an appearance of the first surface in the first user interface object in accordance with the change in biometric data of the first user. While displaying the first user interface object with the appearance that has been changed based on the change in the biometric data of the first user, the computer system detects first movement of the first user, and in response, changes the appearance of the first user interface object in accordance with the first movement of the first user.

Abstract: Disclosed herein are various embodiments of systems and methods for evaluating a device and presenting a price quote for the device by using an optical character recognition (OCR) system and/or other associated devices and systems. The system can utilize a wireless charger to obtain information about the device while charging the device. The information can include the make of the device. The system can guide the user to navigate the device to provide additional information such as model, memory capacity, unique identifier, and carrier associated with the device, which can be extracted from the device using OCR. Based on, e.g., the unique identifier, the system can determine whether the device has been stolen. If the device has not been stolen, the system can determine a price based on information obtained from the device and can present the price to the user. If the user accepts the price, the system can facilitate purchase of the device.

Abstract: Disclosed are a head-mounted display system and a manned device. The head-mounted display system comprises: a head-mounted display apparatus, a touch device and a manned terminal apparatus, wherein the touch device is arranged on the manned device and is in communication connection with the manned terminal apparatus, the manned terminal apparatus is arranged on the manned device and comprises a power supply module and a processor module, and the head-mounted display apparatus comprises at least one display screen, is in communication connection with the manned terminal apparatus, and is configured to execute a control operation corresponding to a touch operation performed on the touch device by a passenger user of the manned device. The head-mounted display system enables the passenger to watch the screen content without maintaining a certain posture for a long time, and improves the personalized experience and privacy experience of the passenger.

Abstract: An immersive experience system provides interactive content to a user while isolating the user from external stimuli in an isolation chamber. The user floats in a high-density suspension liquid maintained at the user's body temperature while consuming the interactive content which can provide audio, video and tactile inputs to the user. The user can interact with the immersive experience system via different modalities such as eye movements or gestures or via providing thought input through a visual cortex thought recorder comprised in the immersive experience system.

Abstract: One example provides a computing device configured to capture, via the camera, first image data imaging a first physical world location, create a first spatial representation of the first physical world location based on the first image data, receive a user input defining a pose of a first virtual spatial anchor point relative to a feature imaged in the first image data, track user movement to a second physical world location, capture second image data imaging the second physical world location, receive a user input defining a pose of a second virtual spatial anchor point relative to a feature imaged in the second image data, and send, to a remote computing device, data representing the first spatial representation, the pose of first spatial anchor point, the second spatial representation, the pose of second spatial anchor point, and a positional relationship between first and second spatial anchor points.

Abstract: Methods, systems, and apparatus for controlling smart devices are described. In one aspect a method includes receiving image data for an image captured by a camera of a mobile device of a user and determining that the image depicts at least one of a smart device or a physical control for the smart device. In response to determining that that the image depicts a smart device or a physical control for the smart device, identifying one or more user interface controls for controlling the smart device, and generating and presenting, at a display of the mobile device, the one or more user interface controls for controlling the smart device. The method can further include detecting, at the display of the mobile device, user interaction with at least one of the one or more user interface controls, and controlling the smart device based on the detected user interaction.

Abstract: An illumination rendering method and apparatus includes obtaining a first picture at a target viewing angle from a virtual three-dimensional (3D) scene. The first picture includes a virtual object to be subject to illumination rendering in the virtual 3D scene at the target viewing angle. A target virtual light source point set is determined that performs illumination rendering on the virtual object in the first picture. Illumination rendering is performed on the virtual object in the first picture by using the target virtual light source point set. This illumination rendering improves efficiency in rendering on the virtual object in the virtual 3D scene.

Abstract: An augmented reality device includes an eyeglass frame and a combiner mounted on the eyeglass frame. The combiner includes an inner surface and an outer surface disposed opposite the inner surface. The device further includes an active shutter lens mounted on the combiner and an image projector mounted on the eyeglass frame and configured to project display light to the combiner such that a first portion of the display light is emitted from the inner surface of the combiner and a second portion of the display light is emitted from the outer surface of the combiner. The device additionally includes a processor coupled to the image projector and the active shutter lens. The active shutter lens is configured to shield the display light emitted from the outer surface of the combiner. The combiner is configured to emit ambient light from the inner surface thereof.

Abstract: The present specification relates to image capture. More specifically, it relates to selective image capture for sensor carrying devices or floats deployed, for example, on the open sea. In one form, data is generated on the sensor carrying devices or floats by an on-board Inertial Measurement Unit (IMU) and is used to automatically predict the wave motion of the sea. These predictions are then used to determine an acceptable set of motion parameters that are used to trigger the on-board camera(s). The camera(s) then capture images. One consideration is that images captured at or near the peak of a wave crest with minimal pitch and roll will contain fewer obstructions (such as other waves). Such images provide a view further into the horizon to, for example, monitor maritime sea traffic and other phenomenon. Therefore, the likelihood of capturing interesting objects such as ships, boats, garbage, birds, . . . etc. is increased.