Abstract: A system for providing customized content includes a recognition device. A media transmission device is in communication with a recognition device and a media delivery device. The media transmission device is in communication with a processor and a memory storing computer instructions configured to instruct the processor to receive a primary media presentation including primary media content for a point of interest of a venue. The media transmission device generates a secondary media presentation including secondary media content for the user based on identified user preferences for the user. The media transmission device generates a custom media presentation including custom media content for the user based on the identified user preferences. The media transmission device transmits the primary media presentation, the secondary media presentation, or the custom media presentation and presents the primary media content, the secondary media content, or the custom media content to the user.

Abstract: An example process includes: displaying, on a display of an electronic device, an extended reality (XR) environment corresponding to a copresence session including the electronic device and a second electronic device; while displaying the XR environment: sampling, with a microphone of the electronic device, a first audio input; determining whether the first audio input is intended for a first digital assistant operating on an external electronic device; and in accordance with a determination that the first audio input is intended for the first digital assistant: causing the first digital assistant to provide an audible response to the first audio input, where the audible response is not transmitted to the second electronic device over a shared communication channel for the copresence session.

Abstract: Embodiments of the invention provide a tracking performance evaluation method and a host. The method includes: establishing a virtual environment; obtaining a tracking result of a virtual tracking device executing a tracking function in the virtual environment; and providing a tracking performance evaluation result for the virtual environment based on the tracking result.

Abstract: The present disclosure provides systems and methods for calibration-free instant motion tracking useful, for example, for rending virtual content in augmented reality settings. In particular, a computing system can iteratively augment image frames that depict a scene to insert virtual content at an anchor region within the scene, including situations in which the anchor region moves relative to the scene. To do so, the computing system can estimate, for each of a number of sequential image frames: a rotation of an image capture system that captures the image frames; and a translation of the anchor region relative to an image capture system, thereby providing sufficient information to determine where and at what orientation to render the virtual content within the image frame.

Abstract: A computer-readable medium is configured to translate movements performed by a person onto a user in augmented or virtual reality has a headset display for wearing by the person, and a first controller and a second controller for holding by the person. Each of the first and second controllers are communicatively connectable to the headset display and including a thumbstick, a first button, a second button, a grip button, and a trigger button. The computer-readable medium is further configured to execute the computer-readable program code to enable the user in the augmented or virtual reality to walk forward by the person forwardly actuating the thumbstick of the first controller and turn by the person tilting left or right the thumbstick of the second controller or by the person physically turning left or right.

Abstract: In one embodiment, a method includes capturing images of a first user wearing a VR display device in a real-world environment. The method includes receiving a VR rendering of a VR environment. The VR rendering is from the perspective of the mobile computing device with respect to the VR display device. The method includes generating a first MR rendering of the first user in the VR environment. The first MR rendering of the first user is based on a compositing of the images of the first user and the VR rendering. The method includes receiving an indication of a user interaction with one or more elements of the VR environment in the first MR rendering. The method includes generating, in real-time responsive to the indication of the user interaction with the one or more elements, a second MR rendering of the first user in the VR environment. The one or more elements are modified according to the interaction.

Abstract: Provided are a defect detection method and apparatus, and a computer-readable storage medium. Specifically, the method includes: obtaining a to-be-detected image; obtaining a feature map of the to-be-detected image based on the to-be-detected image, where the feature map of the to-be-detected image includes a feature map of spatial position coordinate information; and performing defect detection on the to-be-detected image based on the feature map of the to-be-detected image. By modifying a neural network structure of defect detection and extracting the feature map of spatial position coordinate information during the detection, this application makes the neural network for use of defect detection sensitive to a spatial position, thereby enhancing sensitivity of a detection neural network to the spatial position, and in turn, increasing accuracy of detecting some specific defect types by the detection neural network, and increasing accuracy of defect detection.

Abstract: An augmented reality system to generate and cause display of a presentation of a space at a first client device, receive one or more selections of points within the presentation of the space at the first client device, and render graphical elements at the one or more points within the presentation of the space at the first client device. The augmented reality system is further configured to receive a display request to display the space at a second client device, and in response, may render a second presentation of the space at the second client device, wherein the second presentation of the space includes the graphical elements at the one or more points.

Abstract: Exosuit systems and methods according to various embodiments are described herein. The exosuit system can be a suit that is worn by a wearer on the outside of his or her body. It may be worn under the wearer's normal clothing, over their clothing, between layers of clothing, or may be the wearer's primary clothing itself. The exosuit may be assistive, as it physically assists the wearer in performing particular activities, or can provide other functionality such as communication to the wearer through physical expressions to the body, engagement of the environment, or capturing of information from the wearer.

Abstract: A first device coupled with a first display and an image sensor receives output data from a second device having a second display different from the first display. The output data represents content displayable by the second device on the second display. The first device determines, using the image sensor, a position of the second display relative to the first device and causes the first display to display content based on the output data received from the second device and the determined position of the second display relative to the first device.

Abstract: The present invention relates to a method for determining the spatial position of objects, in particular medical objects. First position data is acquired that describes a spatial position of an object in a first coordinate system. First transformation data is acquired that transforms the object's position from the first coordinate system to a second coordinate system. Based on the foregoing data, second position data is acquired that specifies the spatial position of the object in the second coordinate system. Second transformation data is acquired that transforms the object's position from the second coordinate system to an inertial coordinate system. Based on the second position data and the second transformation data, inertial position data is determined that specifies a position of the object in the inertial coordinate system.

Abstract: According to aspects herein, methods and systems for capacity management of virtual space are provided. The method begins with estimating a capacity size need of a virtual space in a metaverse. A first quantity of avatars is monitored as avatars enter, leave, and occupy the virtual space. The first quantity of avatars in the virtual space is compared with a predetermined fill threshold. Based on the comparing, a duplicate virtual space may be created if the first quantity of avatars in the virtual space exceeds the predetermined fill threshold. A user may transmit a request to join a virtual space in a metaverse to a network. The request to join the virtual space contains at least one avatar identification and may be a request for a group to join. The user or group receives a sequential identifier of the virtual space and then joins the virtual space.

Abstract: In one implementation, a method of transferring the display of video content from a first location to a second location is performed at a first device including an image sensor, a first display, one or more processors, and non-transitory memory. The method includes detecting a second device in a field-of-view of the image sensor displaying video content at a first location in the physical environment on a second display. The method includes detecting a transfer trigger to display the video content at a second location in the physical environment. The method includes, in response to detecting the transfer trigger, displaying, on the first display, a transfer animation of the video content being moved from the first location to the second location.

Abstract: An image processing apparatus includes an imaging unit that images a user and a real object, an analysis unit that analyzes an attitude of the real object based on imaging information captured by the imaging unit, and a control unit that controls display of an image related to the real object based on the attitude of the real object.

Abstract: Aspects of the present disclosure are directed to an artificial intelligence (“AI”) application running in conjunction with an artificial reality (“XR”) space. The AI Builder responds to user commands, verbal or gestural, to build or edit spaces or objects in space. If the requested object is of a type recognized by the AI Builder, then the AI Builder builds the object from one or more stored templates. The new object's location is determined by the objects that already exist in the user's XR environment and on commands or gestures from the user. If the AI Builder does not recognize the requested object, the user can show an image to the AI Builder, and the AI builds a 3D object in the XR space according to that image. To ease collaboration among users, the AI Builder may present its user interface as a non-player character within the XR world.

Abstract: Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

Abstract: A contact determination system includes a processor, and determines whether a contact event occurs between target objects in a vicinity of a host vehicle. The processor is configured to acquire, regarding each of the target objects including at least one of (i) a road user other than the host vehicle and (ii) a road-installed object, a type and a moving speed of the target object. The processor is also configured to perform an overlap determination regarding whether at least two target models respectively modeling the target objects overlap with each other. The processor is further configured to determine whether or not the contact event occurs for a pair of the target objects that have been determined to have an overlap based on the type and moving speed of the target object.

Abstract: A computer-implemented method includes obtaining 3D model of real-world environment; receiving image of real-world environment captured using camera and pose information indicative of camera pose from which image is captured; utilising 3D model of real-world environment to generate reconstructed depth map from perspective of camera pose; and applying extended depth-of-field correction to image segment(s) of image that is/are out of focus, by using point spread function determined for camera, based on optical depths in segment(s) of reconstructed depth map corresponding to image segment(s) of image.

Abstract: In some embodiments, the present disclosure includes techniques and user interfaces for interacting with virtual objects in an extended reality environment. In some embodiments, the techniques and user interfaces are for interacting with virtual objects in an extended reality environment, including repositioning virtual objects relative to the environment. In some embodiments, the techniques and user interfaces are for interacting with virtual objects, in an extended reality environment, including virtual objects that aid a user in navigating within the environment. In some embodiments, the techniques and user interfaces are for interacting with virtual objects, including objects displayed based on changes in a field-of-view of a user, in an extended reality environment, including repositioning virtual objects relative to the environment.

Abstract: In an example, a prescreening method includes: capturing an image of one or more items for a user; processing the image of the one or more items by image recognition; analyzing the image processed by image recognition to determine whether any of the one or more items are listed on a list; if none of the one or more items in the image is listed on the list, informing the user that none of the one or more items in the image is listed on the list; and if any of the one or more items in the image is listed on the list, identifying each item in the image which is listed on the list to the user.

Abstract: A system for treatment and/or psychology intervention of attention deficit disorders in a user in need thereof. The system uses a head mounted display equipped with an eye tracking device. The system is configured to display a scene that includes a principal object and at least one secondary object, then use the eye tracking device to determine whether the user's gaze is directed to one of the objects. If the user's gaze is directed to one of the secondary objects beyond a threshold criterion, that secondary object is removed from the scene. If the user's gaze is directed to the principal object beyond a threshold criterion, a previously removed secondary object is reintroduced into the scene. The principal object is a desired object of the user's attention, while the secondary object(s) may be potentially or known distracting objects. Corresponding software and methods are also disclosed.

Abstract: An information processing apparatus (2000) acquires a shelf rack image (12) in which a product shelf rack on which a product is displayed is imaged. The information processing apparatus (2000) performs image analysis on the shelf rack image (12), and generates information (actual display information) relevant to a display situation of the product on a product shelf rack (20). The information processing apparatus (2000) acquires reference display information representing a reference for display of the product on the product shelf rack (20). The information processing apparatus (2000) compares the actual display information generated by performing the image analysis on the shelf rack image (12) with the acquired reference display information, and generates comparison information representing a result.

Abstract: A processing system including at least one processor may identify a user viewpoint of a user at a first venue, match a viewpoint at a second venue to the user viewpoint of the user at the first venue, detect a trigger condition to provide visual content of the second venue to the user at the first venue, obtain the visual content of the second venue, wherein the visual content of the second venue is obtained from the viewpoint at the second venue, and provide the visual content of the second venue to an augmented reality device of the user at the first venue, where the augmented reality device presents the visual content of the second venue as a visual overlay within a field of view of the user.

Abstract: A method for avoiding a field of view disturbance for an operator of an object. The field of view disturbance IS caused by the overlay of virtual additional information via a display unit of a transportation vehicle or data glasses. The overlay of additional information supports the operator in the operation of an object. The method carries out an image analysis by which it is checked whether a field of view disturbance is caused by the overlay of the virtual additional information, and a measure for suppressing the field of view disturbance is carried out when a field of view disturbance is detected.

Abstract: Eye and hand tracking systems in head-mounted display (HMD) devices are arranged with lensless camera systems using optical masks as encoding elements that apply convolutions to optical images of body parts (e.g., eyes or hands) of HMD device users. The convolved body images are scrambled or coded representations that are captured by a sensor in the system, but are not human-recognizable. A machine learning system such as a neural network is configured to extract body features directly from the coded representation without performance of deconvolutions conventionally utilized to reconstruct the original body images in human-recognizable form. The extracted body features are utilized by the respective eye or hand tracking systems to output relevant tracking data for the user's eyes or hands which may be utilized by the HMD device to support various applications and user experiences. The lensless camera and machine learning system are jointly optimizable on an end-to-end basis.

Abstract: Various implementations disclosed herein include devices, systems, and methods for augmenting a physical writing surface. In various implementations, a device includes a display, a non-transitory memory and one or more processors coupled with the display and the non-transitory memory. In various implementations, a method includes presenting, via the display, a pass-through representation of a physical writing surface that corresponds to an application installed on the device. In some implementations, the method includes detecting a difference between the physical writing surface and an electronic record stored in association with the application. In some implementations, the method includes overlaying an element on the pass-through representation of the physical writing surface based on the difference between the physical writing surface and the electronic record.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for selectively rendering augmented reality content based on predictions regarding a user's ability to visually process the augmented reality content. For instance, the disclosed systems can identify eye tracking information for a user at an initial time. Moreover, the disclosed systems can predict a change in an ability of the user to visually process an augmented reality element at a future time based on the eye tracking information. Additionally, the disclosed systems can selectively render the augmented reality element at the future time based on the predicted change in the ability of the user to visually process the augmented reality element.

Abstract: Systems and methods described herein in accordance with some embodiments may display augmented reality (AR) information related to an object based on a predicted persistence of the object within view of the user. In some embodiments, display of AR information is prioritized based upon a prediction of how soon an object will disappear from view, perhaps if the predicted disappearance will occur sooner than some threshold time. Systems and methods described herein in accordance with some embodiments extend to mixed reality (MR) systems, and provide for displaying an image of an object that has become obstructed or has disappeared from view and/or for displaying AR information related to an object that has become obstructed or has disappeared from view.

Abstract: A remote-control system, a remote-controller, and a remote-control method are provided. The remote-control method includes: obtaining environmental image data by an image capture device of a remote-controller; building a map according to the environmental image data based on simultaneous localization and mapping (SLAM) algorithm, and obtaining first location information of a first display in the map according to the environmental image data by the remote-controller; and receiving the first location information from the remote-controller and controlling the first display according to the first location information by a computing device.

Abstract: A method includes obtaining a photographed image from a photographing device carried by a movable object, determining a target object in the photographed image to track the target object, determining location estimation information of the target object based on shooting information of the photographing device, and in response to a disappearance of the target object in the photographed image, controlling the photographing device to search for the target object around a location associated with the location estimation information.

Abstract: Augmented reality guidance for guiding a user through an environment using an eyewear device. The eyewear device includes a display system and a position detection system. A user is guided though an environment by monitoring a current position of the eyewear device within the environment, identifying marker positions within a threshold of the current position, the marker positions defined with respect to the environment and associated with guidance markers, registering the marker positions, generating overlay image including the guidance markers, and presenting the overlay image on a display of the eyewear device.

Abstract: Described are improved systems and methods for navigation and manipulation of interactable objects in a 3D mixed reality environment. Improved systems and methods are provided to implement physical manipulation for creation and placement of interactable objects, such as browser windows and wall hangings. A method includes receiving data indicating a selection of an interactable object contained within a first prism at the start of a user interaction. The method also includes receiving data indicating an end of the user interaction with the interactable object. The method further includes receiving data indicating a physical movement of the user corresponding to removing the interactable object from the first prism between the start and the end of the user interaction. Moreover, the method includes creating a second prism to contain the data associated with the interactable object at the end of the user interaction with the interactable object.

Abstract: Systems and methods are provided for displaying supplemental content for print media using augmented reality. A user profile for a user of an augmented reality device is determined. Content of the print media is searched to identify a first portion of the print media that matches the user profile and a second portion of the print media that does not match the user profile. Supplemental content is obtained based on content of the first portion of the print media. A display of the supplemental content is positioned over the second portion of the print media.

Abstract: A method represents an environment via a display unit arranged on a person and visible for the person as a display image within the scope of a simulation. The simulation is carried out in an interaction environment, wherein a number of actuatable interaction elements are arranged in the interaction environment. An interaction environment image capture, depicting the interaction environment, is created by use of a first image capturing unit arranged on the person or relative to the person. A position of the person is determined in the interaction environment and based on the position of the person an environment image is provided. An image mask is provided which depicts the individual interaction elements contained in the interaction environment image capture and is represented in the display image. The interaction environment image capture and the environment image are superimposed using the image mask and then displayed on the display unit.

Abstract: The disclosed technology provides for secure storage and usage of data at an electronic device. A device may include a hardened data manager that securely stores data associated with users of the device and/or another device. An application that does not have access to the securely stored data can still use the securely stored data by requesting that another system process at the device perform operations on the securely stored data and/or one or more updates to the securely stored data. For example, an application can request rendering of an avatar of a user. A rendering engine, separate from the application, can obtain securely stored base data from the hardened data manager, obtain updates to the securely stored base data, and render the avatar using the updates to the securely stored base data, without exposing the securely stored base data to the application.

Abstract: Methods and systems are described for extended and mixed reality experience adaptation, processing, and handover from one computing or processing entity to another. In response to a change in a condition impacting resource usage, delivery of content is transferred from one edge node to another. One or more changes in a viewing client, a first edge node, a second edge node, a communication network, and content signal a handover. Artificial intelligence systems, including neural networks, and models are trained and developed for improving the adaptation, processing, and handover. Related apparatuses, devices, techniques, and articles are also described.

Abstract: An image display method includes: obtaining a homography matrix between a first target plane and a second target plane according to the first target plane and the second target plane; obtaining a target displacement according to the homography matrix and an attitude; obtaining a target pose according to the target displacement, the target pose including a position and an attitude of a camera coordinate system of a current frame image in a world coordinate system; and displaying an AR image according to the target pose.

Abstract: Systems and methods for providing scene understanding can include obtaining a plurality of images, stitching images associated with the scene, detecting objects in the scene, and providing information associated with the objects in the scene. The systems and methods can include determining filter tags or query tags that can be selected to filter the plurality of objects, which can then be provided as information to the user to provide further insight on the scene. The information may be provided in an augmented-reality experience via text or other user-interface elements anchored to objects in the images.

Abstract: Systems, methods, and apparatus are provided for intelligent dynamic rendering of an augmented reality (AR) display. An AR device may capture an image of a physical environment. Feature extraction combined with deep learning may be used for object recognition and detection of changes in the environment. Contextual analysis of the environment based on the deep learning outputs may enable improved integration of AR rendering with the physical environment in real time. A BLE beacon feed may provide supplemental information regarding the physical environment. The beacon feed may be extracted, classified, and labeled based on user interests using machine learning algorithms. The beacon feed may be paired with the AR advice to incorporate customized location-based graphics and text into the AR display.

Abstract: A method for AR-guided depth estimation is described. The method includes identifying a virtual object rendered in a first frame that is generated based on a first pose of an augmented reality (AR) device, determining a second pose of the AR device, the second pose following the first pose, identifying an augmentation area in the second frame based on the virtual object rendered in the first frame, and the second pose, determining depth information for the augmentation area in the second frame, and rendering the virtual object in the second frame based on the depth information.

Abstract: An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Abstract: One or more embodiments of the invention is a mixed reality system for displaying a mixed-reality image on a display, the system including a mobile terminal device having the display and a photographing device. The system includes a plurality of feature point sets arranged in the prescribed real space. The mobile terminal device recognizes each of the feature point sets photographed by the photographing device, determines a viewpoint position of a virtual camera, in a virtual space, corresponding to a position and a photographing direction of the photographing device in the real space, based on arrangement position and posture information for each of the feature point sets and relative position and posture information of the mobile terminal device with respect to each of the feature point sets, and generates a mixed-reality image in which an image of the virtual object is superimposed on a photographed image of the real space.

Abstract: A system for generating interactive content for a user includes an environment identification unit operable to identify a virtual environment for interaction by a user, an element arrangement unit operable to generate an arrangement of a plurality of elements within the environment in dependence upon one or more parameters, a content modification unit operable to modify the virtual environment so as to include the plurality of elements in the generated arrangement, and a content rendering unit operable to render one or more images of the modified virtual environment.

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine a wrist measurement or watch band size using depth data captured by a depth sensor from one or more rotational orientations of the wrist. In some implementations, depth data captured by a depth sensor including at least two depth map images of a wrist from different angles is obtained. In some implementations, an output is generated based on inputting the depth data into a machine learning model, the output corresponding to circumference of the wrist or a watch band size of the wrist. Then, a watch band size recommendation is provided based on the output.

Abstract: An embodiment for augmented reality (AR)-based visualization of an accidental situation in a multi-machine environment is provided. The embodiment may include receiving real-time and historical data relating to an activity. The embodiment may also include identifying a context of the activity and at least one property of one or more objects associated with the activity. The embodiment may further include identifying one or more items of safety equipment used by a user. The embodiment may also include assigning a compatibility score to the user. The embodiment may further include in response to determining the compatibility score is not adequate, executing a digital twin simulation of an avatar of the user. The embodiment may also include creating a visual animation of a resultant state of the avatar of the user consistent with the executed digital twin simulation. The embodiment may further include displaying the visual animation to the user.

Abstract: A method for dynamic detection and presentation of obscured real-world objects in augmented or mixed reality virtual content may include an object of interest computer program executed by an electronic device: (1) receiving data for a virtual or augmented reality object to display on an image of a physical environment on a display of the electronic device; (2) receiving data for a physical object in the physical environment; (3) determining that the physical object is obscured by the virtual or augmented reality object; (4) generating a representation of the physical object; and (5) providing the representation of the physical object to a virtual reality or augmented reality computer program executed by the electronic device, wherein the virtual reality or augmented reality computer program is configured to display the physical object over the virtual or augmented reality object.

Abstract: The present technology relates to devices and systems for multidimensional data visualization and interaction in an augmented reality, virtual reality, or mixed reality environment. The disclosed embodiment provides a tool for a physician or other medical specialist to load and review medical scans in an AR/VR/MR environment, assisting medical diagnostics, surgical planning, medical education, or patient engagement.

Abstract: An augmented reality (AR) device for obtaining depth information of an object in a real world, includes: a gaze tracking sensor configured to obtain a gaze point by tracking a gaze direction of a user's eye; a depth sensor configured to obtain depth information of the object; a memory storing one or more instructions; and at least one processor configured to execute the one or more instructions to: determine a region of interest (ROI) confidence level indicating a degree to which at least one partial region within a field of view (FOV) of the AR device is predicted as an ROI, based on at least one of a moving speed, an acceleration, a fixation time, a fixation number, or a location of the gaze point; determine an ROI based on the ROI confidence level; and set a parameter for controlling an operation of the depth sensor to obtain depth information of the object within the ROI.

Abstract: Provided is a method of providing an interpretation result using visual information, and the method includes: acquiring a spatial domain image including line-of-sight information of a user and gaze position information in the spatial domain image; segmenting the acquired spatial domain image into a plurality of images; detecting text areas including text for each of the segmented images; generating text blocks, each of which is a text recognition result for each of the detected text areas, and determining the text block corresponding to the gaze position information; converting a first language included in the determined text block into a second language that is a target language; and providing the user with a conversion result of the second language.

Abstract: A system that includes a mobile application to create a virtual overlay over a field of view of a mobile device, to virtually measure athletic performance of an athlete performing a sports activity. The system is configured to enable virtual boundary-based measurements in order to objectively assess athletes based on scores calculated using the virtual measurements. Displayed to the user are virtual boundaries including virtual planes extending from one or more floor markers within a mobile camera field of view.