Abstract: Systems, methods, and non-transitory media are provided for generating private control interfaces for extended reality (XR) experiences. An example method can include determining a pose of an XR device within a mapped scene of a physical environment associated with the XR device; detecting a private region in the physical environment and a location of the private region relative to the pose of the XR device, the private region including an area estimated to be within a field of view (FOV) of a user of the XR device and out of a FOV of a person in the physical environment, a recording device in the physical environment, and/or an object in the physical environment; based on the pose of the XR device and the location of the private region, mapping a virtual private control interface to the private region; and rendering the virtual private control interface within the private region.

Abstract: Usage metrics for augmented reality content may be identified and analyzed to determine measures of fitness for respective usage metrics. The measures of fitness may indicate a level of correlation with an outcome specified by an augmented reality content creator and an amount of interaction with an augmented reality content item by users of a client application. Recommendations may be provided to augmented reality content creators indicating modifications to augmented reality content items that have at least a threshold probability of increasing the level of interaction between users of the client application and the augmented reality content item.

Abstract: Systems, methods and techniques for automatically recognizing two-dimensional real world objects with an augmented reality display device, and augmenting or enhancing the display of such real world objects by superimposing virtual images such as a still or video advertisement, a story or other virtual image presentation. In non-limiting embodiments, the real world object includes visible features including visible security features and a recognition process takes the visible security features into account when recognizing the object and/or displaying superimposed virtual images.

Abstract: Methods and systems are disclosed for displaying an augmented reality virtual object on a multimedia device. One method comprises detecting, in an augmented reality environment displayed using a first device, a virtual object; detecting, within the augmented reality environment, a second device, the second device comprising a physical multimedia device; and generating, at the second device, a display comprising a representation of the virtual object.

Abstract: A head-worn device system includes one or more cameras, one or more display devices and one or more processors. The system also includes a memory storing instructions that, when executed by the one or more processors, configure the system to generate a virtual object, generate a virtual object collider for the virtual object, determine a conic collider for the virtual object, provide the virtual object to a user, detect a landmark on the user's hand in the real-world, generate a landmark collider for the landmark, and determine a selection of the first virtual object by the user based on detecting a collision between the landmark collider with the conic collider and with the virtual object collider.

Abstract: An information distribution apparatus includes an imaging unit configured to capture a captured image including a specific object, an object image acquisition unit configured to acquire an object image of the specific object from the captured image, and a try-on data generation unit configured to generate try-on data based on a registration avatar in a virtual space and the object image.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for selectively rendering augmented reality content based on predictions regarding a user's ability to visually process the augmented reality content. For instance, the disclosed systems can identify eye tracking information for a user at an initial time. Moreover, the disclosed systems can predict a change in an ability of the user to visually process an augmented reality element at a future time based on the eye tracking information. Additionally, the disclosed systems can selectively render the augmented reality element at the future time based on the predicted change in the ability of the user to visually process the augmented reality element.

Abstract: Systems and methods for detecting data acquisition conditions using color-based penalties can include a computing device obtaining a sequence of images acquired by a photodetector. The computing device can determine, for each pixel position of a plurality of pixel positions associated with the sequence of images, a respective penalty score indicative of a similarity between a color value of a pixel of the pixel position and a desired color value. The desired color value can represent a color property of light emitted from body parts of users when placed opposite to the photodetector. The computing device can determine, using penalty scores of the plurality of pixel positions, a relative position of a body part of a user with respect to a desired position.

Abstract: An image processing and displaying method, AR device, image processing device, and a display system are provided. The image processing and displaying method is applied to a display system, wherein the display system includes an Augmented Reality (AR) device and an image processing device, and the image processing and displaying method includes: acquiring, by the AR device, a real-time image of a current view field of a user wearing the AR device in a real environment; sending, by the AR device, the real-time image to the image processing device; receiving, by the AR device, a display direction, an imaging distance, and a target image sent by the image processing device, and displaying, by the AR device, the target image in a virtual display area according to the display direction and the imaging distance.

Abstract: An information providing apparatus includes an acquisition section that acquires position information from a terminal device, a specifying section that specifies a target publication from plural publications on which maps have been posted, by performing processing with the position information, and a providing section that provides the terminal device with information indicating a correspondence relationship between a map posted on the target publication and a position indicated by the position information.

Abstract: According to an embodiment, a display device comprises a frame defining an opening, a grip extending from the frame, and a display panel which is held by the frame and whose background is visually recognizable. Further, the opening includes a region surrounded by an inner edge of the frame and the display panel.

Abstract: The disclosed system may include a housing dimensioned to secure various components including at least one physical processor and various sensors. The system may also include a camera mounted to the housing, as well as physical memory with computer-executable instructions that, when executed by the physical processor, cause the physical processor to: acquire images of a surrounding environment using the camera mounted to the housing, identify features of the surrounding environment from the acquired images, generate a map using the features identified from the acquired images, access sensor data generated by the sensors, and determine a current pose of the system in the surrounding environment based on the features in the generated map and the accessed sensor data. Various other methods, apparatuses, and computer-readable media are also disclosed.

Abstract: A virtual local presence display apparatus, system and method is disclosed.

Abstract: Systems, methods, and non-transitory computer readable media containing instructions for causing at least one processor to perform operations for changing display of virtual content based on temperature are provided. In one implementation, the processor is configured to perform operations comprising displaying virtual content via a wearable extended reality appliance, wherein during displaying of the virtual content, heat is generated by at least one component of the wearable extended reality appliance; receiving information indicative of a temperature associated with the wearable extended reality appliance; determining a need to change display settings of the virtual content based on the received information; and based on the determination, changing the display settings of the virtual content to achieve a target temperature.

Abstract: An illustrated view of an absolute teaching device for presenting an image of the patient's medical information to the doctor. The absolute teaching device is useful to aid to educate the patient or to close a mode of a treatment plan. The absolute teaching device removes the blood and gut, that most non-healthcare individuals can be squeamish about by animating with transparencies to see from frontal to supine. The absolute teaching device may highlight, or glow, for areas of concern during a consultation and direct any other areas of concern related to the origin of concern.

Abstract: An interactive troubleshooting assistant and method for troubleshooting a system in real time to repair (fix) one or more problems in a system is disclosed. The interactive troubleshooting assistant and method may include receiving multimodal inputs from sensors, wearable devices, a person, etc. that may be input into a feature extractor including attention layers and pre-processing units of a cloud computing system hosted by one or more servers, such as a private cloud system. A pre-processing unit converts the raw multimodal input into a structed form so that an attention layer can give weights to features provided by the pre-processing unit according to their importance. The weighted extracted features may be provided to an actions predictor. The actions predictor generates the most suitable action based on the weighted extracted features generated by the feature extractor based on the multimodal inputs.

Abstract: A head-mounted device (HMD) of a first user has a transparent display. The HMD determines location information of a second user relative to the HMD of the first user. The second user is located within a predefined distance of the HMD. The location information identifies a distance and a direction of the second user relative to the HMD. The HMD receives audio content from the second user, generates augmented reality (AR) content based on the audio content, and displays the AR content in the transparent display based on the location information of the second user. The AR content appears coupled to the second user.

Abstract: A method for visualizing an image combining an image (Ic) of a real object (200) originating from a video capture system (300) with digital information (In) originating from a three-dimensional model of the equipment, comprising: carrying out a processing operation to superimpose, in real time, a reference point (402) of the three-dimensional model with a reference point (302) of the video capture system and an object reference point (202) of the real object, and displaying at least some of the digital information superimposed on the image captured by the video capture system, further comprising: an initial step (Ei) of recording the reference texture (T200) of the real object, and a step (Ea) of analyzing the images transmitted by the video capture system, the analysis step comprising: generating a synthesis image from the captured image, and from the three-dimensional model of the equipment textured using the recorded texture; a step of calculating a composite image by mixing the synthesis image and the cap

Abstract: In one embodiment, a method includes rendering, for one or more displays of a VR display device, a first output image of a VR environment. The VR environment includes a personal UI associated with a first user. The personal UI has a first form factor and a first pose with respect to the first user, is a virtual object comprising one or more 2D virtual displays, and is operable to execute a plurality of applications. The method includes determining whether to adapt the personal UI based on a detected change in a context of the first user with respect to the VR environment. The method includes rendering a second output image of the VR environment, where the personal UI is adapted to have a second form factor and a second pose with respect to the first user responsive to determining to adapt the personal UI based on the detected change in the context of the first user.

Abstract: A system and method for a master platform includes receiving first pose data associated with an image sensor of a first device, and a first semantic map generated by the first device, the first semantic map including a simplified object representation in a coordinate space of the first device. The master platform also receives second pose data associated with an image sensor of a second device, and a second semantic map generated by the second device, the second semantic map including a simplified object representation in a coordinate space of the second device. A shared simplified object representation common to the first and semantic maps is identified. The master platform further combines the first semantic and second semantic maps based on the first and second pose data. The first pose data, first semantic map, second pose data, and second semantic map are associated with a common time interval.

Abstract: Various embodiments, methods and systems for implementing a distributed computing system scene assembly engine are provided. Initially, a selection of a first synthetic data asset and a selection of a second synthetic data asset are received from a distributed synthetic data as a service (SDaaS) integrated development environment (IDE). A synthetic data asset is associated with asset-variation parameters and scene-variation parameters, the asset-variation parameters and scene-variation parameters are programmable for machine-learning. Values for generating a synthetic data scene are received. The values correspond to asset-variation parameters or scene-variation parameters. Based on the values, the synthetic data scene is generated using the first synthetic data asset and the second synthetic data asset.

Abstract: Systems and methods for providing conversation assistance include receiving from at least one user device of a user, conversation information and determining that the conversation information is associated with a conversation involving the user and a first person that is associated with first conversation assistance information in a non-transitory memory. Body measurement data of the user is retrieved from the at least first user device. A need for conversation assistance in the conversation involving the user and the first person is detected using the body measurement data. First conversation assistance information associated with the first person is retrieved from the non-transitory memory. The first conversation assistance information associated with the first person is provided through the at least one user device.

Abstract: A method includes displaying a user interface having a first region with one or more user interface elements and determining whether a shield is in a locked state or an unlocked state. The shield covers the first region when in the locked state and uncovers at least part of the first region when in the unlocked state. The method includes, in response to determining that the shield is in the locked state, displaying the shield covering the first region and disabling the user interface element(s). The method includes, in response to receiving user input on the shield while the shield is in the locked state, changing the shield to the unlocked state. Additionally, the method includes, in response to determining that the shield is in the unlocked state, displaying the first region such that the user interface element(s) is/are not covered by the shield and enabling the user interface element(s).

Abstract: A mobile device is fitted with a camera and an extended reality (XR) software application program executing on a processor within an XR system. Via the XR software application program, techniques are performed for manipulating virtual objects in an XR environment. In one technique, the XR software application program facilitates the movement of a virtual object from a first location to a second location, within a plane parallel to a touchscreen of a mobile device, responsive to a displacement determined based on user input detected via the touchscreen.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for annotation of 3D models with signs of use that are visible in 2D images. In one aspect, methods are performed by data processing apparatus. The methods can include projecting signs of use in a relatively larger field of view image of an instance of an object onto a 3D model of the object based on a pose of the instance in the relatively larger field of view image, and estimating a relative pose of the instance of the object in a relatively smaller field of view image based on matches between the signs of use in the relatively larger field of view image and the same signs of use in the relatively smaller field of view image.

Abstract: Systems and methods are provided for performing operations including receiving, via a messaging application, a request to scan an image captured by a camera of the user device; determining an attribute of an object depicted in the image; identifying, from a plurality of augmented reality items, an augmented reality item that is associated with the attribute of the object; combining the identified augmented reality item with the image captured by the camera; and displaying, via the messaging application, the image captured by the camera combined with the identified augmented reality item.

Abstract: For displaying a window in an augmented reality view, a processor detects a new augmented reality placetime that includes a new augmented reality position and/or a new augmented reality time of an augmented reality device. The processor calculates new window characteristics for a window at the new augmented reality placetime based on previous window characteristics. The processor further displays the window with the new window characteristics.

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable improved display of virtual content in computer generated reality (CGR) environments. In some implementations, the virtual content includes a reminder associated with a user and the user's physical environment.

Abstract: The subject technology receives, at a client device, a selection of a first selectable graphical item, the first selectable graphical item comprising a first augmented reality content generator corresponding to a particular geolocation. The subject technology causes display, at the client device, of a graphical interface comprising a plurality of selectable augmented reality content items, each selectable augmented reality content item corresponding to a particular activity based at in part on the particular geolocation. The subject technology receives, at the client device, a second selection of a particular selectable augmented reality content item from the plurality of selectable augmented reality content items.

Abstract: An information processing apparatus (2000) acquires a shelf rack image (12) in which a product shelf rack on which a product is displayed is imaged. The information processing apparatus (2000) performs image analysis on the shelf rack image (12), and generates information (actual display information) relevant to a display situation of the product on a product shelf rack (20). The information processing apparatus (2000) acquires reference display information representing a reference for display of the product on the product shelf rack (20). The information processing apparatus (2000) compares the actual display information generated by performing the image analysis on the shelf rack image (12) with the acquired reference display information, and generates comparison information representing a result.

Abstract: A sensor data providing unit included in a control device provides, when an HMD is connected, detection data acquired by a sensor driver as standard sensor data, and continues an operation of acquiring detection data of a first sensor by the sensor driver, and provides, when the HMD is not connected, detection data acquired by a built-in sensor driver as standard sensor data.

Abstract: Some embodiments provide display eyewear with adjustable camera direction. In general, one aspect disclosed features an electronic device, comprising: a structure configured to be worn on the head of a user; a camera movably coupled to the structure and arranged to capture images in a field of view of the user; and a display panel fixedly coupled to the structure and arranged to display, to the user, the images captured by the camera.

Abstract: There is provided a system for displaying information to a user of a vehicle, including a smart glass for displaying the information as three dimensional (3D) elements, a navigation-assisting component and a central database, wherein the smart glass and the navigation-assisting component are in communication with the central database. This database stores the information and aids in sending and displaying relevant and updated information on the smart glass in real-time, depending on a location through which the vehicle is being maneuvered. Further, the information is displayed on the smart glass through markerless Augmented Reality (AR) technology, thereby eliminating a need for external devices or projectors for displaying the information.

Abstract: Augmented/mixed reality virtual venue pipeline optimization is provided. A method can include acquiring, by a device comprising a processor, information relating to a group of augmented reality applications operating in an area; designating, by the device, an augmented reality application of the group of augmented reality applications as a priority application based on the information relating to the group of augmented reality applications; and granting, by the device, prioritized access to edge network resources in a sub-area comprising at least a portion of the area to the priority application relative to at least one non-priority application of the group of augmented reality applications.

Abstract: A method for providing a personalized augmented reality (AR) display includes obtaining, at an augmented reality apparatus, image data of a camera field of view, the camera field of view covering a field of view of an externally oriented camera of the augmented reality apparatus, identifying a first object in the camera field of view based on the image data, and mapping a position of the first object relative to an augmented reality (AR) field of view. The method further includes determining, based on the identity of the first object, whether the first object is an augmentation target, and responsive to determining that the first object is the augmentation target, displaying, on the internally-facing display, an item of AR content associated with the augmentation target in the AR field of view.

Abstract: In example embodiments, techniques are provided to automatically classify individual elements of an infrastructure model by training one or more machine learning algorithms on classified infrastructure models, producing a classification model that maps features to classification labels, and utilizing the classification model to classify the individual elements of the infrastructure model. The resulting classified elements may then be readily subject to analytics, for example, enabling the display of dashboards for monitoring project performance and the impact of design changes. Such techniques enable classification of elements of new infrastructure models or in updates to existing infrastructure models.

Abstract: The invention generally relates to managing and sharing augmented reality (AR) content, and, more specifically, to an AR platform providing synchronized sharing of AR content in real time across multiple AR devices.

Abstract: A virtual reality (VR) system that includes a three-dimensional (3D) point cloud having a plurality of points, a VR viewer having a current position, a graphics processing unit (GPU), and a central processing unit (CPU). The CPU determines a field-of-view (FOV) based at least in part on the current position of the VR viewer, selects, using occlusion culling, a subset of the points based at least in part on the FOV, and provides them to the GPU. The GPU receives the subset of the plurality of points from the CPU and renders an image for display on the VR viewer based at least in part on the received subset of the plurality of points. The selecting a subset of the plurality of points is at a first frame per second (FPS) rate and the rendering is at a second FPS rate that is faster than the first FPS rate.

Abstract: Methods and systems are disclosed for collecting and releasing virtual objects between disparate augmented reality environments. One method comprises receiving user selection to collect an object in a first environment displayed on a user device at a first time. A search request may then be transmitted to a content server from which a virtual object corresponding to the collected object is received in response. The virtual object may be stored in a user library associated with a user identifier of the user. When a second environment is displayed on the user device, the virtual object may be added to the second environment in response to receiving user selection to add the virtual object and determining that the user is associated with the user identifier.

Abstract: Provided is an information processing apparatus that includes a determination unit and a display control unit. The determination unit obtains a position of a virtual object relative to a display region and determines whether or not a correction allowable region set in a region different from the virtual object overlaps at least a part of the display region when the virtual object is located outside the display region. The display control unit causes at least a part of a display object showing the virtual object in the display region to be displayed in a case where the determination unit determines that the correction allowable region overlaps at least the part of the display region.

Abstract: An image processing method, an image processing apparatus, an electronic device and a computer readable storage medium are provided. The image processing method includes the following. A background image and a portrait region image of a current user that a preset parameter of the background image matching the preset parameter of the portrait region image are acquired. The portrait region image and the background image are merged to obtain a merged image.

Abstract: A head mounted display (HMD) is provided. The HMD includes a housing and a view port of the housing. The view port has a screen for rendering an augmented reality scene. Included is a communications device for exchanging streaming data over a network. A depth camera integrated in the housing and oriented to capture depth data of an environment in front of the housing is included. A processor is configured to use the depth data captured by the depth camera to identify spatial positioning of real objects in the environment. A real object is rendered into the augmented reality scene, and the real object is tracked such that insertion of augmented reality objects are placed in coordination with movements of the real object shown in the augmented reality scene. The real object captured by the depth camera is the environment where a user wearing the HMD is located.

Abstract: A supervisory computer vision (CV) system may include a secondary CV system running in parallel with a native CV system on a mobile device. The secondary CV system is configured to run less frequently than the native CV system. CV algorithms are then run on these less-frequent sample images, generating information for localizing the device to a reference point cloud (e.g., provided over a network) and for transforming between a local point cloud of the native CV system and the reference point cloud. AR content may then be consistently positioned relative to the convergent CV system's coordinate space and visualized on a display of the mobile device. Various related algorithms facilitate the efficient operation of this system.

Abstract: A method and apparatus for controlling a movement of a virtual object in AR, and a terminal are provided. The method includes: a virtual moving area is generated within a current image capture range of a control device; a virtual object is generated in the virtual moving area; and a position of a virtual identifier in the virtual moving area is adjusted according to a real movement of the control device, and the virtual object is guided to virtually move according to the position of the virtual identifier. The present disclosure solves a technical problem in the related art that a joystick-based racing control mode provided in an AR scene is complex and game experience is poor.

Abstract: Systems and methods are provided for depicting information about a vehicle in augmented reality. An exemplary method may include determining identification information of a vehicle based on input received from a client device and obtaining, from an information source, vehicle information based on the identification information. The method may also include analyzing the vehicle information using an artificial intelligence engine and extracting vehicle history data from the vehicle information based on the analysis. The method may also include identifying a portion of the vehicle associated with the vehicle history data and providing instructions, based on the vehicle history data, to an augmented reality-enabled device to superimpose an indicator on a visual representation of the vehicle at a location corresponding to the portion of the vehicle.

Abstract: The disclosure provides a method for dynamically showing a virtual boundary, an electronic device and a computer readable storage medium thereof. The method includes: obtaining a virtual scene boundary of a virtual reality (VR) environment; determining a distance threshold based on a moving speed of a specific element of a VR system, wherein the specific distance threshold is positively related to the moving speed of the specific element; monitoring a specific distance between the specific element and the virtual scene boundary of the VR environment; in response to determining that the specific distance is smaller than the distance threshold, showing the virtual scene boundary of the VR environment.

Abstract: A system is provided for recommending guidance instructions to a user. The system includes a memory having computer readable instructions and a processor for executing the computer readable instructions. The computer readable instructions control the processor to perform operations of monitoring an ongoing task comprising at least one act performed by a user, generating image data depicting the ongoing task, and displaying the ongoing task based on the image data. The system analyzes the ongoing task and generates an augmented image. The augmented image is overlaid on the image data so that the augmented image is displayed simultaneously with the ongoing task to direct the user to progress the ongoing task.

Abstract: A computer-implemented method of diagnosing a technology environment using augmented reality may include: receiving status information corresponding to the technology environment; generating, based on the status information, a 3D visualization of the technology environment; causing an augmented reality device to display the 3D visualization, the 3D visualization including one or more first level icons; receiving first selection information from the augmented reality device, the first selection information corresponding to a first selection gesture performed by a user of the augmented reality device; and in response to the first selection information, causing the augmented reality device to display one or more second level icons as part of the 3D visualization.

Abstract: In some examples, an electronic device receives selection of a target image relating to an augmented reality presentation, displays, in a display screen of the electronic device, captured visual data of an environment acquired by the electronic device, and displays, in the display screen, guidance information relating to the target image to assist a user in finding a physical target, corresponding to the target image, in the captured visual data of the environment.

Abstract: A system of properly displaying chroma key content is presented. The system obtains a digital representation of a 3D environment, for example a digital photo, and gathers data from that digital representation. The system renders the digital representation in an environmental model and displays that digital representation upon an output device. Depending upon the context, content anchors of the environmental model are selected which will be altered by suitable chroma key content. The chroma key content takes into consideration the position and orientation of the chroma key content relative to the content anchor and relative to the point of view that the environmental model is displayed from in order to accurately display chroma key content in a realistic manner.