Abstract: Systems and methods are described to enable the creation of user interfaces that may adapt to different environments and may be automatically created. User interfaces may be two-dimensional or three-dimensional and may be used in virtual reality or augmented reality applications. An interface creator may create or receive digital assets associated with a content item, define virtual planes and associated digital asset templates, associate the digital assets with the virtual planes, and enable display of the virtual planes with associated digital assets to a user for user interaction. Digital assets may be automatically edited to meet the specifications of the templates associated with the virtual planes. Virtual planes and templates may also be standardized and aggregated so that a completed user interface may be easily delivered and presented with other content items in a uniform manner.

Abstract: Techniques are described herein that enable a user to provide speech inputs to control an extended reality environment, where relationships between terms in a speech input are represented in three dimensions (3D) in the extended reality environment. For example, a language processing component determines a semantic meaning of the speech input, and identifies terms in the speech input based on the semantic meaning. A 3D relationship component generates a 3D representation of a relationship between the terms and provides the 3D representation to a computing device for display. A 3D representation may include a modification to an object in an extended reality environment, or a 3D representation of a concepts and sub-concepts in a mind map in an extended reality environment, for example. The 3D relationship component may generate a searchable timeline using the terms provided in the speech input and a recording of an extended reality session.

Abstract: In one implementation, a method includes: obtaining locality data characterizing objects and relative spatial information of a volumetric region around a user; synthesizing a mesh map of the volumetric region based on the locality data; selecting synthesized reality (SR) content based on the mesh map, wherein the SR content satisfies a dimensional variance threshold relative to one or more portions of the mesh map; compositing at least a portion of the SR content with the mesh map in order to generate composite SR content; and presenting the composite SR content to the user in order to occlude at least a portion of a visual presentation of the volumetric region. In some implementations, the SR content is adapted to fit the one or more portions of the mesh map. In some implementations, the SR content is updated as the user location changes or the user interacts with the SR content.

Abstract: Methods and systems for three-dimensional asset modeling. A method includes: initializing a georeferenced data structure for each of a plurality of discrete sub-regions of an asset based on a model representing the asset, wherein the model includes a plurality of points representing features of the asset, wherein each georeferenced data structure includes a subset of the plurality of points representing features of a respective sub-region of the plurality of sub-regions; and populating each georeferenced data structure with input data including three-dimensional (3D) modeling data and nonspatial data, wherein each portion of the input data for a georeferenced data structure is used to populate a respective portion of the georeferenced data structure, wherein the nonspatial data used to populate each georeferenced data structure is organized with respect to geometry of the georeferenced data structure and with respect to time of recording of the nonspatial data.

Abstract: A system and method for indoor spatial mapping. The system includes a mobile mapping device having a camera; a plan database configured to store 2-D floor plans; and a processor configured to integrate a selected floor plan from the plan database with object data received from the mobile mapping device to create an interactable 3-D spatial mapping of an interior portion of a structure so that one or more objects are moveable relative to a surrounding environment within the 3-D spatial mapping. The processor is configured to recognise objects from an image from the camera and anchor the recognised object within the interactable 3-D spatial mapping.

Abstract: Various implementations disclosed herein include devices, systems, and methods for synthesizing an environment based on an image. In various implementations, a device includes a non-transitory memory and one or more processors coupled with the non-transitory memory. In various implementations, a method includes determining an engagement score that characterizes a level of engagement between a user and a representation of a subject included in an image. In some implementations, the method includes, in response to the engagement score satisfying an engagement threshold, obtaining stored information regarding the subject, and synthesizing an environment based on the image and the stored information regarding the subject.

Abstract: This application provides a method for generating an animation expression and an electronic device. The method for generating an animation expression includes: obtaining an initial three-dimensional mesh, where a vertex in the initial three-dimensional mesh is used to represent an expression feature of a face; transforming the initial three-dimensional mesh to obtain a target three-dimensional mesh, where a topological structure of the target three-dimensional mesh is the same as a topological structure of a basic blendshape; determining a personalized blendshape fitting the face based on the basic blendshape; determining personalized blendshape coefficients based on the target three-dimensional mesh and the personalized blendshape; and generating the animated expression based on the personalized blendshape coefficients.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for displaying augmented reality content. The program and method provide for receiving, from a first device associated with a first user, a first request to submit a media content item for viewing, the media content item corresponding to display of a first image, captured by the first device, with an augmented reality content item, the first request including an identifier of the augmented reality content item; storing the media content item together with the identifier of the augmented reality content item; providing, to a second device associated with a second user, the media content item; receiving, from the second device, a second request to identify the augmented reality content item; and providing, to the second device, the identifier of the augmented reality content item.

Abstract: A method is disclosed, the method comprising the steps of identifying a first real object in a mixed reality environment, the mixed reality environment having a user; identifying a second real object in the mixed reality environment; generating, in the mixed reality environment, a first virtual object corresponding to the second real object; identifying, in the mixed reality environment, a collision between the first real object and the first virtual object; determining a first attribute associated with the collision; determining, based on the first attribute, a first audio signal corresponding to the collision; and presenting to the user, via one or more speakers, the first audio signal.

Abstract: According to one aspect, a toy system is adapted for use in a multiplayer game, the toy system comprising a primary user device, a secondary user device, and one or more toys; wherein the primary and secondary user devices are operable in a common multiplayer game session; wherein the primary user device is configured to capture video images of a real-world scene including the one or more toys from a primary view point and to display the captured video images as a primary view of the real-world scene; wherein the secondary user device is configured to display a secondary view of a scene corresponding to the real-world scene as seen from a secondary view point; wherein the one or more toys comprise at least one user-manipulable object adapted to selectively be brought into a user-selected one of a plurality of predetermined states, at least a first one of the predetermined states having a first visual appearance; wherein the primary user device is configured to detect said first predetermined state in a primar

Abstract: An augmented reality system and a display method for anchoring a virtual object thereof are provided. An image sequence is captured from an electronic equipment through an image capturing device on a head-mounted device. The image sequence includes an ith and an (i+1)th environmental image. The electronic equipment includes a component device. A feature matching operation is performed on the ith environmental image and a first template image to obtain a matching result. In response to the matching result meeting a preset condition, a second template image extending from a matching image block of the component device is captured from the ith environmental image. A feature matching operation is performed on the (i+1)th environmental image and the second template image to display a virtual object through the head-mounted device according to another matching result. The virtual object is displayed as being anchored to the electronic equipment in a real scene.

Abstract: A display system including a PC and an HMD, and in the display system, the PC includes an I/F unit configured to output image data, the HMD includes an I/F unit configured to acquire the image data output by the PC, an image display unit configured to superimpose an image on an outside scene visually recognized in a state where the HMD is worn, and to display the image, and a display controller configured to cause the image display unit to display the image data acquired by the I/F unit, and the display controller causes the image display unit to display the image data in correspondence to a position of the PC visually recognized as an outside scene.

Abstract: A system for integrating high-density laser scanner data with mixed-reality is disclosed. In some implementations, the system may acquire, using a laser scanner, a collection of data points representing a three-dimensional point cloud, wherein the laser scanner includes: a laser configured to generate an optical beam; a beam-steering device configured to steer the optical beam; and a detector configured to receive light from the optical beam, after light from the optical beam is reflected from an object in an environment. The system may transmit data of the point cloud to the augmented-reality device, wherein the augmented-reality device comprises: one or more cameras configured to acquire images of the environment; and a display configured to render graphics on the display. The system may align a reference frame of the augmented-reality device to the data of the point cloud.

Abstract: A method for arranging and manipulating augmented reality (AR) content in the 3D space according to an embodiment of the present disclosure, performed by a creation application executed by at least one or more processors of a mobile terminal, comprises detecting a 3D arrangement space which is a 3D space representing a surrounding environment; displaying AR content on the detected 3D arrangement space; providing an attribute information edit interface through which position information of the displayed AR content is modified according to a user input; modifying position information of the AR content based on a user's gesture input through the provided attribute information edit interface; reconfiguring the modified position information as the position information of the AR content; and storing the reconfigured AR content.

Abstract: Augmented reality glasses are used, with a transducer array, to give images, produced by the glasses, a feel in midair. The glasses images can be shown as three-dimensional images, that are located in midair. The images are located near the transducer array. The array produces a touch feel in mid-air. The mid-air touch feel occupies the same space as the image. The feel gives the image a touch feel, when a user touches the image. The touch sense, can give the image a more real interaction when touched, such as, giving an image of a food item like a sandwich, a feel similar to a sandwich feel. The images with feel, allow for an immersive experience, within the augmented reality environment. The feel can move, when the image moves, such as, the user views an image, of another user's hand moving, and feels the movement of the hand.

Abstract: A representation of a surface of one or more objects is positioned in a virtual space is obtained in a computer animation system. Thereafter, a guide curve specification of a guide curve in the virtual space relative to the surface is received. Thereafter, the computer animation system computes a first set of tangent vector values for differentiable locations along the guide curve and computes a second set of tangent vector values for nondifferentiable locations along the guide curve. Using the first set and second set, the computer animation system computes a third set of tangent vector values for locations on the surface other than locations along the guide curve and computes a tangent vector field over the surface from at least the first set of tangent vector values, the second set of tangent vector values, and the third set of tangent vector values.

Abstract: Systems and methods for reducing pose space dimensionality. A plurality of example poses can define an input pose space. Each of the example poses can include a set of joint rotations for a virtual character. The joint rotations can be expressed with a singularity-free mathematical representation. The plurality of example poses can then be clustered into one or more clusters. A representative pose can be determined for each cluster. An output pose space with a reduced dimensionality, as compared to the input pose space, can then be provided.

Abstract: An illustrative multiscale data system determines a first distance between a first object in a scene and a virtual vantage point at the scene. The multiscale data system also determines a second distance between a second object in the scene and the virtual vantage point. In an example in which the second distance is greater than the first distance, the multiscale data system generates, based on the first and second distances, a tiled representation associated with the virtual vantage point. The tiled representation in this example includes a first representation of the first object rendered at a first quality level and a second representation of the second object rendered at a second quality level lower than the first quality level. Corresponding methods and systems are also disclosed.

Abstract: Various post-production and distribution tools for volumetric video data are enabled. For instance, a method comprises receiving, by a device comprising a processor, a group of images, determining, by the device, first points of interest of a first image of the group of images and second points of interest of a second image of the group of images, comparing, by the device, the first points of interest to the second points of interest, and determining, by the device, from the first points of interest and the second points of interest and based on a result of the comparing, sets of candidate coordinate positions of a joint of a character represented in the first image and the second image.

Abstract: Systems, methods and apparatus are provided through which a 3D source mesh object is generated by a three-dimensional-augmented-reality engine from medical data, a copy mesh object is generated by copying the 3D source mesh object, a center of the 3D source mesh object is calculated from the 3D source mesh object, an orientation bar is generated and placed orthogonally in the center of the copy mesh object, a scene mesh object a generated by scanning the room, the scene mesh object being a game object, a patient mesh object is generated by cutting the scene mesh object in reference to original dimensions of the 3D source mesh object and a patient mesh object is generated from the patient mesh object by starting from the index tip of a finger and placing the 3D source mesh object in a direction that is downwards.