Abstract: An extended reality system and extended reality method for digital twins, in which the digital twin can be a virtual asset of a real asset. The real asset can be a real object. For example, an initiation of an event in relation to the real asset causes the extended reality method to generate one or more predicted virtual states which are predicted to achieve the event in the virtual asset. The event can be initiated through the real asset and through the virtual asset. The extended reality method can receive one or more further real states of the real asset which achieve the event. The extended reality method can generate a reality 3D map in an extended reality application which concurrently displays, in the 3D space, the virtual asset in the one or more predicted virtual states and the real asset in the one or more further real states.

Abstract: A floorplan modelling method and system. The floorplan modelling method includes receiving 2D images of each corner of an interior space from a camera, generating a corresponding camera position and camera orientation in a 3D coordinate system in the interior space for each 2D image, generating a depth map for each 2D image to estimate depth for each pixel, generating a corresponding edge map for each 2D image, and generating a 3D point cloud for each 2D image using the corresponding depth map and parameters of the camera. The floorplan modelling method includes transforming the 3D point clouds with the corresponding edge map into a 2D space in the 3D coordinate system of the camera, regularizing the 3D point clouds into 2D boundary lines, and generating a 2D plan of the interior space from the boundary lines.

Abstract: An object and camera localization system and localization method for mapping of the real world. The mapping can be done in real-time or near real-time to the detection of the real objects by a camera device which is used to capture one or more images of an object. The localization method can be used to generate an object label of the object and a bounding box of the object in the image. The localization method can be used to generate anchor points in real world coordinates of the real 3D space of the object, a cuboid of the object, and a centroid of the cuboid. A virtual 3D map can be generated that which includes the location and pose of the real object in the real-world coordinates.

Abstract: A method for sharing a mixed reality experience (mixed reality content, mixed reality event) between one or more computing devices is disclosed. The method includes: determining a spatial location and a spatial orientation (spatial data) of the one or more computing devices each having a camera; mapping the (spatial) location and/or the spatial orientation (spatial data) of each of the one or more computing devices into a mixed reality manager; and presenting an event that is shared among the one or more computing devices, and, the presenting of the event is experienced simultaneously and varies among each of the one or more computing devices depending on the location or the orientation or both.