Abstract: A mixed reality (MR) system and method performs alignment of a digital twin and the corresponding real-world object using 3D deep neural network structures using multimodal fusion and simplified machine learning to cluster label distributions (output of 3D deep neural network trained by generic 3D benchmark dataset) that are used to reduce the training data requirements to directly train a 3D deep neural network structures. In one embodiment, multiple 3D deep neural network structures, such as PointCNN, 3D-Bonet, RandLA, etc., may be trained by different generic 3D benchmark datasets, such as ScanNet, ShapeNet, S3DIS, inadequate 3D training dataset, etc.

Abstract: A mixed reality (MR) system and method performs three dimensional (3D) tracking using 3D deep neural network structures in which multimodal fusion and simplified machine learning to only cluster label distribution (output of 3D deep neural network trained by generic 3D benchmark dataset) is used to reduce the training data requirements of to directly train a 3D deep neural network structures for non-generic user case. In one embodiment, multiple 3D deep neural network structures, such as PointCNN, 3D-Bonet, RandLA, etc., may be trained by different generic 3D benchmark datasets, such as ScanNet, ShapeNet, S3DIS, inadequate 3D training dataset, etc.

Abstract: A wireless network and video/audio system and method for gaming and virtual reality are provided. The system and method intelligently harnesses the computing power of network of local-devices and the cloud to make powerful computing possible, anywhere.