Abstract: A system for fusion of multimodal receives a spatial input and a temporal input, wherein the spatial input comprises spatial data having spatial embeddings and the temporal input comprises temporal data having temporal embeddings. The spatial embeddings and the temporal embeddings have different time dimensions. A spatial data output with the spatial embeddings having a same time dimension as the temporal embeddings is generated from the spatial data based on a spatial perception model. The spatial perception model is pre-trained. A temporal data output is generated from the temporal data based on a temporal model. The spatial data output and the temporal data output are combined into an output representing dependencies between the spatial input and the temporal input using a fusion model. A desired target variable is obtained from the output and one of an estimated or predicted value is generated based on the desired target value.

Abstract: A method, apparatus and system for efficient navigation in a navigation space includes determining semantic features and respective 3D positional information of the semantic features for scenes of captured image content and depth-related content in the navigation space, combining information of the determined semantic features of the scene with respective 3D positional information using neural networks to determine an intermediate representation of the scene which provides information regarding positions of the semantic features in the scene and spatial relationships among the sematic features, and using the information regarding the positions of the semantic features and the spatial relationships among the sematic features in a machine learning process to provide at least one of a navigation path in the navigation space, a model of the navigation space, and an explanation of a navigation action by the single, mobile agent in the navigation space.