Abstract: Techniques and systems are provided for pose prediction. For instance, a process can include combining image features detected from an obtained image with estimated image features to generate combined features; generating temporally encoded features by temporally encoding the combined features; combining detected motion tracking information with estimated motion tracking information to generate combined motion tracking information; generating temporally encoded motion tracking information by temporally encoding the combined motion tracking information; generating spatially encoded multi-modal information by spatially encoding the temporally encoded features and the temporally encoded motion tracking information; and predicting a body pose by regressing the spatially encoded multi-modal information.

Abstract: Techniques and systems are provided for pose generation. For instance, a process can include generating, using a machine learning (ML) model, a first estimated body pose based on sensor information received from one or more sensors of a set of sensors; generating, using a physics-based model, a second estimated body pose based on the sensor information; detecting a failure of the physics-based model; and based on detecting the failure of the physics-based model, switching from generating a skeletal pose based on the second estimated body pose to generating the skeletal pose based on the first estimated body pose.

Abstract: Techniques and systems are provided for pose prediction. For instance, a process can include combining image features detected from an obtained image with estimated image features to generate combined features; generating temporally encoded features by temporally encoding the combined features; combining detected motion tracking information with estimated motion tracking information to generate combined motion tracking information; generating temporally encoded motion tracking information by temporally encoding the combined motion tracking information; generating spatially encoded multi-modal information by spatially encoding the temporally encoded features and the temporally encoded motion tracking information; and predicting a body pose by regressing the spatially encoded multi-modal information.