Abstract: In one embodiment, a method includes receiving, by a XR display device, a gesture-based input from a first user of the XR display device, processing, using a gesture-detection model, the gesture-based input to identify a first gesture, receiving, by the XR display device, an audio input from the first user, where the audio input includes a first voice command, processing, using a natural-language model, the audio input to identify one or more intents or one or more slots associated with the first voice command, determining whether the identified first gesture matches the first voice command, and executing, responsive to the identified first gesture matching the first voice command and by the XR display device, a first task corresponding to the first voice command based on the identified first gesture and the identified one or more intents or one or more slots.