Abstract: The system includes a voice assistant receiving a voice command as input from user. The speech to text convertor converts the voice command into a text. A feature extractor extracts acoustic features from raw waveform of voice command and textual features from converted text for determining nearby context tokens. A multi modal unified attention sequence tagger determines a connection between the audio and the text based on an individual contextual embedding and a fused contextual embedding at context tokens level. It further tags replacement, cue and correction words sequentially based on determined connection between the audio and the text. An on-the-fly decoder decodes revised text on-the-fly based on tagged replacement, cue and correction words, to display the decoded revised text on user interface and sends the decoded revised text to NLP to generate a response corresponding to the input speech.