Abstract: A virtual assistant application, system, and service that interacts with a caller over a telephone, cellular, or VoIP network as the called party's representative when the called party is unavailable, providing answers to queries and performing commands that were provisioned by the called party. The application includes a telephony interface, speech recognition and synthesis, a dialog manager, and knowledge domain. The application can also be configured to interact with a sender of SMS, MMS, or IM messages.

Abstract: A virtual assistant application, system, and service that interacts with a caller over a telephone, cellular, or VoIP network as the called party's representative when the called party is unavailable, providing answers to queries and performing commands that were provisioned by the called party. The application includes a telephony interface, speech recognition and synthesis, a dialog manager, and knowledge domain. The application can also be configured to interact with a sender of SMS, MMS, or IM messages.

Abstract: A virtual assistant application, system, and service that interacts with a caller over a telephone, cellular, or VoIP network as the called party's representative when the called party is unavailable, providing answers to queries and performing commands that were provisioned by the called party. The application includes a telephony interface, speech recognition and synthesis, a dialog manager, and knowledge domain. The application can also be configured to interact with a sender of SMS, MMS, or IM messages.

Abstract: A virtual assistant application, system, and service that interacts with a caller over a telephone, cellular, or VoIP network as the called party's representative when the called party is unavailable, providing answers to queries and performing commands that were provisioned by the called party. The application includes a telephony interface, speech recognition and synthesis, a dialog manager, and knowledge domain. The application can also be configured to interact with a sender of SMS, MMS, or IM messages.

Abstract: A method for managing multimodal interactions can include the step of registering a multitude of modality components with a modality component server, wherein each modality component handles an interface modality for an application. The modality component can be connected to a device. A user interaction can be conveyed from the device to the modality component for processing. Results from the user interaction can be placed on a shared memory are of the modality component server.

Abstract: A method for managing multimodal interactions can include the step of registering a multitude of modality components with a modality component server, wherein each modality component handles an interface modality for an application. The modality component can be connected to a device. A user interaction can be conveyed from the device to the modality component for processing. Results from the user interaction can be placed on a shared memory are of the modality component server.

Abstract: A method for managing multimodal interactions can include the step of registering a multitude of modality components with a modality component server, wherein each modality component handles an interface modality for an application. The modality component can be connected to a device. A user interaction can be conveyed from the device to the modality component for processing. Results from the user interaction can be placed on a shared memory are of the modality component server.

Abstract: A method for enhancing voice interactions within a portable multimodal computing device using visual messages. A multimodal interface can be provided that includes an audio interface and a visual interface. A speech input can then be received and a voice recognition task can be performed upon at least a portion of the speech input. At least one message within the multimodal interface can be visually presented, wherein the message is a prompt for the speech input and/or a confirmation of the speech input.

Abstract: A method for managing application modalities using dialogue states can include the step of asserting a set of activation conditions associated with a dialogue state of an application. Each of the activation conditions can be linked to at least one programmatic action, wherein different programmatic actions can be executed by different modality components. The application conditions can be monitored. An application event can be detected resulting in an associated application condition being run. At least one programmatic action linked to the application condition can be responsively initiated.

Abstract: Ordering recognition results produced by an automatic speech recognition (‘ASR’) engine for a multimodal application implemented with a grammar of the multimodal application in the ASR engine, with the multimodal application operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to the ASR engine through a VoiceXML interpreter, includes: receiving, in the VoiceXML interpreter from the multimodal application, a voice utterance; determining, by the VoiceXML interpreter using the ASR engine, a plurality of recognition results in dependence upon the voice utterance and the grammar; determining, by the VoiceXML interpreter according to semantic interpretation scripts of the grammar, a weight for each recognition result; and sorting, by the VoiceXML interpreter, the plurality of recognition results in dependence upon the weight for each recognition result.

Abstract: A method for managing application modalities using dialogue states can include the step of asserting a set of activation conditions associated with a dialogue state of an application. Each of the activation conditions can be linked to at least one programmatic action, wherein different programmatic actions can be executed by different modality components. The application conditions can be monitored. An application event can be detected resulting in an associated application condition being run. At least one programmatic action linked to the application condition can be responsively initiated.

Abstract: Ordering recognition results produced by an automatic speech recognition (‘ASR’) engine for a multimodal application implemented with a grammar of the multimodal application in the ASR engine, with the multimodal application operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to the ASR engine through a VoiceXML interpreter, includes: receiving, in the VoiceXML interpreter from the multimodal application, a voice utterance; determining, by the VoiceXML interpreter using the ASR engine, a plurality of recognition results in dependence upon the voice utterance and the grammar; determining, by the VoiceXML interpreter according to semantic interpretation scripts of the grammar, a weight for each recognition result; and sorting, by the VoiceXML interpreter, the plurality of recognition results in dependence upon the weight for each recognition result.

Abstract: Providing VOIP barge-in support for a half-duplex DSR client on a full-duplex network by buffering, in a half-duplex DSR client, input audio from the full-duplex network; playing, through the half-duplex DSR client, the buffered input audio; pausing, during voice activity on the half-duplex DSR client, the playing of the buffered input audio; sending, during voice activity on the half-duplex DSR client, speech for recognition through the full-duplex network to a voice server; receiving in the half-duplex DSR client through the full-duplex network from the voice server notification of speech recognition, the notification bearing a time stamp; and, responsive to receiving the notification, resuming the playing of the buffered input audio, including playing only buffered VOIP audio data bearing time stamps later than the time stamp of the recognition notification.

Abstract: Streaming audio from a full-duplex network through a half-duplex device, including buffering, in the half-duplex device, input audio from the full-duplex network, where the input audio includes audio data for output through the half-duplex device; playing, through the half-duplex device, the buffered input audio; pausing, during voice activity on the half-duplex device, the playing of the buffered input audio; recording, during the voice activity, output audio to be sent to the full-duplex network; and resuming the playing of the buffered input audio after the voice activity.

Abstract: A method for providing help to voice-enabled applications, including multimodal applications, can include a step of identifying at least one speech grammar associated with a voice-enabled application. Help fields can be defined within the speech grammar. The help fields can include available speech commands for the voice enabled application. When the speech grammar is activated for use by the voice-enabled application, the available speech commands can be presented to a user of the voice-enabled application. The presented speech commands can be obtained from the help fields.

Abstract: A method of building a mixed-initiative grammar can include identifying a plurality of directed dialog grammars for inclusion in the mixed-initiative grammar and automatically generating the mixed-initiative grammar, in accordance with a selected grammar generation technique, such that the mixed-initiative grammar specifies the plurality of directed dialog grammars.

Abstract: A method of verifying a user identity using a Web-based multimodal interface can include sending, to a remote computing device, a multimodal markup language document that, when rendered by the remote computing device, queries a user for a user identifier and causes audio of the user's voice to be sent to a multimodal, Web-based application. The user identifier and the audio can be received at about a same time from the client device. The audio can be compared with a voice print associated with the user identifier. The user at the remote computing device can be selectively granted access to the system according to a result obtained from the comparing step.

Abstract: A method to facilitate programming of multimodal access. The method can include receiving a user selection of at least one graphical user interface (GUI) component defined in visual markup code, and receiving a user selection of at least one voice component. Responsive to the user selections, voice markup code corresponding to the selected voice component can be automatically generated and linked to the GUI component.

Abstract: A multimodal browser for rendering a multimodal document on an end system defining a host can include a visual browser component for rendering visual content, if any, of the multimodal document, and a voice browser component for rendering voice-based content, if any, of the multimodal document. The voice browser component can determine which of a plurality of speech processing configuration is used by the host in rendering the voice-based content. The determination can be based upon the resources of the host running the application. The determination also can be based upon a processing instruction contained in the application.

Abstract: A computer-implemented method and system are provided for filling a graphic-based form field in response to a speech utterance. The computer-implemented method includes generating a grammar corresponding to the form field, the grammar being based on a user profile and comprising a semantic interpretation string. The method further includes creating an auto-fill event based upon the at least one grammar and responsive to the speech utterance, the auto-fill event causing the filling of the form field with data corresponding to the user profile. The system includes a grammar-generating module for generating a grammar corresponding to the form field, the grammar being based on a user profile and comprising a semantic interpretation string. The system also includes an event module for creating an auto-fill event based upon the at least one grammar and responsive to the speech utterance, the event causing the filling of the form field with data corresponding to the user profile.