Abstract: A method for processing multi-modal input includes receiving multiple signal inputs, each signal input having a corresponding input mode. Each signal input is processed in a series of mode-specific processing stages. Each successive mode-specific stage is associated with a successively longer scale of analysis of the signal input. A fused output is generated based on the output of a series of fused processing stages. Each successive fused processing stage is associated with a successively longer scale of analysis of the signal input. Multiple fused processing stages receive inputs from corresponding mode-specific processing stages, so that the fused output depends on the multiple of signal inputs.

Abstract: A computer implemented method for analyzing media content includes a step of providing a plurality of narrative files formatted in human readable format. Each narrative file includes a script and/or dialogues tagged with character names along with auxiliary information. Each script includes a plurality of portrayals performed by an associated actor or character. Linguistic representations of content of the narrative files in both abstract and semantic forms is determined. The linguistic representations are connected to higher order representations and mental states. The linguistic representations are connected to behavior and action. Interplay between language constructs and demographics of content creators is analyzed. Content representations towards individuals/groups are adapted to reflect heterogeneity in preferences.

Abstract: A computer implemented method for analyzing media content includes a step of providing a plurality of narrative files formatted in human readable format. Each narrative file includes a script and/or dialogues tagged with character names along with auxiliary information. Each script includes a plurality of portrayals performed by an associated actor or character. Linguistic representations of content of the narrative files in both abstract and semantic forms is determined. The linguistic representations are connected to higher order representations and mental states. The linguistic representations are connected to behavior and action. Interplay between language constructs and demographics of content creators is analyzed. Content representations towards individuals/groups are adapted to reflect heterogeneity in preferences.

Abstract: A computer implemented method for analyzing media content includes a step of providing a plurality of narrative files formatted in human readable format. Each narrative file includes a script and/or dialogues tagged with character names along with auxiliary information. Each script includes a plurality of portrayals performed by an associated actor or character. Linguistic representations of content of the narrative files in both abstract and semantic forms is determined. The linguistic representations are connected to higher order representations and mental states. The linguistic representations are connected to behavior and action. Interplay between language constructs and demographics of content creators is analyzed. Content representations towards individuals/groups are adapted to reflect heterogeneity in preferences.

Abstract: Data augmentation is used for speech emotion recognition tasks where certain emotional labels, e.g., sadness, are significantly underrepresented in a training dataset. This is typical for data collected in real-life applications. We propose conditioned data augmentation using Generative Adversarial Networks (GANs), in order to generate samples for underrepresented emotions. We propose a conditional GAN architecture to generate synthetic spectrograms for the minority class. For comparison purposes, we implement a series of signal-based data augmentation methods. Results on the speech emotion recognition task show that the proposed data augmentation method significantly improves classification performance as compared to traditional speech data augmentation methods.

Abstract: A method for processing multi-modal input includes receiving multiple signal inputs, each signal input having a corresponding input mode. Each signal input is processed in a series of mode-specific processing stages. Each successive mode-specific stage is associated with a successively longer scale of analysis of the signal input. A fused output is generated based on the output of a series of fused processing stages. Each successive fused processing stage is associated with a successively longer scale of analysis of the signal input. Multiple fused processing stages receive inputs from corresponding mode-specific processing stages, so that the fused output depends on the multiple of signal inputs.

Abstract: Behavioral profiling and shaping is used in a “closed-loop” in that an interaction with at least one human is monitored and based on inferred characteristics of the interaction with that human (e.g., their behavioral profile) the interaction is guided. In one exemplary embodiment, the interaction is between two humans, for example, a “customer” and an “agent” and the interaction is monitored and the agent is guided according to the inferred behavioral profile of the customer (or optionally of the agent themselves).

Abstract: A computer implemented method for analyzing media content includes a step of providing a plurality of narrative files formatted in human readable format. Each narrative file includes a script and/or dialogues tagged with character names along with auxiliary information. Each script includes a plurality of portrayals performed by an associated actor or character. Linguistic representations of content of the narrative files in both abstract and semantic forms is determined. The linguistic representations are connected to higher order representations and mental states. The linguistic representations are connected to behavior and action. Interplay between language constructs and demographics of content creators is analyzed. Content representations towards individuals/groups are adapted to reflect heterogeneity in preferences.

Abstract: A translation between a source language and a target language is disclosed. The source language items are divided, with primary and secondary source language items or named entities being identified, where the primary and secondary source languages being different from each other and from the target language. The entities in the second source language are translated in a different way. For example, they may be copied into the target language, or translated in a special way that is based on their meaning, e.g, into a term that has a more descriptive meaning in the target language.

Abstract: A socially-cognizant translation system that takes social state between speaker and listener into account when making the translation. The translation may be more formal or less formal based on the social relationship between speaker and listener. Profanities in the source speech may be detected, and used to determine the social cognizance. Also, the source speech can be translated without the profanity in the target language, instead using the meaning of the profanity.

Abstract: Spoken translation system which detects both speech from the information and also detects meta information streams from the information. A first aspect produces an enriched training corpus of information for use in the machine translation. A second aspect uses two different extraction techniques, and combines them by lattice rescoring.

Abstract: An application (computer program, an embodiment can be a game) which requires translation as one of its metrics is used to help the user can learn a language while operating the system (in a game embodiment, playing the game). The interaction is carried out only in a foreign language, but the application also includes translation capability. A virtual buddy can be used to translate between the native language and the foreign language so that the user can translate information and eventually learn information about the language by the process of interacting with the system (in an embodiment playing the game).

Abstract: Forming and/or improving a language model based on data from a large collection of documents, such as web data. The collection of documents is queried using queries that are formed from the language model. The language model is subsequently improved using the information thus obtained. The improvement is used to improve the query. As data is received from the collection of documents, it is compared to a rejection model, that models what rejected documents typically look like. Any document that meets the test is then rejected. The documents that remain are characterized to determine whether they add information to the language model, whether they are relevant, and whether they should be independently rejected. Rejected documents are used to update the rejection model; accepted documents are used to update the language model. Each iteration improves the language model, and the documents may be analyzed again using the improved language model.

Abstract: A translation between a source language and a target language. The items are divided, with secondary source language items or named entities being identified. Those entities are translated in a different way. For example, they may be copied into the target language, or translated in a special way that is based on their meaning, e.g., into a term that has a more descriptive meaning in the target language.

Abstract: Spoken translation system which detects both speech from the information and also detects meta information streams from the information. A first aspect produces an enriched training corpus of information for use in the machine translation. A second aspect uses two different extraction techniques, and combines them by lattice rescoring.

Abstract: An emotion recognition system for assessing human emotional behavior from communication by a speaker includes a processing system configured to receive signals representative of the verbal and/or non-verbal communication. The processing system derives signal features from the received signals. The processing system is further configured to implement at least one intermediate mapping between the signal features and one or more elements of an emotional ontology in order to perform an emotion recognition decision. The emotional ontology provides a gradient representation of the human emotional behavior.

Abstract: An application (computer program, an embodiment can be a game) which requires translation as one of its metrics is used to help the user can learn a language while operating the system (in a game embodiment, playing the game). The interaction is carried out only in a foreign language, but the application also includes translation capability. A virtual buddy can be used to translate between the native language and the foreign language so that the user can translate information and eventually learn information about the language by the process of interacting with the system (in an embodiment playing the game).

Abstract: A socially-cognizant translation system that takes social state between speaker and listener into account when making the translation. The translation may be more formal or less formal based on the social relationship between speaker and listener. Profanities in the source speech may be detected, and used to determine the social cognizance. Also, the source speech can be translated without the profanity in the target language, instead using the meaning of the profanity.

Abstract: Forming and/or improving a language model based on data from a large collection of documents, such as web data. The collection of documents is queried using queries that are formed from the language model. The language model is subsequently improved using the information thus obtained. The improvement is used to improve the query. As data is received from the collection of documents, it is compared to a rejection model, that models what rejected documents typically look like. Any document that meets the test is then rejected. The documents that remain are characterized to determine whether they add information to the language model, whether they are relevant, and whether they should be independently rejected. Rejected documents are used to update the rejection model; accepted documents are used to update the language model. Each iteration improves the language model, and the documents may be analyzed again using the improved language model.