Abstract: A system includes a computing platform including processing hardware and a system memory storing software code. The processing hardware executes the software code to receive a transcript of speech by a user, generate a phoneme stream corresponding to the transcript, partition the phoneme stream into words, and aggregate subsets of the words to form candidate sentences. The software code is further executed to determine, using one or both of an entity identified in the transcript and a history of the user, one or both of a user intent and a context of the speech by the user, rank the candidate sentences and the transcript based on one or both of the user intent and the context of the speech by the user, and identify, based on the ranking, one of the candidate sentences or the transcript as the best transcription of the speech by the user.

Abstract: A system includes a computing platform having processing hardware and a memory storing a software code. The processing hardware is configured to execute the software code to receive multiple content units each including a start descriptor for an initial content segment and an end descriptor for a last content segment, identify the start descriptor and the end descriptor for each of the content units, and select a first content unit for beginning a content compilation. The processing hardware is further configured to execute the software code to determine multiple similarity metrics each comparing the end descriptor of the first content unit with the start descriptor of a respective one of the other content units, rank, using the similarity metrics, the other content units with respect to one another, select, based on the rank, a second content unit, and composite the content compilation using the second content unit.

Abstract: A system for emotionally enhancing dialogue includes a computing platform having processing hardware and a system memory storing a software code including a predictive model. The processing hardware is configured to execute the software code to receive dialogue data identifying an utterance for use by a digital character in a conversation, analyze, using the dialogue data, an emotionality of the utterance at multiple structural levels of the utterance, and supplement the utterance with one or more emotional attributions, using the predictive model and the emotionality of the utterance at the multiple structural levels, to provide one or more candidate emotionally enhanced utterance(s).

Abstract: According to one implementation, a method of guiding an interaction between a companion module and a user includes identifying a media content for playout by a media player device, transmitting, to a remote server, an interaction schema request identifying the media content and the companion module, and receiving, from the remote server, behavioral manifest data including an instruction for guiding the interaction between the companion module and the user. Such a method also includes obtaining a play head state of the media player device and a playout timecode state of the media content, and identifying a user interaction behavior for the companion module based on the behavioral manifest data, the play head state of the media player device, and the playout timecode state of the media content.

Abstract: A system for producing conversation-driven character animation includes a computing platform having processing hardware and a system memory storing software code, the software code including multiple trained machine learning (ML) models. The processing hardware executes the software code to obtain a conversation understanding feature set describing a present state of a conversation between a digital character and a system user, and to generate an inference, using at least a first trained ML model of the multiple trained ML models and the conversation understanding feature set, the inference including labels describing a predicted next state of a scene within the conversation. The processing hardware further executes the software code to produce, using at least a second trained ML model of the multiple trained ML models and the labels, an animation stream of the digital character participating in the predicted next state of the scene within the conversation.

Abstract: A system for producing conversation-driven character animation includes a computing platform having processing hardware and a system memory storing software code, the software code including multiple trained machine learning (ML) models. The processing hardware executes the software code to obtain a conversation understanding feature set describing a present state of a conversation between a digital character and a system user, and to generate an inference, using at least a first trained ML model of the multiple trained ML models and the conversation understanding feature set, the inference including labels describing a predicted next state of a scene within the conversation. The processing hardware further executes the software code to produce, using at least a second trained ML model of the multiple trained ML models and the labels, an animation stream of the digital character participating in the predicted next state of the scene within the conversation.

Abstract: A system for emotionally enhancing dialogue includes a computing platform having processing hardware and a system memory storing a software code including a predictive model. The processing hardware is configured to execute the software code to receive dialogue data identifying an utterance for use by a digital character in a conversation, analyze, using the dialogue data, an emotionality of the utterance at multiple structural levels of the utterance, and supplement the utterance with one or more emotional attributions, using the predictive model and the emotionality of the utterance at the multiple structural levels, to provide one or more candidate emotionally enhanced utterance(s).

Abstract: Systems and methods are presented for detecting physical contacts effectuated by actions performed by an entity participating in an event. An action, performed by the entity, is detected based on a sequence of pose data associated with the entity's performance in the event. A contact with another entity in the event is detected based on data associated with the detected action. The action and the contact detections are employed by neural-network based detectors.

Abstract: A system providing a multi-persona social agent includes a computing platform having a hardware processor, a system memory storing a software code, and multiple neural network (NN) based predictive models accessible by the software code. The hardware processor executes the software code to receive input data corresponding to an interaction with a user, determine a generic expression for use in the interaction, and identify one of the character personas as a persona to be assumed by the multi-persona social agent. The software code also generates, using the generic expression and one of the NN based predictive models corresponding to the persona to be assumed by the multi-persona social agent, a sentiment driven personified response for the interaction with the user based on a vocabulary, phrases, and one or more syntax rules idiosyncratic to the persona to be assumed, and renders the sentiment driven personified response using the multi-persona social agent.

Abstract: Systems and methods are presented for generating statistics associated with a performance of a participant in an event, wherein pose data associated with the participant, performing in the event, are processed in real time. Pose data associated with the participant may comprise positional data of a skeletal representation of the participant. Actions performed by the participant may be determined based on a comparison of segments of the participant's pose data to motion patterns associated with actions of interests.

Abstract: A system includes a computing platform including processing hardware and a system memory storing software code. The processing hardware executes the software code to receive a transcript of speech by a user, generate a phoneme stream corresponding to the transcript, partition the phoneme stream into words, and aggregate subsets of the words to form candidate sentences. The software code is further executed to determine, using one or both of an entity identified in the transcript and a history of the user, one or both of a user intent and a context of the speech by the user, rank the candidate sentences and the transcript based on one or both of the user intent and the context of the speech by the user, and identify, based on the ranking, one of the candidate sentences or the transcript as the best transcription of the speech by the user.

Abstract: According to one implementation, a method of guiding an interaction between a companion module and a user includes identifying a media content for playout by a media player device, transmitting, to a remote server, an interaction schema request identifying the media content and the companion module, and receiving, from the remote server, behavioral manifest data including an instruction for guiding the interaction between the companion module and the user. Such a method also includes obtaining a play head state of the media player device and a playout timecode state of the media content, and identifying a user interaction behavior for the companion module based on the behavioral manifest data, the play head state of the media player device, and the playout timecode state of the media content.

Abstract: According to one implementation, a method of guiding an interaction between a companion module and a user includes identifying a media content for playout by a media player device, transmitting, to a remote server, an interaction schema request identifying the media content and the companion module, and receiving, from the remote server, behavioral manifest data including an instruction for guiding the interaction between the companion module and the user. Such a method also includes obtaining a play head state of the media player device and a playout timecode state of the media content, and identifying a user interaction behavior for the companion module based on the behavioral manifest data, the play head state of the media player device, and the playout timecode state of the media content.

Abstract: Systems and methods are presented for generating statistics associated with a performance of a participant in an event, wherein pose data associated with the participant, performing in the event, are processed in real time. Pose data associated with the participant may comprise positional data of a skeletal representation of the participant. Actions performed by the participant may be determined based on a comparison of segments of the participant's pose data to motion patterns associated with actions of interests.

Abstract: Systems and methods are presented for detecting physical contacts effectuated by actions performed by an entity participating in an event. An action, performed by the entity, is detected based on a sequence of pose data associated with the entity's performance in the event. A contact with another entity in the event is detected based on data associated with the detected action. The action and the contact detections are employed by neural-network based detectors.

Abstract: A system includes a computing platform having processing hardware and a memory storing a software code. The processing hardware is configured to execute the software code to receive multiple content units each including a start descriptor for an initial content segment and an end descriptor for a last content segment, identify the start descriptor and the end descriptor for each of the content units, and select a first content unit for beginning a content compilation. The processing hardware is further configured to execute the software code to determine multiple similarity metrics each comparing the end descriptor of the first content unit with the start descriptor of a respective one of the other content units, rank, using the similarity metrics, the other content units with respect to one another, select, based on the rank, a second content unit, and composite the content compilation using the second content unit.

Abstract: A system includes a computing platform having one or more processor(s) configured to receive input data corresponding to an interaction with a user, to determine a character archetype, an intent, and a sentiment of the user, to generate, using the input data and the character archetype, an output data that includes a token describing a payload, and to identify, using the token, a database corresponding to the payload. The processor(s) are further configured to obtain, by searching the database based on the character archetype, the intent, and the sentiment of the user, the payload from the database, to transform, using the character archetype and the intent and the sentiment of the user, the output data and the payload to a response to the interaction, and to render the response using a social agent that assumes the character archetype.

Abstract: A system providing a multi-persona social agent includes a computing platform having a hardware processor, a system memory storing a software code, and multiple neural network (NN) based predictive models accessible by the software code. The hardware processor executes the software code to receive input data corresponding to an interaction with a user, determine a generic expression for use in the interaction, and identify one of the character personas as a persona to be assumed by the multi-persona social agent. The software code also generates, using the generic expression and one of the NN based predictive models corresponding to the persona to be assumed by the multi-persona social agent, a sentiment driven personified response for the interaction with the user based on a vocabulary, phrases, and one or more syntax rules idiosyncratic to the persona to be assumed, and renders the sentiment driven personified response using the multi-persona social agent.

Abstract: According to one implementation, a method of guiding an interaction between a companion module and a user includes identifying a media content for playout by a media player device, transmitting, to a remote server, an interaction schema request identifying the media content and the companion module, and receiving, from the remote server, behavioral manifest data including an instruction for guiding the interaction between the companion module and the user. Such a method also includes obtaining a play head state of the media player device and a playout timecode state of the media content, and identifying a user interaction behavior for the companion module based on the behavioral manifest data, the play head state of the media player device, and the playout timecode state of the media content.

Abstract: A device, system, and method cast a ray projection from a perspective view. The method includes determining a first mask for a first object from a first frame captured by a first camera. The method includes determining a second mask for the first object from a second frame captured by a second camera. The method includes generating a 3D mask by associating the first mask and the second mask. The method includes determining a location of the 3D mask. The method includes generating the ray projection of the 3D mask from a perspective of a second object.