Abstract: Techniques for sign language control for a virtual meeting are disclosed. In an example method, a client device displays a control overlay over a displayed video stream of a hearing-impaired participant of a virtual meeting captured by the client device. The client device detects an interaction of a hand of the hearing-impaired participant with an element of the control overlay. The client device determines a control selection based on the interaction. The client device updates one or more settings for the virtual meeting based on the control selection.

Abstract: Techniques for implementing hybrid retrieval augmented generation for rich document queries using a large language model (“LLM”) are disclosed. In an example method, a computing system receives a query about documents in a vector database including terms. The computing system generates tokens based on the terms. The computing system determines a ranking of the documents using a probabilistic ranking method based on the tokens. The computing system outputs, to the vector database, a query to cause a vector similarity search based on an embedded representation of the query. The computing system receives, from the vector database, a ranking of the documents. The computing system determines top documents based on the ranking and the ranking. The computing system determines a knowledge graph based on the top documents. The computing system receives, from an LLM, a response to a prompt including the query, the top documents, and the knowledge graph.

Abstract: Systems and methods for generating virtual characters for video conferencing are provided. In an example, a computing device accesses a source human face model, a target human face model, and a source virtual character face model as well as feature curves marked on the source human face model and the source virtual character face model. The computing device deforms the source virtual character face model based on the source human face model and the target human face model to generate a target virtual character face model. Deforming the source virtual character face model includes minimizing a loss function that includes terms defined based on the feature curves to preserve features of the source virtual character face model on the target virtual character face model. The computing device further renders the target virtual character face model.

Abstract: Voicemail spam detection is performed based on content of voicemail messages. The content of an incoming voicemail message is compared to a spam template that includes a representation of a spam voicemail. Spam templates may be generated based on spam indications provided by users for voicemail messages they have received. User indications for sufficiently similar voicemail messages may be aggregated by maintaining a vote count for a spam template that reflects how many times a user has indicated a matching voicemail message is spam. A spam template may also include an occurrence count that reflects how many times voicemail messages matching a spam template have been detected in a telephony system. An incoming voicemail message may be compared to spam templates and, responsive to a match of content and/or a corresponding vote count or occurrence count meeting a condition, the voicemail message may be identified as spam.

Abstract: A conference video stream is annotated with participant names determined using speech recognition. A computing device used with a video conference obtains voiceprints representing speech of individual conference participants invited to the video conference. The computing device obtains, as output of a speech recognition process performed based on the voiceprints, name information of a conference participant whose speech is represented within an audio segment captured during the video conference. The computing device outputs an annotation representing the name information to configure one or more participant devices connected to the video conference to display the name information proximate to a depiction of the conference participant within a video stream depicting the conference participant.

Abstract: Conference item selection is automated by a server that retrieves one or more items from a data store based on a determination that the text includes one or more keywords or a change in the subject of the call. The keywords can include phrases. The retrieved items include one or more of scripts, articles, manuals, daily bulletins regarding a system state, or any resource that can be used to assist with a customer call or interaction. The software running on the server generates a user interface (UI) output based on the retrieve items, and transmits the UI output to an agent device. Software running on the agent device receives the UI output and displays the retrieved items on a display of the agent device.

Abstract: Methods and systems provide hybrid DSP-AI acoustic echo cancellation for virtual conferences. A linear acoustic echo cancelation (AEC) can be performed on an input audio signal to filter out linear echo present in the input audio signal and generate a first filtered audio signal. A level of nonlinear echo present in the first filtered audio signal can then be determined. When the level of nonlinear echo satisfies a threshold, an artificial intelligence (AI)-based nonlinear AEC can be performed on the first filtered audio signal to generate an AI-filtered audio signal. When the level of nonlinear echo does not satisfy the threshold, a digital signal processing (DSP)-based nonlinear AEC can be performed on the first filtered audio signal to generate a second filtered audio signal.

Abstract: Example methods and systems provide a system that stores, in a storage device, a file and a descriptor corresponding to the file. The descriptor includes a retention time for the file. The system also sets an archival status retention flag to prevent the deletion of the file from the storage device, notwithstanding the retention time set by the descriptor. The system can transmit a copy of the file to an archive, and when a copy of the file has been safely stored in the archive, the collaboration platform receives an input from the archive confirming the archival status of the file. The input is configured to clear the archival status retention flag. Once the flag has been cleared, the file will be deleted from the collaboration platform in accordance with the retention policy for the file.

Abstract: One example method includes receiving an audio stream comprising speech; generating, by automatic speech recognition (“ASR”) software, a plurality of hypotheses, each hypothesis comprising a transcription of a first portion of the speech; rescoring, using a first trained language model, each hypothesis of the plurality of hypotheses; and responsive to a first hypothesis not satisfying a threshold, generating and outputting, using a trained large language model (“LLM”), a final transcription based on the plurality of hypotheses.

Abstract: A system for providing off-the-record functionality is provided herein. The system may include a processor configured to execute processor-executable instructions stored in non-transitory computer-readable medium to establish a video conference having a plurality of participants, each participant of the plurality of participants exchanging a plurality of audio or video streams via the video conference. The processor may also be configured to receive, from a first client device associated with one of the plurality of participants, a first audio stream or a first video stream of the plurality of audio or video streams, and record the plurality of audio or video streams within a recording. The processor may also be configured to receive an off-the-record request to begin an off-the-record time period, and in response to the off-the-record request, prevent at least one of the first audio stream or the first video stream from being included in the recording.

Abstract: A foreground portion and a background portion are obtained from an image of a conference participant. Using a background replacement image, a replacement background portion corresponding to the background portion is obtained. An indication of an image portion that is included in the image and is outside the foreground portion is obtained. An output image that includes the replacement background portion, the foreground portion, and the image portion is generated such that the foreground portion and the image portion are visible in the output image. The output image is transmitted.

Abstract: A server receives, from a host device of a video conference, an identifier of a presenter device of the video conference and parameters for the video conference. The server receives, from the presenter device, at least one of presentation slides for the video conference or indicia of a trial performance of a presentation for the video conference. The server determines, using a language model, whether the at least one of the presentation slides or the indicia of the trial performance meet at least a subset of the parameters. The server transmits, to at least one of the host device or the presenter device, an output based on whether the at least one of the presentation slides or the indicia of the trial performance meet the at least the subset of the parameters.

Abstract: Automated content recommendation is used in conferencing. In one embodiment, a system receives a list of content recommendation actions. The system receives a number of utterances associated with the participants in real time. For each utterance, the system determines whether a prediction of relatedness is present between the utterance and one or more trigger phrases associated with a content recommendation action. Upon determining that a prediction of relatedness is present, the system performs the associated content recommendation action by transmitting, to one or more client devices, one or more pieces of content to be recommended.

Abstract: One example method includes receiving, during a virtual conference from a client device, a request to generate a real-time summary of the virtual conference, a plurality of client devices participating in the virtual conference and exchanging audio and video streams; receiving, during the virtual conference, a plurality of utterances generated by automatic speech recognition (“ASR”) of the audio streams; generating a group of consecutive utterances from the plurality of utterances; determining a segment based on the group of consecutive utterances; generating, using a large language model (“LLM”), a segment summary based on the segment; and providing the segment summary to the client device.

Abstract: Systems and methods for providing non-fungible tokens (NFTs) for virtual accessories during virtual meetings are provided herein. The system may include a non-transitory computer-readable medium; a communications interface; and a processor communicatively coupled to the non-transitory computer-readable medium and the communications interface. The processor may be configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to: receive, from a first client device, a request to use a virtual accessory during a virtual meeting; determine a NFT associated with the virtual accessory; determine, based on the NFT, whether the first client device is authorized to use the virtual accessory; and transmit, to the first client device, a response based on whether the first client device is authorized to use the virtual accessory.

Abstract: Systems and methods for providing prerecording user pronunciation information for video conferencing are described. In an example method, a computing system receives an audio stream of a pronunciation of one or more words captured using a voice recording application. The computing system associates the audio stream of the pronunciation with a user of a second client device. The computing system joins a video conference hosted by a video conference provider, the video conference comprising a plurality of client devices including the second client device. The computing system receives a first indication to play back the pronunciation. The computing system plays back the pronunciation.

Abstract: One example method includes receiving a set of communication records, the set of communication records representing one or more communications between a first person and a second person; receiving a user query, the user query comprising one or more constraints; generating a plurality of segments from the communication records, at least a subset of the plurality of segments based on the one or more constraints; selecting one or more segments based on the one or more constraints; generating one or more queries based on the one or more constraints; providing, to a trained large language model, the one or more selected segments and the one or more generated queries; and receiving, from the trained LLM, and outputting a multidimensional analysis of the set of communication records.

Abstract: A system can automatically adjust a virtual background used by a participant in a video conference to match lighting level conditions present in a foreground of a participant video stream captured at a participant device. The matching can be done by adjusting one or more virtual background parameters, including but not limited to, exposure, brightness, contrast, and ISO. In some implementations, the system can dynamically or automatically adjust the lighting levels of the virtual background based on changes that may occur to the lighting level of the foreground during the video conference.

Abstract: One example method includes joining, from a client device, a virtual meeting hosted by a virtual meeting provider, the virtual meeting comprising a plurality of display areas, each of the display areas associated with a one or more participants in the virtual meeting, displaying the virtual meeting on the display of the client device, receiving an eye tracking signal from an eye tracking sensor, the eye tracking signal associated with a first user of the client device, and determining, based at least in part on the eye tracking signal, a first display area of the plurality of display areas on which the first user is focused. The method further includes identifying a first audio stream associated with the first display area, and responsive to identifying the first audio stream, increasing a volume of the first audio stream.

Abstract: A device of a proposed participant of the video conference receives an invitation to a video conference. The device provides, via a display of the device and based on the invitation, a graphical user interface element comprising a first graphical user interface icon for accepting the invitation, a second graphical user interface icon for rejecting the invitation, and a third graphical user interface icon for requesting a recording of the video conference. The first graphical user interface icon, the second graphical user interface icon, and the third graphical user interface icon are displayed simultaneously within the graphical user interface element. The device receives receiving a selection of the third graphical user interface icon. The device transmits a signal requesting the recording in response to the selection.