Abstract: Distance-based framing includes obtaining at least a video stream during an online conference session. The video stream, an audio stream received with the video stream, or both the video stream and the audio stream are analyzed and a framing that either focuses on a speaker in the video stream or provides an overview of participants in the video stream, the framing being is composed based on the analyzing. A potential error in the framing is detected based on further analysis of at least one of the video stream, the audio stream, or distance sensor data received with the video stream. If the distance sensor data contradicts the potential error, the framing is maintained, but if the distance sensor data confirms the potential error, a new framing is generated.

Abstract: Distance-based framing includes obtaining at least a video stream during an online conference session. The video stream, an audio stream received with the video stream, or both the video stream and the audio stream are analyzed and a framing that either focuses on a speaker in the video stream or provides an overview of participants in the video stream, the framing being is composed based on the analyzing. A potential error in the framing is detected based on further analysis of at least one of the video stream, the audio stream, or distance sensor data received with the video stream. The potential error may be contradicted or confirmed based on an amount of motion when the framing focuses on the speaker. If the distance sensor data contradicts the potential error, the framing is maintained, but if the distance sensor data confirms the potential error, a new framing is generated.

Abstract: A method, computer system, and computer program product are provided for virtual background replacement during a video communication session. A frame comprising a user image and a captured background is obtained from a video capture device as part of a video stream. The frame is processed to replace the captured background with a virtual background. A change to an exposure characteristic of the video capture device is determined. In response to determining the change to the exposure characteristic, the virtual background is modified to match a virtual background brightness with a user image brightness to produce a modified virtual background. The frame is modified by combining the user image with the modified virtual background.

Abstract: A method, computer system, and computer program product are provided for virtual background replacement during a video communication session. A frame comprising a user image and a captured background is obtained from a video capture device as part of a video stream. The frame is processed to replace the captured background with a virtual background. A change to an exposure characteristic of the video capture device is determined. In response to determining the change to the exposure characteristic, the virtual background is modified to match a virtual background brightness with a user image brightness to produce a modified virtual background. The frame is modified by combining the user image with the modified virtual background.

Abstract: Distance-based framing includes obtaining at least a video stream during an online conference session. The video stream, an audio stream received with the video stream, or both the video stream and the audio stream are analyzed and a framing that either focuses on a speaker in the video stream or provides an overview of participants in the video stream, the framing being is composed based on the analyzing. A potential error in the framing is detected based on further analysis of at least one of the video stream, the audio stream, or distance sensor data received with the video stream. If the distance sensor data contradicts the potential error, the framing is maintained, but if the distance sensor data confirms the potential error, a new framing is generated.

Abstract: The present disclosure provides methods and systems related to automatic adjustment of screen brightness for optimized presentation to both physically present and remote audience during the multimedia collaboration session. In one aspect, a method includes detecting presence of a screen in the field of view of a camera in a meeting room; determining if exposure of the camera or brightness of the screen is to be adjusted, to yield a determination; and controlling at least one of the exposure of the camera or the brightness of the screen based on the determination such that viewing of meeting room and the screen are legible for one or more audience and the screen is legible for one or more audience present in the meeting room.

Abstract: The present disclosure provides methods and systems related to automatic adjustment of screen brightness for optimized presentation to both physically present and remote audience during the multimedia collaboration session. In one aspect, a method includes detecting presence of a screen in the field of view of a camera in a meeting room; determining if exposure of the camera or brightness of the screen is to be adjusted, to yield a determination; and controlling at least one of the exposure of the camera or the brightness of the screen based on the determination such that viewing of meeting room and the screen are legible for one or more audience and the screen is legible for one or more audience present in the meeting room.

Abstract: Systems, methods, and computer-readable media for automatically setting exposure levels in a session based on an active participant. In some embodiments, a method can include detecting faces of one or more participants in one or more images in a captured video feed at a location of the participants illuminated at one or more illumination levels at the location. The one or more detected faces can be associated with brightness levels of the one or more participants based on the one or more illumination levels at the location. Audio input can be received for the one or more participants at the location and a first participant can be identified as an active participant using the audio input. Further, an exposure level of the captured video feed can be set based on the first participant acting as the active participant according to a brightness level in the one or more images associated with a face detection of the first participant in the one or more images.

Abstract: Crop synchronized auto focus and exposure may be provided. First, a command corresponding to a new crop region of a sensor of a camera may be received. Then new capture settings corresponding to the new crop region may be obtained. Next, a frame may be captured from the sensor of the camera according to the obtained new capture settings. The captured frame may then be cropped according to the new crop region.

Abstract: Systems, methods, and computer-readable media for automatically setting exposure levels in a session based on an active participant. In some embodiments, a method can include detecting faces of one or more participants in one or more images in a captured video feed at a location of the participants illuminated at one or more illumination levels at the location. The one or more detected faces can be associated with brightness levels of the one or more participants based on the one or more illumination levels at the location. Audio input can be received for the one or more participants at the location and a first participant can be identified as an active participant using the audio input. Further, an exposure level of the captured video feed can be set based on the first participant acting as the active participant according to a brightness level in the one or more images associated with a face detection of the first participant in the one or more images.

Abstract: The present disclosure provides systems, methods, and computer-readable media for generating resolution based content to be streamed at various remote displaying devices. In one aspect, a device includes at least one processor and at least one memory having computer-readable instructions, which when executed by the at least one processor, configure the at least one processor to determine one or more streaming resolutions according to which a video stream is displayed at one or more receiving endpoints; generate a resolution based content of a video stream for each of the one or more streaming resolutions, each resolution based content being a different representation of an environment captured by the video stream from other resolution based contents; and transmit each resolution based content to a corresponding one of the one or more receiving endpoints for display thereon.

Abstract: A system that automatically calibrates multiple speaker tracking systems with respect to one another based on detection of an active speaker at a collaboration endpoint is presented herein. The system collects a first data point set of an active speaker at the collaboration endpoint using at least a first camera and a first microphone array. The system then receives a plurality of second data point sets from one or more secondary speaker tracking systems located at the collaboration endpoint. Once enough data points have been collected, a reference coordinate system is determined using the first data point set and the one or more second data point sets. Finally, after a reference coordinate system has been determined, the system generates the locations of the one or more secondary speaker tracking systems with respect to the first speaker tracking system.

Abstract: A video conference endpoint includes a camera to capture video and a microphone array to sense audio. One or more preset views are defined. Images in the captured video are processed with a face detection algorithm to detect faces. Active talkers are detected from the sensed audio. The camera is controlled to capture video from the preset views, and from dynamic views created without user input and which include a dynamic overview and a dynamic close-up view. The camera is controlled to dynamically adjust each of the dynamic views to track changing positions of detected faces over time, and dynamically switch the camera between the preset views, the dynamic overview, and the dynamic close-up view over time based on positions of the detected faces and the detected active talkers relative to the preset views and the dynamic views.

Abstract: A video conference endpoint includes one or more cameras to capture video of different views and a microphone array to sense audio. One or more closeup views are defined. The endpoint detects faces in the captured video and active audio sources from the sensed audio. The endpoint detects any active talker having detected face positions that coincide with detected active audio sources, and also uses speaker clustering to detect whether any active talker is associated with a previously stored closeup views. Based on whether an active talker is detected in any of the stored closeup views, the endpoint switches between capturing video of one of the closeup views and a best overview of the participants in the conference room.

Abstract: A video conference endpoint includes a camera to capture video and a microphone array to sense audio. One or more preset views are defined. Images in the captured video are processed with a face detection algorithm to detect faces. Active talkers are detected from the sensed audio. The camera is controlled to capture video from the preset views, and from dynamic views created without user input and which include a dynamic overview and a dynamic close-up view. The camera is controlled to dynamically adjust each of the dynamic views to track changing positions of detected faces over time, and dynamically switch the camera between the preset views, the dynamic overview, and the dynamic close-up view over time based on positions of the detected faces and the detected active talkers relative to the preset views and the dynamic views.

Abstract: A video conference endpoint includes one or more cameras to capture video of different views and a microphone array to sense audio. One or more preset views are defined. The endpoint detects faces in the captured video and active audio sources from the sensed audio. The endpoint detects any active talker detected faces that coincide positionally with detected active audio sources, and also detects whether any active talker is in one of the preset views. Based on whether an active talker is detected in any of the preset views, the endpoint switches between capturing video of one of the preset views, and capturing video of a dynamic view.

Abstract: A video conference endpoint includes one or more cameras to capture video of different views and a microphone array to sense audio. One or more preset views are defined. The endpoint detects faces in the captured video and active audio sources from the sensed audio. The endpoint detects any active talker detected faces that coincide positionally with detected active audio sources, and also detects whether any active talker is in one of the preset views. Based on whether an active talker is detected in any of the preset views, the endpoint switches between capturing video of one of the preset views, and capturing video of a dynamic view.