Abstract: Video content within a video communication session of a video communication platform is received, the video content having multiple video frames. An appearance adjustment request is received from a client device associated with a user. A face region within the video content is detected. A smoothing process is applied to low gradient parts of the face region. The smoothing process is applied in a gradient. An edge-aware smoothing filter is applied to the face region to preserve facial feature structures while smoothing areas adjacent to the facial feature structures. A modified frame is generated based on the applying of the smoothing process and the applying of the edge-aware smoothing filter.

Abstract: This application provides network-based device connection methods and devices. One method includes: receiving a first broadcast message from a second device and a second broadcast message from a third device, in response to determining that a quantity of available Wi-Fi connections of the second device is not greater than zero, and the second device is not connected to the first device through Wi-Fi, determining, based on connected device information of the second device and available Wi-Fi connection information of the third device, that the third device is connected to the second device through Wi-Fi and that a quantity of available Wi-Fi connections of the third device is greater than zero, and establishing a Wi-Fi link between the first device and the third device for the third device to forward the first packet received from the first device to the second device through a Wi-Fi link between the third device and the second device.

Abstract: Techniques for method for filtering artifacts from digital video are disclosed. In an example, a method includes accessing a video stream that includes a block with sub-blocks. The method includes, for each candidate filter strength parameter of a set of candidate filter strength parameters associated with the block, filtering each sub-block using a filter having the candidate filter strength parameter, creating a filtered block by combining the filtered sub-blocks, and calculating, for the filtered block, a distortion error measurement between a corresponding source block and the filtered block. The method includes selecting the filter strength parameters associated with a lowest distortion error measurement of the distortion error measurements. The method includes applying the selected filter strength parameters to an in-loop deringing filter to generate a filtered block of video.

Abstract: A best available video stream is determined for each of multiple conference participants within a conference room including multiple cameras based on scores determined for video streams obtained from the cameras. The scores are determined based on representations of the conference participants within the video streams, for example, based on percentages of conference participant faces visible within the video streams, directions of conference participant faces relative to the cameras, directions of eye gaze of the conference participants relative to the cameras, and/or degrees to which conference participant faces are obscured within the video streams. The best available video streams are output for rendering within separate user interface tiles of conferencing software.

Abstract: Systems and methods for personalized realistic video generation. In one example, a client device joins a video conference. The client device accesses a source video clip including a set of source video frames related to a user associated with the client device. The client device receives source audio data related to the user. The client device generates target video data based on the set of source video frames and the source audio data using a trained video generator model. The client device streams the target video data during the video conference.

Abstract: In one example, a video encoder divides a superblock of a video frame into subblocks. The dividing includes dividing the superblock according to a first and second partitioning schemes to generate a first and second combination of subblocks, respectively. The number of non-zero residuals in each subblock in the first and second combinations of subblocks is less than a corresponding threshold set for a size of the subblock. The encoder determines the subblocks for the superblock as the first combination of subblocks based on a first cost value associated with the first combination of subblocks being lower than a second cost value associated with the second combination of subblocks. The encoder further encodes the superblock into a video bitstream representing the video by encoding the subblocks.

Abstract: A system for implementing augmented reality in connection with a conferencing platform obtains, via an imaging device, interaction space image information indicative of an interaction space. The system generates, based on the interaction space image information, virtual interaction space information corresponding to a virtual interaction space comprising a virtual three-dimensional (3D) structure representing the interaction space. The system obtains virtual object information for displaying a virtual object within the virtual interaction space. The system provides, for output, rendering information configured to cause a computing device to present, via a display device, the virtual object within the virtual interaction space.

Abstract: Fast parameter search for chroma from luma (CFL) intra prediction in video encoding is provided. A video encoder determines a value of a parameter of a linear CFL predictor for a coding block of a frame of a video. The determining includes dividing a search range within a value range of the parameter into three portions by a starting value: the starting value and values on two sides of the starting value. The encoder determines cost values of parameter values from the three portions and select the value of the parameter that corresponds to the lowest cost value. The encoder further encodes the coding block of the frame into a bitstream representing the video using the CFL predictor with the selected value of the parameter.

Abstract: A lighting setting for detecting a low light environment associated with a user in a video communication session is received from a client device associated with the user. Imagery of the user within at least one frame of video content of the video communication session is detected. An illumination of the at least one frame is modified based on the lighting setting, to obtain a modified frame, by diffusing light to achieve at least a semi-evenly lit frame. The modified frame can then be transmitted to other users of the video communication session.

Abstract: Noise can be reduced in video frames by performing one or more techniques described herein. For example, a system can divide a video frame into a set of blocks, determine a variance associated with a selected block, generate first frequency residuals associated with the selected block, and determine whether the variance is below a predefined threshold. If the variance is below the predefined threshold, the system can adjust an offset to be used during a quantization operation from a first offset value to a second offset value. The system can then apply the quantization operation to the first frequency residuals using the second offset value, to thereby generate second frequency residuals. Using the second offset value during the quantization operation can result in the second frequency residuals having a larger number of zero-valued frequency residuals, which in turn can reduce noise with respect to the selected block.

Abstract: Deriving in-loop filter parameters via training for video encoding is provided. A video encoder performs inter prediction for a frame in a set of frames of the video to generate prediction residuals for the frame. The inter prediction for the frame is performed based on a reconstructed frame in the set of frames filtered using an in-loop filter. The value of a parameter of the in-loop filter is determined by determining, for each candidate in-loop filter parameter value, a visual quality metric for a set of training frames in training video sequences filtered by the in-loop filter. The candidate in-loop filter parameter value that corresponds the highest visual quality metric can be selected as the value of the parameter of the in-loop filter. The video encoder further encodes the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.

Abstract: In one example, a video encoder divides a superblock of a video frame into subblocks. The dividing includes dividing the superblock according to a first and second partitioning schemes to generate a first and second combination of subblocks, respectively. The number of non-zero residuals in each subblock in the first and second combinations of subblocks is less than a corresponding threshold set for a size of the subblock. The encoder determines the subblocks for the superblock as the first combination of subblocks based on a first cost value associated with the first combination of subblocks being lower than a second cost value associated with the second combination of subblocks. The encoder further encodes the superblock into a video bitstream representing the video by encoding the subblocks.

Abstract: Matched points are identified using at least a first camera and a second camera as a person moves within a physical space during a calibration of the physical space. The matched points may be identified between each pair of cameras. A physical space calibration matrix that is solved based on the matched points is used during a video conference to determine whether a first bounding box of a first person captured by the first camera and a second bounding box of a second person captured by the second camera identify a single conference participant within the physical space.

Abstract: Constructing a motion vector candidate list for inter-prediction in video coding is provided. A video encoder performs inter prediction for a frame of the video to generate prediction residuals. The inter prediction is performed using motion vectors (MVs) for blocks of the frame. To determine the motion vector of a block, the encoder determines an integer-valued MV for the block based on a reference frame and selects, based on the integer-valued MV, a subset of candidate MVs from a list of MVs including MVs of neighboring blocks. The encoder identifies a MV from the subset of candidate MVs for the block and searches in a neighborhood of the identified MV for a refined MV. The encoder calculates the prediction residuals for the block based on a reference block in the reference frame pointed by the refined MV and encodes the prediction residuals into a bitstream representing the video.

Abstract: Methods and systems provide for applying a video effect to a video corresponding to a participant within a video communication session. The system displays a video for each of at least a subset of the participants and a user interface including a selectable video effects UI element. The system receives a selection by a participant of the video effects UI element. In response to receiving the selection, the system displays a variety of video effects options for modifying the appearance of the video and/or modifying a visual representation of the participant. The system then receives a selection by the participant of a video effects option, and further receives a subselection for customizing the amount of the video effect to be applied. The system then applies, in real time or substantially real time, the selected video effect in the selected amount to the video corresponding to the participant.

Abstract: Fast parameter search for chroma from luma (CFL) intra prediction in video encoding is provided. A video encoder determines a value of a parameter of a linear CFL predictor for a coding block of a frame of a video. The determining includes dividing a search range within a value range of the parameter into three portions by a starting value: the starting value and values on two sides of the starting value. The encoder determines cost values of parameter values from the three portions and select the value of the parameter that corresponds to the lowest cost value. The encoder further encodes the coding block of the frame into a bitstream representing the video using the CFL predictor with the selected value of the parameter.

Abstract: A unified virtual background image is generated for multiple participants of a video conference to create an immersive conference experience based on its use within video streams of those multiple participants. Generative artificial intelligence software associated with a conferencing system obtains input associated with a video conference. The generative artificial intelligence software generates a virtual background image based on the input.

Abstract: A virtual background image used with a video stream of a conference participant during a video conference is updated to visually represent one or more key points of the video conference. Generative artificial intelligence software associated with a conferencing system evaluates content of a video conference to determine one or more key points related to the video conference. The generative artificial intelligence software updates a virtual background image of a participant of the video conference to include one or more visual elements representing the one or more key points. The updated virtual background image is then output for use within a video stream of the participant during the video conference.

Abstract: A virtual background image is generated for a participant of a video conference and output for use within a video stream of the participant during the video conference. Generative artificial intelligence software associated with a conferencing system obtains, during a video conference, input associated with a participant of the video conference. The generative artificial intelligence software generates a virtual background image for the participant based on the input. The virtual background image is then output for use within a video stream of the participant during the video conference.

Abstract: Deriving in-loop filter parameters via training for video encoding is provided. A video encoder performs inter prediction for a frame in a set of frames of the video to generate prediction residuals for the frame. The inter prediction for the frame is performed based on a reconstructed frame in the set of frames filtered using an in-loop filter. The value of a parameter of the in-loop filter is determined by determining, for each candidate in-loop filter parameter value, a visual quality metric for a set of training frames in training video sequences filtered by the in-loop filter. The candidate in-loop filter parameter value that corresponds the highest visual quality metric can be selected as the value of the parameter of the in-loop filter. The video encoder further encodes the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.