Abstract: Scrolling motion is detected within a video stream to output an indication of a scrolling motion vector for use in encoding a current picture of the video stream. A first line of pixels within a motion region of the current picture is identified. A second line of pixels matching the first line of pixels is identified within a last played picture of the video stream. The scrolling motion vector is determined based on a comparison of lines of pixels nearby the second line of pixels within the last played picture. The indication of the scrolling motion vector is then output for use in encoding the current picture.

Abstract: A best available video stream is determined for each of multiple conference participants within a conference room including multiple cameras based on scores determined for video streams obtained from the cameras. The scores are determined based on representations of the conference participants within the video streams, for example, based on percentages of conference participant faces visible within the video streams, directions of conference participant faces relative to the cameras, directions of eye gaze of the conference participants relative to the cameras, and/or degrees to which conference participant faces are obscured within the video streams. The best available video streams are output for rendering within separate user interface tiles of conferencing software.

Abstract: Systems and methods for generating virtual characters for video conferencing are provided. In an example, a computing device accesses a source human face model, a target human face model, and a source virtual character face model as well as feature curves marked on the source human face model and the source virtual character face model. The computing device deforms the source virtual character face model based on the source human face model and the target human face model to generate a target virtual character face model. Deforming the source virtual character face model includes minimizing a loss function that includes terms defined based on the feature curves to preserve features of the source virtual character face model on the target virtual character face model. The computing device further renders the target virtual character face model.

Abstract: A conference video stream is annotated with participant names determined using speech recognition. A computing device used with a video conference obtains voiceprints representing speech of individual conference participants invited to the video conference. The computing device obtains, as output of a speech recognition process performed based on the voiceprints, name information of a conference participant whose speech is represented within an audio segment captured during the video conference. The computing device outputs an annotation representing the name information to configure one or more participant devices connected to the video conference to display the name information proximate to a depiction of the conference participant within a video stream depicting the conference participant.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media related to multi-stream video encoding for screen sharing a communications session. The system may determine an active pixel area and a remaining pixel area of a video region. of a video region. A first video stream of the active pixel area is generated at a first frame rate. A second video stream of the remaining pixel area is generated at second frame rate, where the second frame rate is a frame rate lower than the first frame rate. A client device may transmit the first video stream and the second video stream to a second client device.

Abstract: Noise can be reduced in video frames by performing one or more techniques described herein. For example, a system can divide a video frame into a set of blocks, determine a variance associated with a selected block, generate first frequency residuals associated with the selected block, and determine whether the variance is below a predefined threshold. If the variance is below the predefined threshold, the system can adjust an offset to be used during a quantization operation from a first offset value to a second offset value. The system can then apply the quantization operation to the first frequency residuals using the second offset value, to thereby generate second frequency residuals. Using the second offset value during the quantization operation can result in the second frequency residuals having a larger number of zero-valued frequency residuals, which in turn can reduce noise with respect to the selected block.

Abstract: Video content within a video communication session of a video communication platform is received, the video content having multiple video frames. An appearance adjustment request is received from a client device associated with a user. A face region within the video content is detected. A smoothing process is applied to low gradient parts of the face region. The smoothing process is applied in a gradient. An edge-aware smoothing filter is applied to the face region to preserve facial feature structures while smoothing areas adjacent to the facial feature structures. A modified frame is generated based on the applying of the smoothing process and the applying of the edge-aware smoothing filter.

Abstract: This application provides network-based device connection methods and devices. One method includes: receiving a first broadcast message from a second device and a second broadcast message from a third device, in response to determining that a quantity of available Wi-Fi connections of the second device is not greater than zero, and the second device is not connected to the first device through Wi-Fi, determining, based on connected device information of the second device and available Wi-Fi connection information of the third device, that the third device is connected to the second device through Wi-Fi and that a quantity of available Wi-Fi connections of the third device is greater than zero, and establishing a Wi-Fi link between the first device and the third device for the third device to forward the first packet received from the first device to the second device through a Wi-Fi link between the third device and the second device.

Abstract: Techniques for method for filtering artifacts from digital video are disclosed. In an example, a method includes accessing a video stream that includes a block with sub-blocks. The method includes, for each candidate filter strength parameter of a set of candidate filter strength parameters associated with the block, filtering each sub-block using a filter having the candidate filter strength parameter, creating a filtered block by combining the filtered sub-blocks, and calculating, for the filtered block, a distortion error measurement between a corresponding source block and the filtered block. The method includes selecting the filter strength parameters associated with a lowest distortion error measurement of the distortion error measurements. The method includes applying the selected filter strength parameters to an in-loop deringing filter to generate a filtered block of video.

Abstract: A best available video stream is determined for each of multiple conference participants within a conference room including multiple cameras based on scores determined for video streams obtained from the cameras. The scores are determined based on representations of the conference participants within the video streams, for example, based on percentages of conference participant faces visible within the video streams, directions of conference participant faces relative to the cameras, directions of eye gaze of the conference participants relative to the cameras, and/or degrees to which conference participant faces are obscured within the video streams. The best available video streams are output for rendering within separate user interface tiles of conferencing software.

Abstract: Systems and methods for personalized realistic video generation. In one example, a client device joins a video conference. The client device accesses a source video clip including a set of source video frames related to a user associated with the client device. The client device receives source audio data related to the user. The client device generates target video data based on the set of source video frames and the source audio data using a trained video generator model. The client device streams the target video data during the video conference.

Abstract: In one example, a video encoder divides a superblock of a video frame into subblocks. The dividing includes dividing the superblock according to a first and second partitioning schemes to generate a first and second combination of subblocks, respectively. The number of non-zero residuals in each subblock in the first and second combinations of subblocks is less than a corresponding threshold set for a size of the subblock. The encoder determines the subblocks for the superblock as the first combination of subblocks based on a first cost value associated with the first combination of subblocks being lower than a second cost value associated with the second combination of subblocks. The encoder further encodes the superblock into a video bitstream representing the video by encoding the subblocks.

Abstract: A system for implementing augmented reality in connection with a conferencing platform obtains, via an imaging device, interaction space image information indicative of an interaction space. The system generates, based on the interaction space image information, virtual interaction space information corresponding to a virtual interaction space comprising a virtual three-dimensional (3D) structure representing the interaction space. The system obtains virtual object information for displaying a virtual object within the virtual interaction space. The system provides, for output, rendering information configured to cause a computing device to present, via a display device, the virtual object within the virtual interaction space.

Abstract: Fast parameter search for chroma from luma (CFL) intra prediction in video encoding is provided. A video encoder determines a value of a parameter of a linear CFL predictor for a coding block of a frame of a video. The determining includes dividing a search range within a value range of the parameter into three portions by a starting value: the starting value and values on two sides of the starting value. The encoder determines cost values of parameter values from the three portions and select the value of the parameter that corresponds to the lowest cost value. The encoder further encodes the coding block of the frame into a bitstream representing the video using the CFL predictor with the selected value of the parameter.

Abstract: A lighting setting for detecting a low light environment associated with a user in a video communication session is received from a client device associated with the user. Imagery of the user within at least one frame of video content of the video communication session is detected. An illumination of the at least one frame is modified based on the lighting setting, to obtain a modified frame, by diffusing light to achieve at least a semi-evenly lit frame. The modified frame can then be transmitted to other users of the video communication session.

Abstract: Noise can be reduced in video frames by performing one or more techniques described herein. For example, a system can divide a video frame into a set of blocks, determine a variance associated with a selected block, generate first frequency residuals associated with the selected block, and determine whether the variance is below a predefined threshold. If the variance is below the predefined threshold, the system can adjust an offset to be used during a quantization operation from a first offset value to a second offset value. The system can then apply the quantization operation to the first frequency residuals using the second offset value, to thereby generate second frequency residuals. Using the second offset value during the quantization operation can result in the second frequency residuals having a larger number of zero-valued frequency residuals, which in turn can reduce noise with respect to the selected block.

Abstract: Deriving in-loop filter parameters via training for video encoding is provided. A video encoder performs inter prediction for a frame in a set of frames of the video to generate prediction residuals for the frame. The inter prediction for the frame is performed based on a reconstructed frame in the set of frames filtered using an in-loop filter. The value of a parameter of the in-loop filter is determined by determining, for each candidate in-loop filter parameter value, a visual quality metric for a set of training frames in training video sequences filtered by the in-loop filter. The candidate in-loop filter parameter value that corresponds the highest visual quality metric can be selected as the value of the parameter of the in-loop filter. The video encoder further encodes the prediction residues of the frame and the parameter of the in-loop filter into a bitstream representing the video.

Abstract: In one example, a video encoder divides a superblock of a video frame into subblocks. The dividing includes dividing the superblock according to a first and second partitioning schemes to generate a first and second combination of subblocks, respectively. The number of non-zero residuals in each subblock in the first and second combinations of subblocks is less than a corresponding threshold set for a size of the subblock. The encoder determines the subblocks for the superblock as the first combination of subblocks based on a first cost value associated with the first combination of subblocks being lower than a second cost value associated with the second combination of subblocks. The encoder further encodes the superblock into a video bitstream representing the video by encoding the subblocks.

Abstract: Matched points are identified using at least a first camera and a second camera as a person moves within a physical space during a calibration of the physical space. The matched points may be identified between each pair of cameras. A physical space calibration matrix that is solved based on the matched points is used during a video conference to determine whether a first bounding box of a first person captured by the first camera and a second bounding box of a second person captured by the second camera identify a single conference participant within the physical space.

Abstract: Constructing a motion vector candidate list for inter-prediction in video coding is provided. A video encoder performs inter prediction for a frame of the video to generate prediction residuals. The inter prediction is performed using motion vectors (MVs) for blocks of the frame. To determine the motion vector of a block, the encoder determines an integer-valued MV for the block based on a reference frame and selects, based on the integer-valued MV, a subset of candidate MVs from a list of MVs including MVs of neighboring blocks. The encoder identifies a MV from the subset of candidate MVs for the block and searches in a neighborhood of the identified MV for a refined MV. The encoder calculates the prediction residuals for the block based on a reference block in the reference frame pointed by the refined MV and encodes the prediction residuals into a bitstream representing the video.