Abstract: The technology disclosed relates to a method of realistic rendering of a real object as a virtual object in a virtual space using an offset in the position of the hand in a three-dimensional (3D) sensory space. An offset between expected positions of the eye(s) of a wearer of a head mounted device and a sensor attached to the head mounted device for sensing a position of at least one hand in a three-dimensional (3D) sensory space is determined. A position of the hand in the three-dimensional (3D) sensory space can be sensed using a sensor. The sensed position of the hand can be transformed by the offset into a re-rendered position of the hand as would appear to the wearer of the head mounted device if the wearer were looking at the actual hand. The re-rendered hand can be depicted to the wearer of the head mounted device.

Abstract: An imaging apparatus, not including a display that displays setting information for operation setting in the imaging apparatus, includes: an image sensor that captures a subject image to generate image data; an output interface connected to an external apparatus to output the image data; and a controller that controls an output setting for outputting the image data in a predetermined format from the output interface to the external apparatus. In a state where the output setting is applied, no setting information for canceling the output setting is displayed on the external apparatus serving as an output destination of the image data in the predetermined format by the output interface. The imaging apparatus further comprises an input interface that receives an instruction to cancel the output setting from the state where the output setting is applied.

Abstract: A mobile computing device includes a touchscreen configured to display a user interface for receiving user input, memory hardware configured to store computer-executable instructions and information regarding scheduled conference calls, and processor hardware configured to execute the computer-executable instructions. The computer-executable instructions include communicating with a server via one or more wired/and or wireless communication networks, and parsing the information stored on the mobile computing device according to a parsing protocol to obtain information regarding at least one scheduled conference call. The instructions further include, in response to receiving user input via the user interface, transmitting at least a portion of the obtained information regarding the at least one scheduled conference call to the server to control a system controller in communication with the server to initiate the at least one scheduled conference call via a videoconferencing codec.

Abstract: A display apparatus includes a communication interface configured to communicate with another display apparatus, a display configured to display contents being shared with the other display apparatus and a video call user interface (UI) for a video call with a user of the other display apparatus, and a processor, in response to at least one of a gesture and a voice of the user included in video call data received from the other display apparatus satisfying a predetermined condition, configured to control the display to change a size of the video call UI displayed on the display.

Abstract: A lighting fixture is auto-focused on a person in a venue who is wearing, or carrying, or holding, or speaking into a microphone at the venue by detecting when a microphone having a real-time location system (RTLS) affixed thereto is active, identifying the position of RTLS tag, and auto-focusing the lighting fixture on the identified position of the RTLS tag.

Abstract: Techniques are described for applying virtual makeup products. Virtual makeup products, in this context, may include a visual effect and a configuration for the effect, and the visual effect includes one or more layered visual filters. In an example, a system may acquire a base image of a face, acquire a defined region of the face over which to apply a selected virtual makeup product, and generate a makeup image including one or more layered and masked filters based on the selected virtual makeup product and the defined region of the face. After generating the makeup image, a composite image including the makeup image overlaid on the base image can be displayed to simulate the application of a real-world makeup product.

Abstract: A system or method executing an intelligent videoframe privacy monitoring management system comprising a processor to execute code instructions of a multimedia multi-user collaboration application to join a videoconference session, a display screen, a speaker, a video camera, and a microphone where the video camera captures a videoframe of a user or background to be transmitted in the videoconference session and the processor to determine from input videoframe data and the detected sensor distance data of the user or the objects or inputting these into the intelligent videoframe privacy monitoring management system to generate optimized videoframe privacy adjustments to apply a privacy alteration to captured videoframes when the user is detected as absent, other faces, or background movement is detected.

Abstract: The subject technology receives, at a client device, a selection of a selectable graphical item from a plurality of selectable graphical items, the selectable graphical item comprising an augmented reality content generator including a 3D effect. The subject technology applies, to image data and depth data, the 3D effect based at least in part on the augmented reality content generator, the applying the 3D effect. The subject technology generates a depth map using at least the depth data, generates a segmentation mask based at least on the image data, and performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect.

Abstract: A method allows a receiving device to control aspects of an image stream transmitting (IST) device. The method includes receiving a segmented video stream from the IST device during a video communication session. The segmented video stream includes a primary segment and a secondary segment presenting a virtual control interface. The method includes presenting, on a display device, a control enabling view of the segmented video stream and monitoring for movement, captured by at least one image capturing device, that is indicative of the input occurring within a spatial location proximate to the selectable feature. The method includes in response to detecting the input, identifying a function associated with the selectable feature and activating the function to affect one or more characteristics of a video image presented within the primary segment.

Abstract: A video conferencing system and a method of removing an interruption thereof are provided. The method includes the following steps. A video conference is activated and a video stream is obtained through an image capturing device. A deep learning model is used to detect at least one first image object in a first video frame of the video stream. Whether the at least one first image object is an interruption object is determined. The at least one first image object is removed from the first video frame in response to the at least one first image object being determined to be the interruption object.

Abstract: A conferencing system terminal device includes an image capture device capturing images of a subject during a videoconference occurring across a network and a communication device transmitting the images to at least one remote electronic device engaged in the videoconference. One or more processors apply a virtual background or virtual background indicia in the images behind depictions of the subject. In response to one or more sensors detecting initiation of a demonstration operation by the subject, the one or more processors cause the virtual background or virtual background indicia to appear behind depictions of a demonstration object of the demonstration operation such that the demonstration object is visible against the virtual background indicia or virtual background.

Abstract: An electronic device is described. In some embodiments, the electronic device includes instructions for: while presenting an extended reality environment, receiving, by the first electronic device, a request to present a virtual representation of a remote participant of a communication session, where the first electronic device is connected to the communication session; obtaining a capability of the remote participant of the communication session; and in response to receiving the request to present the virtual representation of the remote participant of the communication session, presenting the virtual representation of the remote participant of the communication session based on the obtained capability of the remote participant of the communication session.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media provide for composite video with live annotations within a video communications platform. First, the system receives video content depicting imagery of a first user, with the video content having multiple video frames. The system then determines a boundary about the first user in the video frames, with the boundary having an interior portion and an exterior portion. The system provides a media background for the exterior portion, then generates a composite video for display on client devices, with the composite video depicting the media background in a first layer and the first user in a second layer. The system then receives one or more annotation inputs corresponding to the composite video. The annotation inputs can include media-based and/or messaging-based annotations. Finally, the system updates the composite video to additionally depict the annotation inputs within a third layer.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media related to multi-stream video encoding for screen sharing a communications session. The system may determine a sub-area and a remaining area of a display region. A first video stream of the sub-area is generated at a first frame rate. A second video stream of the remaining area is generated at second frame rate, where the second frame rate is a frame rate lower than the first frame rate. A client device may transmit the first video stream and the second video stream to a second client device.

Abstract: A user equipment, media control unit, media resource function, or another device or function capable of receiving, manipulating, and transmitting data may be configured to: receive an omnidirectional video; determine a viewport of a user equipment; determining a delivery mode; determine a region of the omnidirectional video based, at least partially, on the determined viewport and the determined delivery mode; encode the determined region; packetizing the encoded region; and transmit the packetized region to the user equipment based, at least partially, on the determined delivery mode.

Abstract: This disclosure enables various technologies for screensharing to minimize visual distractions, maximize professionalism, or avoid sharing sensitive, confidential, classified, or private information.

Abstract: A computer stores a reference image representing a physical background within a field of view of a camera of a client device. The computer receives, via the camera and during a video conference to which the client device is connected, camera-generated visual data for output to at least one remote device connected to the video conference. The computer identifies, based on facial recognition applied to the camera-generated visual data, foreground imagery representing at least one person and background imagery representing content of the camera-generated visual data other than the foreground imagery. The computer identifies a difference between the background imagery and the reference image. The computer generates a composite image by replacing, within the background imagery of the camera-generated visual data, an item represented within the background imagery and within the identified difference with a co-located part of the reference image.

Abstract: A processing system having at least one processor may establish a communication session between a first communication system of a first user and a second communication system of a second user, the communication session including first video content of a first physical environment of the first user and second video content of a second physical environment of the second user, determine a first visualization action for a first physical object in the first physical environment in accordance with a first configuration setting of the first user for the communication session, obtain the first video content from a first camera of the first communication system, detect the first physical object in the first video content, and perform the first visualization action to modify the first video content. The processing system may then transmit first visualization information including the first video content that is modified to the second communication system.

Abstract: Controlling the illumination at an endpoint to a communication session by transmitting, over the communication session, the color of the light to be emitted by the display of the receiving endpoint. In a particular embodiment, a method includes, during a video communication session between a first endpoint operated by a first user and a second endpoint operated by a second user, transmitting first video, comprising a solid color image, from the first endpoint to the second endpoint over the video communication session, wherein the first video is prominently displayed at the second endpoint. At the first endpoint, the method provides receiving second video from the second endpoint. The second video is captured at the second endpoint while the first video is being displayed at the second endpoint. The method also includes displaying the second video to the first user.

Abstract: Techniques for video content based on multiple capture devices are described and are implementable to enable multiple video capture devices to be utilized for a video feed. Generally, the described implementations enable video content captured by multiple video capture devices to be utilized, such as to integrate different instances of video content into a merged video content stream. In at least one implementation this provides higher quality video attributes to be utilized than is provided by utilizing a single video content source.

Abstract: An information board is displayed and used in a three-dimensional virtual reality environment wherein the information board is extended by adding discrete board segments through control inputs received from a user in the virtual reality environment.

Abstract: An online system customizes video conversations between users of the online system. During a video conversation, the online system presents a composite view to the participating users. The composite view may include visual representations of the users, a background graphic, or other types of graphics such as masks and props that the users can wear or interact with in the environment of the video conversation. The visual representations may be generated based on a live video feed of the users or include avatars of the users. The online system can determine the graphics based on information about the users. For instance, the online system determines a background graphic showing a location that the users have each visited. Upon viewing the background graphic, the users may be encouraged to interact with the background graphic or other graphics included in the composite view, which can promote an engaging video conversation experience.

Abstract: Methods and apparatus for performing virtual relocation during a network conference. In an embodiment, a method is provided for relocating a conference participant during a network conference. The method includes receiving, at a first conference participant, video streams from one or more additional conference participants, and identifying a video stream of a selected conference participant. The method also includes transmitting a request to the selected conference participant to remove a background from the video stream of the selected conference participant to generate a modified video stream, and receiving the modified video stream from the selected conference participant. The method also includes relocating the modified video stream in a background of the first conference participant, and transmitting an instruction for relocating the modified video stream within the background of the first conference participant.

Abstract: A system includes a common display, a display computer to run collaboration software connected to the common display that drives the common display, the display computer being on a first network, a first mobile device to run a sharing application and a streaming application, the first mobile device being on a second network, separate from the first network, the streaming application to convert a display of the mobile device into stream data, a control channel between the mobile device and the display computer, and a stream channel between the display computer and the mobile device. The mobile device sends stream data directly to the display computer, wherein the display computer is to display the stream data on the common display. The stream channel may be directly between the mobile device and the display computer or may be over a relay server.

Abstract: Described herein are technologies related to correcting image degradations in images. An image is received, and values for features that are usable to correct for image degradation associated with blur, noise, and low light conditions are generated by separate encoders based upon the received image. A fusion network learned by way of network architecture search fuses these values, and the fused values are employed to generate an improved image, such that the image degradations associated with blur, noise, and low light conditions are simultaneously corrected.

Abstract: Technology is provided for identifying synthesized conversation features from recorded conversations. The technology can identify, for each of one or more utterances, data for multiple modalities, such as acoustic data, video data, and text data. The technology can extract features, for each particular utterance of the one or more utterances, from each of the data for the multiple modalities associated with that particular utterance. The technology can also apply a machine learning model that receives the extracted features and/or previously synthesized conversation features and produces one or more additional synthesized conversation features.

Abstract: Methods, devices, and systems related to a computing device for capturing an augmented reality (AR) image displaying an AR are described. An example method can include projecting a structured array of light, from a mobile device, onto an object. The method further includes measuring multi-directional sections of the object with the projected light. The method further includes reconstructing a three-dimensional (3D) figure of the object based on the measured multi-directional sections. The method further includes displaying an augmented reality (AR) image associated with the 3D figure on a user interface of the mobile device.

Abstract: Described herein are systems and methods for dynamically adjusting an aspect ratio of a video. Exemplary methods can include receiving (i) a video having an original aspect ratio and at least one user interface (UI) element configured to be selected by a user of the video and (ii) an aspect ratio of a display screen for presenting the video. The methods can include automatically determining a display area of the video to be presented based on (i) the default position of the at least one UI element in the video, (ii) an active area in the video, and/or (iii) a central area in the video, the display area having an aspect ratio equal to the aspect ratio of the display screen; and presenting the video display area in the display screen with the at least one UI element for at least the portion of the video.

Abstract: A non-transitory computer-readable storage medium storing a program that causes a computer to execute a process, the process includes calculating second information indicating a line of sight of the second user in the call performing period based on second user images displayed on the first display; estimating a display region of the first user images in a screen of a second display in the second display that displays the content and the first user images, based on a difference between the line of sight of the second user in the first period and the line of sight of the second user in the second period indicated in the second information; and estimating whether the second user looks at the first user images displayed on the second display in the call performing period based on the estimated display region of the first user images and the second information.

Abstract: An imaging apparatus includes: an imager configured to capture a subject image to generate image data; an audio input device configured to receive audio data indicating sound during; a detector configured to detect a subject and a its type based on the image data generated by the imager; an audio processor configured to process the audio data received by the audio input device based on the type of subject detected by the detector; and an operation member configured to set a target type to be processed by the audio processor among a plurality of types including first and second types different from each other, based on a user operation, wherein the audio processor is configured to process the audio data to emphasize or suppress specific sound corresponding to the target type in audio data received when a subject of the target type is detected in the image data.

Abstract: A stereoscopic display device includes a case, a transparent display screen and one or more light sources. The case includes a chamber, one side of which is a front plate, the front plate is transparent, the transparent display screen is received inside the chamber and directly faces the front plate, the light sources are arranged inside the chamber and give off light towards the transparent display screen, and a picture displayed on the transparent display screen forms a reflection at the transparent display screen or at a bottom of the chamber.

Abstract: A training image to be used in training a learning network is generated. The method of generating the training image includes steps of: (a) a labeling device, in response to acquiring an original image, (i) inputting the original image into an image recognition network to detect privacy-related regions from the original image, (ii) adding dummy regions, different from the detected privacy-related regions, onto the original image, and (iii) setting the privacy-related regions and the dummy regions as obfuscation-expected regions which represent regions to be obfuscated in the original image; (b) the labeling device generating an obfuscated image by obfuscating the obfuscation-expected regions; and (c) the labeling device labeling the obfuscated image to be corresponding to a task of the learning network to be trained, to thereby generate the training image to be used in training the learning network.

Abstract: The disclosure provides a smart strap and a method for defining human posture. The smart strap includes a first shoulder strap, a second shoulder strap, a chest strap, a camera, and a microcontroller. Two ends of the chest strap are respectively connected to the first shoulder strap and the second shoulder strap, and the chest strap includes a stiffness segment. The camera is disposed on a surface of the stiffness segment, faces a first direction, and captures object images of an object on the first direction. The microcontroller receives the object images from the camera, detects a distance between the object and the camera based on the object images, and adjust a screen shown to a wearer of the smart strap according to the distance.

Abstract: Technology is provided for causing a computing system to extract conversation features from a multiparty conversation (e.g., between a coach and mentee), apply the conversation features to a machine learning system to generate conversation analysis indicators, and apply a mapping of conversation analysis indicators to actions and inferences to determine actions to take or inferences to make for the multiparty conversation. In various implementations, the actions and inferences can include determining scores for the multiparty conversation such as a score for progress toward a coaching goal, instant scores for various points throughout the conversation, conversation impact score, ownership scores, etc. These scores can be, e.g., surfaced in various user interfaces along with context and benchmark indicators, used to select resources for the coach or mentee, used to update coach/mentee matchings, used to provide real-time alerts to signify how the conversation is going, etc.

Abstract: A method enables a plurality of users having communication terminals to communicate via a virtual communication space provided by a communication server on a communication network. The method includes configuring a server application so as to create a virtual communication space for a determined group of users. The virtual space provides a plurality of distinct communication modes referred to as “activities”, each activity relating to at least one associated multimedia content, referred to as “activity content” that is shared by the users of the group. Each activity is performed by determined actions that can be triggered by any of the users of the group. The actions includes at least creating, modifying, or deleting a given activity content. Action requests relating to an activity under consideration and coming from the terminals of the users of the group are processed in a sequential manner in the order of their arrival.

Abstract: A system may comprise a main host device; a first splitter having a first input port, a first output port, and a second output port, the first input port configured for electronic communication with the main host device; a first local display monitor configured for electronic communication with the first output port of the first splitter; a first video capture device configured for electronic communication with the second output port of the first splitter; a multi-cam switch having a plurality of input ports and an outlet port; a second splitter having a second input port, a third output port, and a fourth output port, the outlet port of the multi-cam switch configured for electronic communication with the second input port; and a second video capture device configured for electronic communication with the third output port.

Abstract: A system and method are provided for managing a mobile device to selectively act as a two-party endpoint in a videoconference with a remote party. The disclosed technique, by way of example, entails receiving first user video and audio from a first camera and mic located on the front of the mobile device and receiving second user video and audio from a second camera and mic located on the back of the device. Remote user video and audio are received at the device, and the remote user video and audio, and second user video and audio are conveyed via a first display screen and first speaker located on the front of the device. Similarly, the remote user video and audio, and first user video and audio are conveyed via a second display screen and second speaker located on the back of the device.

Abstract: Methods and systems of establishing a shared augmented reality session between a first computing device and a second computing device are disclosed. The first and second computing devices perform proximity device discovery using a first data communication technology. The first and second computing devices then perform real-time network communication using a second data communication technology. Once the real-time network communication is established, the first and second computing devices perform coordinate space synchronization using the second data communication technology.

Abstract: One variation of a method for securing synthetic video conference feeds includes, during a setup period: accessing a target image depicting a face of a user; generating a face model based on a target set of facial features detected in the target image; and linking the face model to a target set of facial biometric values of the user. The method also includes, during an operating period succeeding the setup period: accessing a frame; deriving characteristics of a set of facial features detected in the frame; extracting a set of facial biometric values from the frame; in response to alignment between the target set of facial biometric values and the set of facial biometric values, generating a synthetic face image based on characteristics of the set of facial features, the face model, and a synthetic face generator; and rendering the synthetic face image in place of the frame.

Abstract: Disclosed herein is an immersive video processing method. The immersive video processing method may include classifying a multiplicity of view videos into a base view and an additional view, generating a residual video for the additional view video classified as an additional view, packing a patch, which is generated based on the residual video, into an atlas video, and generating metadata for the patch.

Abstract: An interactive narrative presentation includes a plurality of narrative segments, with a variety of available media content consumer selectable paths or directions, typically specified by a director or editor. The content consumer can select a path or path segment at each of a number of points, e.g., decision points, in the narrative presentation via user interface elements or narrative prompts, providing the consumer the opportunity to follow a storyline they find interesting. Each consumer follows a “personalized” path through the narrative. The narrative prompts can include images from subsequent segments, may be generated to stylistically resemble the underlying narrative presentation, may be animated to move with or with respect to the underlying narrative presentation, and, or may be presented in overlaid with a presentation of the underlying narrative while the underlying narrative continues play.

Abstract: Disclosed is a collaboration system that synchronizes what is seen and heard by all participants to a conference such that participant input provided during the conference are seen and heard with respect to the same state of a presented asset. Additionally, the collaboration system may monitor the conference, may extract a first participant contribution prior to presentation of a first asset in the conference and a second participant contribution that is made during a particular interval of the conference when the first asset is presented. The collaboration system may link the second participant contribution to the first asset, may receive a request for the first asset after the conference has concluded, and may provide the first asset with the second participant contribution in response to the request.

Abstract: The present disclosure generally provides for advanced single camera video conferencing systems and methods related thereto. The advanced single camera video conferencing system features a hybrid optical/digital camera, herein a camera device, having a controller that is configured to execute one or more of the methods set forth herein. In one embodiment, a method includes optically framing a first portion of a video conferencing environment to provide an actual field-of-view, digitally framing a second portion of the video conferencing environment to provide an apparent field-of-view that is encompassed within the actual field-of-view, generating a video stream of the apparent field-of-view, surveying the actual field-of-view to generate survey data, and detecting changes in the survey data over time. The method may be performed using a single camera device using a single image sensor.

Abstract: A method, computer system, and computer program product are provided for virtual background replacement during a video communication session. A frame comprising a user image and a captured background is obtained from a video capture device as part of a video stream. The frame is processed to replace the captured background with a virtual background. A change to an exposure characteristic of the video capture device is determined. In response to determining the change to the exposure characteristic, the virtual background is modified to match a virtual background brightness with a user image brightness to produce a modified virtual background. The frame is modified by combining the user image with the modified virtual background.

Abstract: One example system for providing notifications to videoconferencing sub-meetings includes a processor and at least one memory device. The processor establishes a videoconferencing session associated with a host. The videoconferencing session can include a main meeting, a sub-meeting, and a waiting queue. The processor can move the host from the main meeting to the sub-meeting, and subsequently move a new participant into the waiting queue. The processor can generate a notification that the participant has entered the waiting queue and determine that the host has moved to the sub-meeting. The processor can route, to the host in the sub-meeting using a stored connection of the host to the main meeting, a notification of the participant entering the waiting queue, and subsequently allow the host to admit the new participant.

Abstract: An information processing apparatus, an information processing method, and a program that allow an intuitive grasp of an object indicated by a communication partner in communication through an image are provided. The information processing apparatus includes a control unit that controls interactive communication to display, on a display unit, a captured image of a space on a communication partner side acquired from the communication partner side, the control unit controlling to display an image linking a pointer used by a person on one side to an object indicated by the person.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for liveness detection are provided. One of methods includes: displaying an image to a user, and capturing a face image of the user while displaying the image to the user; determining an eye image of an eye of the user based on the face image; extracting a to-be-verified image from the eye image of the user, wherein the to-be-verified image is reflection of the displayed image in the eye of the user; comparing the displayed image with the to-be-verified image to determine whether the to-be-verified image matches the displayed image; and performing liveness detection on the user based on a result of comparison.

Abstract: A video conference system including a first transmitter device and a receiver device is provided. The first transmitter device transmits a first video data. The receiver device includes a receiver control unit configured to assign a first identification code to the first transmitter device. The receiver control unit combines the first video data and a first identification image corresponding to the first identification code as a first combined video data and the receiver control unit outputs the first combined video data to the display device. Through the present invention, the conference participants can quickly identify which presenter is providing the video signal source corresponding to a particular frame on the split screen, and can raise questions to the right person.

Abstract: Provided are a method, a system, and a non-transitory computer readable record medium for enhancing a video quality of a video call. A video call method may enhance and thereby forward a picture quality of a main object or a main portion in a video and thereby enhance a picture quality of experience of a recipient of a video call.

Abstract: One variation of a method for video conferencing includes, during a setup period: accessing a test video feed; and generating a face model, representing facial characteristics of a first user depicted in the test video feed, based on features detected in the test video feed. The method also includes, during an operating period: accessing a video feed; representing constellations of facial landmarks, detected in frames in the video feed, in a feed of facial landmark containers; representing sets of facial muscle actions, detected in frames in the video feed, in a feed of facial expression containers; and transforming the feed of facial landmark containers, the feed of facial expression containers, and the face model into a feed of synthetic face images according to the synthetic face generator.