Abstract: A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

Abstract: Implementations provide an in-person communication experience by generating a changable virtual viewpoint for a participant in an online communication. For instance, techniques described herein capture visual data about participants in an online communication, and create a realistic geometric proxy from the visual data. A virtual scene geometry is generated that mimics an arrangement of an in-person meeting for the online communication. According to various implementations, a virtual viewpoint displays a changing virtual viewpoint, such as based on a change in position of a participants face.

Abstract: The description relates to remote collaboration via a telepresence experience. One example can include an interactive digital display. The example can also include a virtual user presentation component configured to generate a graphical user interface that includes a virtual representation of a remote user on the interactive digital display. The graphical user interface can be configured to present the remote user in a side by side or mirror image relationship to a local user of the interactive digital display.

Abstract: Image processing for productivity applications is provided. An image may be received by a computing device. The computing device may detect the edges comprising the received image and adjust the image based on a skew state of the detected edges. The computing device may then process the adjusted image to correct imbalances. The computing device may then assign an image classification to the processed image. The computing device may then adjust the processed image based on the assigned image classification.

Abstract: Examples are disclosed herein that are related to gaze tracking via image data. One example provides, on a gaze tracking system comprising an image sensor, a method of determining a gaze direction, the method comprising acquiring image data via the image sensor, detecting in the image data facial features of a human subject, determining an eye rotation center based upon the facial features using a calibrated face model, determining an estimated position of a center of a lens of an eye from the image data, determining an optical axis based upon the eye rotation center and the estimated position of the center of the lens, determining a visual axis by applying an adjustment to the optical axis, determining the gaze direction based upon the visual axis, and providing an output based upon the gaze direction.

Abstract: A perspective-correct communication window system and method for communicating between participants in an online meeting, where the participants are not in the same physical locations. Embodiments of the system and method provide an in-person communications experience by changing virtual viewpoint for the participants when they are viewing the online meeting. The participant sees a different perspective displayed on a monitor based on the location of the participant's eyes. Embodiments of the system and method include a capture and creation component that is used to capture visual data about each participant and create a realistic geometric proxy from the data. A scene geometry component is used to create a virtual scene geometry that mimics the arrangement of an in-person meeting. A virtual viewpoint component displays the changing virtual viewpoint to the viewer and can add perceived depth using motion parallax.

Abstract: The described implementations relate to enhancement images, such as in videoconferencing scenarios. One system includes a poriferous display screen having generally opposing front and back surfaces. This system also includes a camera positioned proximate to the back surface to capture an image through the poriferous display screen.

Abstract: A vehicle has a camera system that captures video while the vehicle moves. The vehicle records the captured video and/or wirelessly transmits the captured video to a remote user device for playback. When the vehicle is moving, a coarse waypoint is identified and a trajectory is determined from the current location of the vehicle to the coarse waypoint that reduces (e.g., minimizes) sudden changes in direction of movement of the vehicle, reduces (e.g., minimizes) sudden changes in speed of the vehicle, and/or reduces (e.g., minimizes) sudden changes in acceleration of the vehicle by reducing (e.g., minimizing) jerk or snap of the vehicle trajectory. One or more fine waypoints along the trajectory are selected and the vehicle moves to the coarse waypoint along the trajectory by passing through those fine waypoints, resulting in smooth movement of the device that reduces or eliminates motion sickness for users viewing the captured video.

Abstract: In embodiments of collaborative camera viewpoint control for interactive telepresence, a system includes a vehicle that travels based on received travel instructions, and the vehicle includes a camera system of multiple cameras that each capture video of an environment in which the vehicle travels from different viewpoints. Viewing devices receive the video of the environment from the different viewpoints, where the video of the environment from a selected one of the viewpoints is displayable to users of the viewing devices. Controller devices that are associated with the viewing devices can each receive a user input as a proposed travel instruction for the vehicle based on the selected viewpoint of the video that is displayed on the viewing devices. A trajectory planner receives the proposed travel instructions initiated via the controller devices, and generates a consensus travel instruction for the vehicle based on the proposed travel instructions.

Abstract: A vehicle camera system captures and transmits video to a user device, which includes a viewing device for playback of the captured video, such as virtual reality or augmented reality glasses. A rendering map is generated that indicates which pixels of the video frame (as identified by particular coordinates of the video frame) correspond to which coordinates of a virtual sphere in which a portion of the video frame is rendered for display. When a video frame is received, the rendering map is used to determine the texture values (e.g., colors) for coordinates in the virtual sphere, which is used to generate the display for the user. This technique reduces the rendering time when a user turns his or her head (e.g., while in virtual reality) and so it reduces motion and/or virtual reality sickness induced by the rendering lag.

Abstract: In embodiments of immersive interactive telepresence, a system includes a vehicle that captures an experience of an environment in which the vehicle travels, and the experience includes audio and video of the environment. User interactive devices receive the audio and the video of the environment, and each of the user interactive devices represent the experience for one or more users who are remote from the environment. A trajectory planner is implemented to route the vehicle based on obstacle avoidance and user travel intent as the vehicle travels in the environment. The trajectory planner can route the vehicle to achieve a location objective in the environment without explicit direction input from a vehicle operator or from the users of the user interactive devices.

Abstract: Techniques for automatically connecting to a service controller are described herein. In one example, a service controller device includes a processor and a computer-readable memory storage device storing executable instructions that cause the processor to broadcast at least one of an access credential, connection information or an access credential hash embedded in an audio signal. The processor can also authenticate a client device based on a transmission of at least one of the connection information, the access credential, or the access credential hash from the client device to the client connector and transmit data to the client device in response to authenticating the client device.

Abstract: A tele-immersive environment is described that provides interaction among participants of a tele-immersive session. The environment includes two or more set-ups, each associated with a participant. Each set-up, in turn, includes mirror functionality for presenting a three-dimensional virtual space for viewing by a local participant. The virtual space shows at least some of the participants as if the participants were physically present at a same location and looking into a mirror. The mirror functionality can be implemented as a combination of a semi-transparent mirror and a display device, or just a display device acting alone. According to another feature, the environment may present a virtual object in a manner that allows any of the participants of the tele-immersive session to interact with the virtual object.

Abstract: The subject disclosure is directed towards eye gaze detection based upon multiple cameras and/or light sources along with an adaptive homography mapping model. Learning of the model includes compensating for spatially-varying gaze errors and head pose dependent errors simultaneously in a unified framework. Aspects including training the model of adaptive homography offline using simulated data at various head positions.

Abstract: The description relates to facial tracking. One example can include an orientation structure configured to position the wearable device relative to a user's face. The example can also include a camera secured by the orientation structure parallel to or at a low angle to the user's face to capture images across the user's face. The example can further include a processor configured to receive the images and to map the images to parameters associated with an avatar model.

Abstract: Image processing for productivity applications is provided. An image may be received by a computing device. The computing device may detect the edges comprising the received image and adjust the image based on a skew state of the detected edges. The computing device may then process the adjusted image to correct imbalances. The computing device may then assign an image classification to the processed image. The computing device may then adjust the processed image based on the assigned image classification.

Abstract: A tele-immersive environment is described that provides interaction among participants of a tele-immersive session. The environment includes two or more set-ups, each associated with a participant. Each set-up, in turn, includes mirror functionality for presenting a three-dimensional virtual space for viewing by a local participant. The virtual space shows at least some of the participants as if the participants were physically present at a same location and looking into a mirror. The mirror functionality can be implemented as a combination of a semi-transparent mirror and a display device, or just a display device acting alone. According to another feature, the environment may present a virtual object in a manner that allows any of the participants of the tele-immersive session to interact with the virtual object.

Abstract: The subject disclosure is generally directed towards eye gaze detection based upon both eyes being fixated on the same gaze location. In one aspect, an image including a subject's left and right eyes is captured, from which left and right glint and pupil center information are extracted. The left and right glint data and the left and right pupil data are used with left and right bias correction matrices to jointly determine a gaze location.

Abstract: Dynamic texture mapping is used to create a photorealistic three dimensional animation of an individual with facial features synchronized with desired speech. Audiovisual data of an individual reading a known script is obtained and stored in an audio library and an image library. The audiovisual data is processed to extract feature vectors used to train a statistical model. An input audio feature vector corresponding to desired speech with which the animation will be synchronized is provided. The statistical model is used to generate a trajectory of visual feature vectors that corresponds to the input audio feature vector. These visual feature vectors are used to identify a matching image sequence from the image library. The resulting sequence of images, concatenated from the image library, provides a photorealistic image sequence with facial features, such as lip movements, synchronized with the desired speech. This image sequence is applied to the three-dimensional model.

Abstract: The large display interaction implementations described herein combine mobile devices with people tracking to enable new interactions including making a non-touch-sensitive display touch-sensitive and allowing personalized interactions with the display. One implementation tracks one or more mobile computing device users relative to a large computer-driven display, and configures content displayed on the display based on a distance a given mobile computing device user is from the display. Another implementation personalizes user interactions with a large display. One or more mobile computing device users are tracked relative to a display. The identity of each of the one or more mobile computing device users is obtained. Content displayed on the display is configured based on a distance an identified mobile computing device user is from the display and the identity of the user that provides the content.