Abstract: A system that captures both whiteboard content and audio signals of a meeting using a digital camera and a microphone. The system can be retrofit to any existing whiteboard. It computes the time stamps of pen strokes on the whiteboard by analyzing the sequence of captured snapshots. It also automatically produces a set of key frames representing all the written content on the whiteboard before each erasure. The whiteboard content serves as a visual index to efficiently browse the audio meeting. The system not only captures the whiteboard content, but also helps the users to view and manage the captured meeting content efficiently and securely.

Abstract: The computer-readable media provides improved procedures to estimate head motion between two images of a face. Locations of a number of distinct facial features are determined in two images. The locations are converted into as a set of physical face parameters based on the symmetry of the identified distinct facial features. An estimation objective function is determined by: (a) estimating each of the set of physical parameters, (b) estimating a first head pose transform corresponding to the first image, and (c) estimating a second head pose transform corresponding to the second image. The motion is estimated between the two images based on the set of physical face parameters by multiplying each term of the estimation objective function by a weighted contribution factor based on the confidence of data corresponding to the estimation objective function.

Abstract: A videoconferencing conferee may be provided with feedback on his or her location relative a local video camera by altering how remote videoconference video is displayed on a local videoconference display viewed by the conferee. The conferee's location may be tracked and the displayed remote video may be altered in accordance to the changing location of the conferee. The remote video may appear to move in directions mirroring movement of the conferee. This effect may be achieved by modeling the remote video as offset and behind a virtual portal corresponding to the display. The remote video may be displayed according to a view of the remote video through the virtual portal. As the conferee's position changes, the view through the portal changes, and the remote video changes accordingly.

Abstract: A method and system for visually tracking a point of contact of an optical output from a computer input device includes an internal camera configured to visually track the point of contact of the optical output against a surface and an optical source to transmit the optical output from the computer input device. The camera also transmits the position of the point of contact as a computer input. In one form, the computer applies the position of the point of contact as an input to an application operating on the computer, such as a gaming application. In one form, the camera can visually track the movement of the computer input device along a surface.

Abstract: An “Image Segmenter” provides a variational energy formulation for segmentation of natural objects from images. In general, the Image Segmenter operates by adopting Gaussian mixture models (GMM) to capture the appearance variation of objects in one or more images. A global image data likelihood potential is then computed and combined with local region potentials to obtain a robust and accurate estimation of pixel foreground and background distributions. Iterative minimization of a “global-local energy function” is then accomplished by evolution of a foreground/background boundary curve by level set, and estimation of a foreground/background model by fixed-point iteration, termed “quasi-semi-supervised EM.” In various embodiments, this process is further improved by providing general object shape information for use in rectifying objects segmented from the image.

Abstract: The system provides improved procedures to estimate head motion between two images of a face. Locations of a number of distinct facial features are identified in two images. The identified locations can correspond to the eye corners, mouth corners and nose tip. The locations are converted into as a set of physical face parameters based on the symmetry of the identified distinct facial features. The set of physical parameters reduces the number of unknowns as compared to the number of equations used to determine the unknowns. An initial head motion estimate is determined by: (a) estimating each of the set of physical parameters, (b) estimating a first head pose transform corresponding to the first image, and (c) estimating a second head pose transform corresponding to the second image. The head motion estimate can be incorporated into a feature matching algorithm to refine the head motion estimation and the physical facial parameters.

Abstract: Multi-device capture and spatial browsing of conferences is described. In one implementation, a system detects cameras and microphones, such as the webcams on participants' notebook computers, in a conference room, group meeting, or table game, and enlists an ad-hoc array of available devices to capture each participant and the spatial relationships between participants. A video stream composited from the array is browsable by a user to navigate a 3-dimensional representation of the meeting. Each participant may be represented by a video pane, a foreground object, or a 3-D geometric model of the participant's face or body displayed in spatial relation to the other participants in a 3-dimensional arrangement analogous to the spatial arrangement of the meeting.

Abstract: A multimodal, multilanguage mobile device which can be employed to enhance note taking and/or annotation of a document, and gaming. Input data types such as optical character recognition (OCR), speech, handwriting, and visual information (e.g., image and/or video), etc., can be fused to generate rich documents with a multidimensional level of data to provide an increased level of context over conventional documents. Such architecture can be utilized by students for homework management, as well as entertainment (e.g., gaming).

Abstract: A multi-party spatial audio conferencing system is configured to receive far end signals from remote participants. The system comprises a speaker array that outputs spatialized sound signals and one or more microphones that capture and relay a sound signal comprising an echo of the spatialized sound signal to a multichannel acoustic echo cancellation (MC-AEC) unit having a plurality of echo cancellers. Respective echo cancellers perform cancellation of an echo associated with a far end signal from one of the multiple participants according to an algorithm based upon echo cancellation coefficients. The echo cancellation coefficients are determined from the input channel signals, the spatialization parameters associated with each input channel, and the audio signals captured by the microphones. This allows respective echo cancellation filters to be updated simultaneously even though the corresponding remote participant is not talking.

Abstract: A method and apparatus determine a likelihood of a speech state based on an alternative sensor signal and an air conduction microphone signal. The likelihood of the speech state is used, together with the alternative sensor signal and the air conduction microphone signal, to estimate a clean speech value for a clean speech signal.

Abstract: A video clip is processed by selecting a plurality of video frames of the video clip. A plurality of the pixels of the selected video frames are modified to form modified video frames. The modification to each of the plurality of the pixels is based on the intensity of the pixel, a change in the intensity of the pixel from the corresponding pixel in at least one related video frame, and the intensity of the corresponding pixel. A second video clip is formed that comprises the modified video clips.

Abstract: A system that facilitates automatically determining an action of an animate object is described herein. The system includes a receiver component that receives video data that includes images of an animate object. The system additionally includes a determiner component that accesses a data store that includes an action graph and automatically determines an action undertaken by the animate object in the received video data based at least in part upon the action graph. The action graph comprises a plurality of nodes that are representative of multiple possible postures of the animate object. At least one node in the action graph is shared amongst multiple actions represented in the action graph.

Abstract: A multimedia conference technique is disclosed that allows physically remote users to participate in an immersive telecollaborative environment by synchronizing multiple data, images and sounds. The multimedia conference implementation provides users with the perception of being in the same room visually as well as acoustically according to an orientation plan which reflects each remote user's position within the multimedia conference environment.

Abstract: Systems and methods for determining a virtual sound source position by determining an output for loudspeakers by the position of the loudspeakers in relation to a listener. The output of respective loudspeakers is generated using aural cues to give the listener knowledge of the virtual position of the virtual sound source. Both a gain in intensity and a delay are simulated.

Abstract: A panoramic camera is configured to automatically determine parameters of a table upon which the camera is situated as well as positional information of the camera relative to the table. In an initialization stage, table edges are detected to create an edge map. A Hough transformation-like symmetry voting operation is performed to clean up the edge map and to determine camera offset, camera orientation and camera tilt. The table is then fit to a table model to determine table parameters. In an operational stage, table edges are detected to create an edge map and the table model is fit to the edge map. The output can then be used for further panoramic image processing such as head size normalization, zooming, compensation for camera movement, etc.

Abstract: A framework for facilitating collaborative interaction between users of a web application is presented. The framework provides persistence and sharing mechanisms for arbitrary application-defined objects. The sharing mechanism is coupled with a consistency mechanism that keeps client states consistent even when users perform conflicting operations. This framework is application-independent in that the persistence, sharing between clients and maintaining a consistent state among the clients is accomplished without the need for specific knowledge of the design of the web application. Thus, web developers can focus on the logic of the application and the UI, instead of spending time designing and implementing collaborative capabilities.

Abstract: In a method for compressing a video clip containing audio content and image content, an image and/or an audio portion of individual video frames of the video clip are analyzed. Next frame scores are calculated for the video frames. Each frame score is based on at least one image attribute of the image of the video frame, and/or an audio attribute of the audio portion of the video frame. Next, key frames are identified that have a frame score that exceeds a threshold frame score. Finally, a compressed video clip is formed in which the images of non-key frames are removed. A system for implementing the method is also disclosed.

Abstract: A classifier is trained on a first set of examples, and the trained classifier is adapted to perform on a second set of examples. The classifier implements a parameterized labeling function. Initial training of the classifier optimizes the labeling function's parameters to minimize a cost function. The classifier and its parameters are provided to an environment in which it will operate, along with an approximation function that approximates the cost function using a compact representation of the first set of examples in place of the actual first set. A second set of examples is collected, and the parameters are modified to minimize a combined cost of labeling the first and second sets of examples. The part of the combined cost that represents the cost of the modified parameters applied to the first set is calculated using the approximation function.

Abstract: Videos are retargeted to a target display for viewing with little to no geometric distortion or video information loss. Salient regions of video frames may be determined using scale-space spatiotemporal information. Video information loss may be a result of spatial loss, due to cropping, and resolution loss, due to resizing. A desired cropping window may be determined using a coarse-to-fine searching strategy. Video frames may be cropped with a window that matches an aspect ratio of the target display, and resized isotropically to match a size of the target display.

Abstract: Systems, methods, computer-readable media, and graphical user interfaces for facilitating advertisement placement over video content are provided. Images within a video are partitioned into image regions. Upon partitioning images into image regions, an intrusiveness score is determined for each image region. Based on the intrusiveness scores, optimal placement of an advertisement within the video is determined.