Abstract: Provides a system for detecting an intersection between more than one panoramic video sequence and detecting the orientation of the sequences forming the intersection. Video images and corresponding location data are received. If required, the images and location data is processed to ensure the images contain location data. An intersection between two paths is then derived from the video images by deriving a rough intersection between two images, determining a neighborhood for the two images, and dividing each image in the neighborhood into strips. An identifying value is derived from each strip to create a row of strip values which are then converted to the frequency domain. A distance measure is taken between strips in the frequency domain, and the intersection is determined from the images having the smallest distance measure between them.

Abstract: An integrated method for modeling the handoff between cameras for tracking a specific individual, including: creating a representation of overlaps, gaps, and allowable movement among the fields of view of the cameras, wherein the representation is modeled as states in a Hidden Markov Model (HMM); training the HMM using video of people walking through the fields of view of the cameras; selecting a person to be tracked; and identifying the best camera area using the HMM.

Abstract: An audio device management system (ADMS) manages remote audio devices via user selections in video links. The system enhances audio acquisition quality by receiving and processing human suggestions, forming customized two-way audio links according to user requests, and learning audio pickup strategies and camera management strategies from user operations. The ADMS control interface for a remote user provides a multi-window GUI that provides an overview window and selection display window. The ADMS provides users with more flexibility to enhance audio signals according to their needs and makes it more convenient to form customized two-way audio links without requiring users to remember a list of phone numbers. The ADMS also automatically manages available microphones for audio pickup based on microphone sound quality and the system's past experience when users monitor a structured audio environment without explicitly expressing their attentions in the video window.

Abstract: Techniques for generating a storyboard are disclosed. In one embodiment of the invention the storyboard is comprised of videos from one or more cameras based on the identification of activity in the video. Various embodiments of the invention include an assessment of the importance of the activity, the creation of a storyboard presentation based on importance and interaction techniques for seeing more details or alternate views of the video. In one embodiment, motion detection is used to determine activity in one or more synchronized video streams. Periods of activity are recognized and assigned importance assessments based on the activity, important locations in the video streams, and events from other sensors. In different embodiments, the interface consists of a storyboard and a map.

Abstract: A method for exchanging information in a shared interactive environment, comprising selecting a first physical device in a first live video image wherein the first physical device has information associated with it, causing the information to be transferred to a second physical device in a second live video image wherein the transfer is brought about by manipulating a visual representation of the information, wherein the manipulation includes interacting with the first live video image and the second live video image, wherein the first physical device and the second physical device are part of the shared interactive environment, and wherein the first physical device and the second physical device are not the same.

Abstract: Systems and computer-implemented methods for use in body pose estimation are provided. Training data is obtained, where the training data includes observation vector data and corresponding pose vector data for a plurality of images. The observation vector data is representative of the images in observation space. The pose vector data is representative of the same images in pose space. Based on the training data, a model is computed that includes parameters of mapping from the observation space to latent space, parameters of mapping from the latent space to the pose space, and parameters of the latent space. The latent space has a lower dimensionality than the observation space and the pose space.

Abstract: Foreground segmentation in real world dynamic scenes, including under various lighting and shadow conditions, is disclosed. It may be used with one or multiple cameras for various automated tasks, such as classification of moving object, tracking moving objects, and event detection in various indoor or outdoor environments. Pixel to pixel subtraction is performed on each frame, followed by a feature-level based foreground segmentation to properly validate the foreground pixels. In this step, for each pixel in the image, a neighborhood of pixels is selected, and the aggregate change in the neighborhood image is used to classify foreground and background pixels. Normalized cross correlation is then applied to the neighborhood of each pixel that was confirmed to be foreground pixel.

Abstract: Passive wands, tracked by one or more cameras, are used as user interface devices. The wands are inexpensive, and are a natural way for users to interact with devices such as large displays. Each wand may be used to specify a full 6 degrees of freedom, which can be helpful for manipulation of 3D applications. The wands may also be used to identify 3D world points, which is useful for multi-camera calibration, and for building 3D models of physical spaces. The described methods provide a more precise estimation of wand pose, and can be implemented with a single camera. Also described is a method for estimation wand pose using information available along a one dimensional ‘line scan’ within the image, which simplifies processing.

Abstract: Systems and methods for providing a status of a teleconference by determining an approximate delay time and providing a status signal in view of the determined approximate delay time are provided. An approximate delay time is approximately the amount of time that will elapse before an occurrence occurring at a first time, which is captured into an occurrence signal by a source unit, will be experienced at a second time after the occurrence signal is received by at least one receiving unit.

Abstract: Techniques for generating action keyframes for a fixed-position camera based on the identification of activity in the video, an assessment of the importance of the activity, object recognition in the video, and interaction techniques for seeing more details of the video are presented. In different embodiments of the invention, the importance of activity is determined based on the amount of activity, important locations in the video streams, detected features such as faces, and events from other sensors.

Abstract: Systems and computer-implemented methods for use in body pose estimation are provided. Training data is obtained, where the training data includes observation vector data and corresponding pose vector data for a plurality of images. The observation vector data is representative of the images in observation space. The pose vector data is representative of the same images in pose space. Based on the training data, a model is computed that includes parameters of mapping from the observation space to latent space, parameters of mapping from the latent space to the pose space, and parameters of the latent space. The latent space has a lower dimensionality than the observation space and the pose space.

Abstract: An integrated method for modeling the handoff between cameras for tracking a specific individual, including: creating a representation of overlaps, gaps, and allowable movement among the fields of view of the cameras, wherein the representation is modeled as states in a Hidden Markov Model (HMM); training the HMM using video of people walking through the fields of view of the cameras; selecting a person to be tracked; and identifying the best camera area using the HMM.

Abstract: Visual tracking of multiple objects in a crowded scene is critical for many applications include surveillance, video conference and human computer interaction. Complex interactions between objects result in partial or significant occlusions, making tracking a highly challenging problem. Presented is a novel efficient approach to tracking a varying number of objects through occlusion. The object tracking during occlusion is posed as a track-based segmentation problem in the joint-object space. Appearance models are used to interpret the foreground into multiple layer probabilistic masks in a Bayesian framework. The search for optimal segmentation solution is achieved by a greedy searching algorithm and integral image for real-time computing. Promising results on several challenging video surveillance sequences have been demonstrated.

Abstract: One embodiment is a method for an interaction technique allowing users to control nonlinear video playback by directly manipulating objects seen in the video playback, comprising the steps of: tracking a moving object on a camera; recording a video; creating an object trail for the moving object which corresponds to the recorded video; allowing the user to select a point in the object trail; and displaying a frame in the recorded video that corresponds with the selected point in the object trail.

Abstract: Foreground segmentation in real world dynamic scenes, including under various lighting and shadow conditions, is disclosed. It may be used with one or multiple cameras for various automated tasks, such as classification of moving object, tracking moving objects, and event detection in various indoor or outdoor environments. Pixel to pixel subtraction is performed on each frame, followed by a feature-level based foreground segmentation to properly validate the foreground pixels. In this step, for each pixel in the image, a neighborhood of pixels is selected, and the aggregate change in the neighborhood image is used to classify foreground and background pixels. Normalized cross correlation is then applied to the neighborhood of each pixel that was confirmed to be foreground pixel.

Abstract: Passive wands, tracked by one or more cameras, are used as user interface devices. The wands are inexpensive, and are a natural way for users to interact with devices such as large displays. Each wand may be used to specify a full 6 degrees of freedom, which can be helpful for manipulation of 3D applications. The wands may also be used to identify 3D world points, which is useful for multi-camera calibration, and for building 3D models of physical spaces. The described methods provide a more precise estimation of wand pose, and can be implemented with a single camera. Also described is a method for estimation wand pose using information available along a one dimensional ‘line scan’ within the image, which simplifies processing.

Abstract: A system, method and apparatus for remotely annotating an object. An embodiment of the present invention includes a video camera projector that captures video images of a local object and projects annotations made by a user at a remote location onto said local object.

Abstract: A method for creating multimedia meeting minutes is disclosed. In an embodiment, the method receives notations from a user. As each notation is received, context information is recorded with the notation. The context information is used to select pertinent portions of multimedia information received concurrently with the notations. An association is then created between the notation and each selected portion of the multimedia information. These associations may be used to access the selected portions of the multimedia information from the notations. The notations and their respective associations are deposited for future retrieval. The deposited notations may be revised by receiving an altered copy of the notations from a user. The deposited notations are modified in accordance with the altered copy.

Abstract: A method for improvement of the consistency of color and brightness across boundaries of multicamera and/or multidisplayed overlapping or near overlapping composite images, reduction of objectionable artifacts at overlapping image seams of composite images, adjustment of color and brightness/intensity on either side of overlapping image seams, application of color correction across a composite image, reduction of color and intensity mismatches of composite images, gradual change of color across image seams of a composite image, including altering color outside of overlap regions is provided so that the seams are less discernible and to avoid sharp color changes across a composite or mosaic image are reduced or avoided.

Abstract: Systems and methods generate a video for virtual reality wherein the video is both panoramic and spatially indexed. In embodiments, a video system includes a controller, a database including spatial data, and a user interface in which a video is rendered in response to a specified action. The video includes a plurality of images retrieved from the database. Each of the images is panoramic and spatially indexed in accordance with a predetermined position along a virtual path in a virtual environment.