Abstract: A system and process for muting the audio transmission from a location of a participant engaged in a multi-party, computer network-based teleconference when that participant is working on a keyboard, is presented. The audio is muted as it is assumed the participant is doing something other than actively participation in the meeting when typing on the keyboard. If left un-muted the sound of typing would distract the other participant in the teleconference.

Abstract: An event-based system and process for recording and playback of collaborative electronic presentations is presented. The present system and process includes a technique for recording collaborative electronic presentations by capturing and storing the interactions between each participant and presentation data where each interaction event is timestamped and linked to a data file comprising the presentation data. The present system and process also includes a technique for playing back the recorded collaborative electronic presentation, which involves displaying the presentation data in an order it was originally presented and reproducing the recorded interactions between each participant and the displayed presentation data at the same point in the presentation that they were originally performed, based on the aforementioned timestamps.

Abstract: A system and process for providing an interactive computer network-based virtual team worksite that combines data storage, team members' presence information, interaction tools and a past history log into one virtual complex is presented. Generally, this is accomplished by integrating a shared data module, a unique presence module and various conferencing tools such as a collaborative presentation module and chat module into a single worksite assessable over a distributed computer network. Thus, everything a team would need related to a project is available in this integrated place. A team member who logs onto the worksite can input data and commands using the worksite window sectors to interface with other team members also logged on to the worksite and to interact with the displayed data in the collaborative presentation sector.

Abstract: A system and process for estimating the time delay of arrival (TDOA) between a pair of audio sensors of a microphone array is presented. Generally, a generalized cross-correlation (GCC) technique is employed. However, this technique is improved to include provisions for both reducing the influence (including interference) from correlated ambient noise and reverberation noise in the sensor signals prior to computing the TDOA estimate. Two unique correlated ambient noise reduction procedures are also proposed. One involves the application of Wiener filtering, and the other a combination of Wiener filtering with a Gnn subtraction technique. In addition, two unique reverberation noise reduction procedures are proposed. Both involve applying a weighting factor to the signals prior to computing the TDOA which combines the effects of a traditional maximum likelihood (TML) weighting function and a phase transformation (PHAT) weighting function.

Abstract: A system and process is described for estimating the location of a speaker using signals output by a microphone array characterized by multiple pairs of audio sensors. The location of a speaker is estimated by first determining whether the signal data contains human speech components and filtering out noise attributable to stationary sources. The location of the person speaking is then estimated using a time-delay-of-arrival based SSL technique on those parts of the data determined to contain human speech components. A consensus location for the speaker is computed from the individual location estimates associated with each pair of microphone array audio sensors taking into consideration the uncertainty of each estimate. A final consensus location is also computed from the individual consensus locations computed over a prescribed number of sampling periods using a temporal filtering technique.

Abstract: An event-based system and process for recording and playback of collaborative electronic presentations is presented. The present system and process includes a technique for recording collaborative electronic presentations by capturing and storing the interactions between each participant and presentation data where each interaction event is timestamped and linked to a data file comprising the presentation data. The present system and process also includes a technique for playing back the recorded collaborative electronic presentation, which involves displaying the presentation data in an order it was originally presented and reproducing the recorded interactions between each participant and the displayed presentation data at the same point in the presentation that they were originally performed, based on the aforementioned timestamps.

Abstract: A system and process for tracking an object state over time using particle filter sensor fusion and a plurality of logical sensor modules is presented. This new fusion framework combines both the bottom-up and top-down approaches to sensor fusion to probabilistically fuse multiple sensing modalities. At the lower level, individual vision and audio trackers can be designed to generate effective proposals for the fuser. At the higher level, the fuser performs reliable tracking by verifying hypotheses over multiple likelihood models from multiple cues. Different from the traditional fusion algorithms, the present framework is a closed-loop system where the fuser and trackers coordinate their tracking information. Furthermore, to handle non-stationary situations, the present framework evaluates the performance of the individual trackers and dynamically updates their object states.

Abstract: A system and method for object tracking using probabilistic mode-based multi-hypothesis tracking (MHT) provides for robust and computationally efficient tracking of moving objects such as heads and faces in complex environments. A mode-based multi-hypothesis tracker uses modes that are local maximums which are refined from initial samples in a parametric state space. Because the modes are highly representative, the mode-based multi-hypothesis tracker effectively models non-linear probabilistic distributions using a small number of hypotheses. Real-time tracking performance is achieved by using a parametric causal contour model to refine initial contours to nearby modes. In addition, one common drawback of conventional MHT schemes, i.e., producing only maximum likelihood estimates instead of a desired posterior probability distribution, is addressed by introducing an importance sampling framework into MHT, and estimating the posterior probability distribution from the importance function.

Abstract: Audio/video programming content is made available to a receiver from a content provider, and meta data is made available to the receiver from a meta data provider. The meta data corresponds to the programming content, and identifies, for each of multiple portions of the programming content, an indicator of a likelihood that the portion is an exciting portion of the content. In one implementation, the meta data includes probabilities that segments of a baseball program are exciting, and is generated by analyzing the audio data of the baseball program for both excited speech and baseball hits. The meta data can then be used to generate a summary for the baseball program.

Abstract: Disclosed are a unique DPC (detect point click) based game system and method. The DPC based game system involves generating one or a plurality of DPC images, presenting them to a game participant, and collecting the participant's clicks (that identify which object in the DPC image the participant believes to be the correct DPC object), and determining whether the participant's clicks represent the correct object. DPC images can be created in part by selecting a base image, altering some portion of the base image to create at least one confusion image, mapping these images to a geometric model, and applying one or more distortion filters to at least one of the base or confusing image to obscure the DPC object from clear view. Locating the DPC object nearly hidden in the DPC image can advance the participant in the DPC based game or other game including DPC images as a part thereof.

Abstract: A system and process for finding the location of a sound source using direct approaches having weighting factors that mitigate the effect of both correlated and reverberation noise is presented. When more than two microphones are used, the traditional time-delay-of-arrival (TDOA) based sound source localization (SSL) approach involves two steps. The first step computes TDOA for each microphone pair, and the second step combines these estimates. This two-step process discards relevant information in the first step, thus degrading the SSL accuracy and robustness. In the present invention, direct, one-step, approaches are employed. Namely, a one-step TDOA SSL approach and a steered beam (SB) SSL approach are employed. Each of these approaches provides an accuracy and robustness not available with the traditional two-step approaches.

Abstract: A system and method for object tracking using probabilistic mode-based multi-hypothesis tracking (MHT) provides for robust and computationally efficient tracking of moving objects such as heads and faces in complex environments. A mode-based multi-hypothesis tracker uses modes that are local maximums which are refined from initial samples in a parametric state space. Because the modes are highly representative, the mode-based multi-hypothesis tracker effectively models non-linear probabilistic distributions using a small number of hypotheses. Real-time tracking performance is achieved by using a parametric causal contour model to refine initial contours to nearby modes. In addition, one common drawback of conventional MHT schemes, i.e., producing only maximum likelihood estimates instead of a desired posterior probability distribution, is addressed by introducing an importance sampling framework into MHT, and estimating the posterior probability distribution from the importance function.

Abstract: A system for communicating audio data signals comprises a source computer that performs an action, generates an event message corresponding to the action, converts the event message into an audio data signal, and communicates the audio data signal through its speaker. A source telephone receives a voice signal from a participant and the audio data signal through its microphone and communicates the audio data signal and voice as coherent sound via an audio communications medium. A recipient telephone receives the audio data signal from the coherent sound communicated via the audio communications medium and communicates the audio data signal via its speaker. A recipient computer receives the audio data signal through its microphone, extracts the event message from the audio data signal, and performs an action based on the event message from the audio data signal. The audio communications medium can comprise a telephone communications system or air.

Abstract: A “virtual video studio”, as described herein, provides a highly portable real-time capability to automatically capture, record, and edit a plurality of video streams of a presentation, such as, for example, a speech, lecture, seminar, classroom instruction, talk-show, teleconference, etc., along with any accompanying exhibits, such as a corresponding slide presentation, using a suite of one or more unmanned cameras controlled by a set of videography rules. The resulting video output may then either be stored for later use, or broadcast in real-time to a remote audience. This real-time capability is achieved by using an abstraction of “virtual cameramen” and physical cameras in combination with a scriptable interface to the aforementioned videography rules for capturing and editing the recorded video to create a composite video of the presentation in real-time under the control of a “virtual director.

Abstract: An automated system and method for broadcasting meetings over a computer network. The meeting is filmed using an omni-directional camera system and capable of being presented to a viewer both live and on-demand. The system of the present invention includes an automated camera management system for controlling the camera system and an analysis module determining the location of meeting participants in the meeting environments. The method of the present invention includes using the system of the present invention to broadcast an event to a viewer over a computer network. In particular, the method includes filming the event using an omni-directional camera system. Next, the method determines the location of each event participant in the event environment. Finally, a viewer is provided with a user interface for viewing the broadcast event. This user interface allows a viewer to choose which event participant that the viewer would like to view.

Abstract: An automated system and method for broadcasting meetings over a computer network. The meeting is filmed using an omni-directional camera system and capable of being presented to a viewer both live and on-demand. The system of the present invention includes an automated camera management system for controlling the camera system and an analysis module determining the location of meeting participants in the meeting environments. The method of the present invention includes using the system of the present invention to broadcast an event to a viewer over a computer network. In particular, the method includes filming the event using an omni-directional camera system. Next, the method determines the location of each event participant in the event environment. Finally, a viewer is provided with a user interface for viewing the broadcast event. This user interface allows a viewer to choose which event participant that the viewer would like to view.

Abstract: A system and process is described for estimating the location of a speaker using signals output by a microphone array characterized by multiple pairs of audio sensors. The location of a speaker is estimated by first determining whether the signal data contains human speech components and filtering out noise attributable to stationary sources. The location of the person speaking is then estimated using a time-delay-of-arrival based SSL technique on those parts of the data determined to contain human speech components. A consensus location for the speaker is computed from the individual location estimates associated with each pair of microphone array audio sensors taking into consideration the uncertainty of each estimate. A final consensus location is also computed from the individual consensus locations computed over a prescribed number of sampling periods using a temporal filtering technique.

Abstract: Systems and methods are disclosed that facilitate real-time information exchange in a multimedia conferencing environment. Data Client(s) facilitate data collaboration between users and are maintained separately from audio/video (AV) Clients that provide real-time communication functionality. Data Clients can be remotely located with respect to one another and with respect to a server. A remote user Stand-in Device can be provided that comprises a display to present a remote user to local users, a digital automatic pan/tilt/zoom camera to capture imagery in, for example, a conference room and provide real-time information to an AV Client in a remote office, and a microphone array that can similarly provide real-time audio information from the conference room to an AV Client in the remote office. The invention further facilitates file transfer and presentation broadcast between Data Clients in a single location or in a plurality of disparate locations.

Abstract: A system and process for estimating the time delay of arrival (TDOA) between a pair of audio sensors of a microphone array is presented. Generally, a generalized cross-correlation (GCC) technique is employed. However, this technique is improved to include provisions for both reducing the influence (including interference) from correlated ambient noise and reverberation noise in the sensor signals prior to computing the TDOA estimate. Two unique correlated ambient noise reduction procedures are also proposed. One involves the application of Wiener filtering, and the other a combination of Wiener filtering with a Gnn subtraction technique. In addition, two unique reverberation noise reduction procedures are proposed. Both involve applying a weighting factor to the signals prior to computing the TDOA which combines the effects of a traditional maximum likelihood (TML) weighting function and a phase transformation (PHAT) weighting function.

Abstract: Automatic detection and tracking of multiple individuals includes receiving a frame of video and/or audio content and identifying a candidate area for a new face region in the frame. One or more hierarchical verification levels are used to verify whether a human face is in the candidate area, and an indication made that the candidate area includes a face if the one or more hierarchical verification levels verify that a human face is in the candidate area. A plurality of audio and/or video cues are used to track each verified face in the video content from frame to frame.