Abstract: A system that captures both whiteboard content and audio signals of a meeting using a digital camera and a microphone. The system can be retrofit to any existing whiteboard. It computes the time stamps of pen strokes on the whiteboard by analyzing the sequence of captured snapshots. It also automatically produces a set of key frames representing all the written content on the whiteboard before each erasure. The whiteboard content serves as a visual index to efficiently browse the audio meeting. The system not only captures the whiteboard content, but also helps the users to view and manage the captured meeting content efficiently and securely.

Abstract: A system and method for automatically modeling a three dimensional object, such as a face, from a single image. The system and method according to the invention can construct one or more three dimensional (3D) face models using a single image. Although use of a frontal image simplifies the processing, the system and method according to the invention can also generate a 3D model from a non-frontal image (e.g., an image where the object to be modeled has an out of plane rotation, such a face that is turned to one side to some extent). The system is fully automatic. It is fast compared to the other modeling systems, Furthermore, it is very robust. It can be used to construct personalized models for games, on-line chat, and so on. It can also be used as a tool to generate a database of faces with various poses which are needed to train most face recognition systems.

Abstract: An “Image Segmenter” provides a variational energy formulation for segmentation of natural objects from images. In general, the Image Segmenter operates by adopting Gaussian mixture models (GMM) to capture the appearance variation of objects in one or more images. A global image data likelihood potential is then computed and combined with local region potentials to obtain a robust and accurate estimation of pixel foreground and background distributions. Iterative minimization of a “global-local energy function” is then accomplished by evolution of a foreground/background boundary curve by level set, and estimation of a foreground/background model by fixed-point iteration, termed “quasi-semi-supervised EM.” In various embodiments, this process is further improved by providing general object shape information for use in rectifying objects segmented from the image.

Abstract: Infrastructure for a multi-modal multilingual communications device (MMCD) is presented. A communications component is provided that includes wireless and wired IP networks (e.g, LANs, MANs, and WANs, . . . ), as well as cellular and/or wired telecommunications networks for cellular communications. A management component can include software and hardware entities that facilitate the activation, authentication, accounting, updating of the MMCD systems, and synchronization to other entities. Additionally, the management component can facilitate the dissemination of applications, third-party services, and subscription information. An access component (e.g., a web server and interface) facilitates access to one or more of these entities such that administrators and/or users can access aspects of setup, configuration, subscriptions, updates, etc.

Abstract: A system and method for automatically modeling a three dimensional object, such as a face, from a single image. The system and method according to the invention can construct one or more three dimensional (3D) face models using a single image. Although use of a frontal image simplifies the processing, the system and method according to the invention can also generate a 3D model from a non-frontal image (e.g., an image where the object to be modeled has an out of plane rotation, such a face that is turned to one side to some extent). The system is fully automatic. It is fast compared to the other modeling systems. Furthermore, it is very robust. It can be used to construct personalized models for games, on-line chat, and so on. It can also be used as a tool to generate a database of faces with various poses which are needed to train most face recognition systems.

Abstract: A system that facilitates managing resources (e.g., functionality, services) based at least in part upon an established context. More particularly, a context determination component can be employed to establish a context by processing sensor inputs or learning/inferring a user action/preference. Once the context is established via context determination component, a power/mode management component can be employed to activate and/or mask resources in accordance with the established context. The power and mode management of the device can extend life of a power source (e.g., battery) and mask functionality in accordance with a user and/or device state.

Abstract: A multi-modal device that can substantially facilitate intelligent shopping. Electronic receipts can be provided to a user wirelessly and stored/indexed on the multi-modal device. Receipts can be categorized (e.g., personal, business, client entertainment) thereby facilitating financial management and accounting. Likewise, such electronic receipts can provide for easier return/exchange of goods. The multi-modal device can also assist in tracking/managing shopping lists and business cards (e.g., provide for business card exchanges). Moreover, the multi-modal device can provide for comparison shopping, catalog shopping, locating products and obtaining more information about a product via visual or audio mechanisms.

Abstract: Described herein is a technique for creating a 3D face model using images obtained from an inexpensive camera associated with a general-purpose computer. Two still images of the user are captured, and two video sequences. The user is asked to identify five facial features, which are used to calculate a mask and to perform fitting operations. Based on a comparison of the still images, deformation vectors are applied to a neutral face model to create the 3D model. The video sequences are used to create a texture map. The process of creating the texture map references the previously obtained 3D model to determine poses of the sequential video images.

Abstract: A first set of signals from an array of one or more microphones, and a second signal from a reference microphone are used to calibrate a set of filter parameters such that the filter parameters minimize a difference between the second signal and a beamformer output signal that is based on the first set of signals. Once calibrated, the filter parameters are used to form a beamformer output signal that is filtered using a non-linear adaptive filter that is adapted based on portions of a signal that do not contain speech, as determined by a speech detection sensor.

Abstract: A system and method, called Image-Based Surface Detail Transfer, to transfer geometric details from one surface of an object in an image to another with simple 2D image operations. The basic observation is that, without knowing its 3D geometry, geometric details (local deformations) can be extracted from a single image of an object in a way independent of its surface reflectance, and furthermore, these geometric details can be transferred to modify the appearance of other objects directly in images. Examples are shown including surface detail transfer between real objects, as well as between real and synthesized objects.

Abstract: A panoramic camera is configured to automatically determine parameters of a table upon which the camera is situated as well as positional information of the camera relative to the table. In an initialization stage, table edges are detected to create an edge map. A Hough transformation-like symmetry voting operation is performed to clean up the edge map and to determine camera offset, camera orientation and camera tilt. The table is then fit to a table model to determine table parameters. In an operational stage, table edges are detected to create an edge map and the table model is fit to the edge map. The output can then be used for further panoramic image processing such as head size normalization, zooming, compensation for camera movement, etc.

Abstract: A real-time approximately 360 degree image correction system and a method for alleviating distortion and perception problems in images captured by omni-directional cameras. In general, the real-time panoramic image correction method generates a warp table from pixel coordinates of a panoramic image and applies the warp table to the panoramic image to create a corrected panoramic image. The corrections are performed using a parametric class of warping functions that include Spatially Varying Uniform (SVU) scaling functions. The SVU scaling functions and scaling factors are used to perform vertical scaling and horizontal scaling on the panoramic image pixel coordinates. A horizontal distortion correction is performed using the SVU scaling functions at at least two different scaling factors. This processing generates a warp table that can be applied to the panoramic image to yield the corrected panoramic image.

Abstract: A system and method for facilitating low bandwidth video image transmission in video conferencing systems. A target is acquired (video image of a person's head) and processed to identify one or more sub-regions (e.g., background, eyes, mouth and head). The invention incorporates a fast feature matching methodology to match a current sub-region with previously stored sub-regions. If a match is found, an instruction is sent to the receiving computer to generate the next frame of video data from the previously stored blocks utilizing a texture synthesis technique. The invention is applicable for video conferencing in low bandwidth environments.

Abstract: Described herein is a technique for creating a 3D face model using images obtained from an inexpensive camera associated with a general-purpose computer. Two still images of the user are captured, and two video sequences. The user is asked to identify five facial features, which are used to calculate a mask and to perform fitting operations. Based on a comparison of the still images, deformation vectors are applied to a neutral face model to create the 3D model. The video sequences are used to create a texture map. The process of creating the texture map references the previously obtained 3D model to determine poses of the sequential video images.

Abstract: A mobile device includes an air conduction microphone and an alternative sensor that provides an alternative sensor signal indicative of speech. A communication interface permits the mobile device to communicate directly with other mobile devices.

Abstract: Described herein is a technique for creating a 3D face model using images obtained from an inexpensive camera associated with a general-purpose computer. Two still images of the user are captured, and two video sequences. The user is asked to identify five facial features, which are used to calculate a mask and to perform fitting operations. Based on a comparison of the still images, deformation vectors are applied to a neutral face model to create the 3D model. The video sequences are used to create a texture map. The process of creating the texture map references the previously obtained 3D model to determine poses of the sequential video images.

Abstract: A multimodal, multilanguage mobile device which can be employed to enhance note taking and/or annotation of a document, and gaming. Input data types such as optical character recognition (OCR), speech, handwriting, and visual information (e.g., image and/or video), etc., can be fused to generate rich documents with a multidimensional level of data to provide an increased level of context over conventional documents. Such architecture can be utilized by students for homework management, as well as entertainment (e.g., gaming).

Abstract: Multi-modal, multi-lingual devices can be employed to consolidate numerous items including, but not limited to, keys, remote controls, image capture devices, audio recorders, cellular telephone functionalities, location/direction detectors, health monitors, calendars, gaming devices, smart home inputs, pens, optical pointing devices or the like. For example, a corner of a cellular telephone can be used as an electronic pen. Moreover, the device can be used to snap multiple pictures stitching them together to create a panoramic image. A device can automate ignition of an automobile, initiate appliances, etc. based upon relative distance. The device can provide for near to eye capabilities for enhanced image viewing. Multiple cameras/sensors can be provided on a single device to provide for stereoscopic capabilities. The device can also provide assistance to blind, privacy, etc. by consolidating services.

Abstract: A synthesized alternative sensor signal is produced from an alternative sensor signal. The synthesized alternative sensor signal is computed using vocal tract resonances estimated based on the alternative sensor signal, and using a waveform synthesis technique that converts the estimated vocal tract resonance sequence into a spectral magnitude sequence. The synthesized alternative sensor signal and the alternative sensor signal are used to estimate a clean speech value.

Abstract: A device and/or devices can be placed on an individual to continuously collect and/or transmit information regarding the health state of the individual. The monitor, for example, can transmit data to doctor(s), hospital(s), emergency personnel, as well as automatically initiate affirmative actions (e.g., call an ambulance). The invention can facilitate preventive medicine by monitoring the individual's day to day actions (e.g., eating habits, exercise, stress, sleep, allergy detection, heart rate, blood sugar). This monitored information can be employed to facilitate the individual in taking proactive steps to living a healthy life. The device can also be used to record a user reaction to an event and/or to broadcast information about oneself whether for health, business or social purposes. Additionally, the invention can be used as a driver monitor to facilitate safe handling of equipment (e.g., alcohol sensor).