Abstract: A self-descriptive microphone array includes a microphone array memory, such as, for example a ROM, EEPROM, or other conventional memory, which contains a microphone array device description. This device description includes parametric information which defines operational characteristics and configuration of the microphone array. In further embodiments, the microphone array uses any of a variety of conventional wired or wireless computer interfaces, including serial, IEEE 1394, USB, Bluetooth™, etc., to connect to a computing device. Once connected, the microphone array provides its device description to the computing device. Sound processing software residing within the computing device is then automatically configured for optimally interacting with one or more analog or digital audio signals provided by the microphone array. In another embodiment, the microphone array performs integrated self calibration for automatically updating the device description.

Abstract: Techniques to minimize problems with dead zones associated with panoramic cameras are described. A dead zone is an area about a zero degree boundary where a three hundred and sixty degree panoramic image is split so that a two-dimensional panorama can be rendered. Redirection of the zero degree boundary to a position where no object appears is described that prevents an object located in the dead zone from being split between margins of the panorama. Another technique involves reproducing one or more portions of the image that appears at one margin adjacent to an opposite margin so that an object in the dead zone that would normally be split is reproduced in whole. The described techniques may be implemented in a panoramic camera endpoint device or in a client device remote from a camera. The techniques may be applied to live video or to previously recorded video.

Abstract: A foveated panoramic camera system includes multiple cameras oriented so that individual images captured by the cameras can be combined to form a panoramic image. Each of the cameras includes a lens having a focal length that corresponds to a field of view for the camera. A field of view for a camera overlaps with the field(s) of view of each adjacent camera. At least one of the cameras has a field of view that differs from fields of view of other cameras for capturing images that are situated at a greater distance from the camera system than are images captured by the other cameras. As a result, a more uniform resolution is achieved across all images captured by the multiple cameras. A mirror assembly is utilized to reflect object images into the multiple cameras to achieve a near center of projection for the camera system.

Abstract: A temperature measuring device measures ambient temperature in which a multi-camera photographic device is operated. A remapping function that maps individual images into a panoramic space is varied according to the measured temperature. The cameras and remapping tables are calibrated at various temperatures. When the ambient temperature is measured, values from a nearest matching remapping table are used, or new values are interpolated/extrapolated from one or more nearest matching tables. The remapping function is thus calibrated to properly remap the individual images into the panoramic image.

Abstract: A camera samples an image area that includes an active region that encompasses a captured photographed image and an extended region. The extended region includes a reference object that is fixed to the camera and is sampled with the photographed image. An image of the reference object is referenced and used for one or more color calibration procedures, such as white balancing, black level calibration, and red and blue channel gains. In a multi-camera configuration, each camera includes a reference object and color calibration is performed for each camera to achieve near-seamless mosaic panoramic images.

Abstract: A system and process for discovery between endpoint components and computing devices residing at remote locations from each other, in cases where the devices are connected to a common computer network, is presented. The present invention allows a device in one location to discover the network address of another device at a different location, which then allows the first device to negotiate with the discovered device to facilitate the transfer of data and other communications over the common network. However, this discovery is accomplished without the use of the common network. Rather the discovery process uses audible or inaudible audio signals transferred between the devices at two different locations using a communication medium, such as a telephone system.

Abstract: A whiteboard view camera, which in one embodiment is mounted on a boom, which is fixed above the whiteboard. However, the whiteboard view camera can also be wall or table mounted. The whiteboard view camera differs from an ordinary camera by allowing the lens to tilt with respect to the sensor plane, and shift with respect to the central ray. When the tilt angle &agr; is zero, then the resolution of whiteboard is uniform (e.g., the top and bottom of the whiteboard have the same resolution). In addition, the focus plane is equal to the whiteboard plane, so depth of field issues are not a problem (e.g., a large aperture can be used). Hence, no image noise is introduced. A key contribution of the whiteboard view camera configuration according to the invention is the use of a view camera to capture whiteboard images at uniform resolutions. One embodiment of the whiteboard view camera employs a microphone array of the invention and uses a computer to optimize the image data and audio signals.

Abstract: An omni-directional camera array that is very small and unobtrusive and allows for higher pixel resolution for the face of meeting participants sitting or standing around a conference room table, as would be typical in a video conferencing situation. The camera system of the invention uses N cameras and an N sided mirror to capture images of the surrounding scene. The cameras are arranged to have a small distance between their virtual centers of projection. The images from the N cameras are stitched together to create a 360-degree panorama using a calibration surface. To stitch the images together in real-time, a static image stitcher is used, which stitches images together perfectly for objects on a calibration surface. The calibration surface is typically defined by the boundary of the table the camera is sitting on.

Abstract: A system and method for teleconferencing and recording of meetings. The system uses a variety of capture devices (a novel 360° camera, a whiteboard camera, a presenter view camera, a remote view camera, and a microphone array) to provide a rich experience for people who want to participate in a meeting from a distance. The system is also combined with speaker clustering, spatial indexing, and time compression to provide a rich experience for people who miss a meeting and want to watch it afterward.

Abstract: A system and method for detecting speech utilizing audio and video inputs. In one aspect, the invention collects audio data generated from a microphone device. In another aspect, the invention collects video data and processes the data to determine a mouth location for a given speaker. The audio and video are inputted into a time-delay neural network that processes the data to determine which target is speaking. The neural network processing is based upon a correlation to detected mouth movement from the video data and audio sounds detected by the microphone.

Abstract: An omni-directional camera (a 360 degree camera) is proposed with an integrated microphone array. The primary application for such a camera is videoconferencing and meeting recording, and the device is designed to be placed on a meeting room table. The microphone array is in a planar configuration, and the microphones are located as close to the desktop as possible to eliminate sound reflections from the table. The camera is connected to the microphone array base with a thin cylindrical rod, which is acoustically invisible to the microphone array for the frequency range [50-4000] Hz. This provides a direct path from the person talking to all of the microphones in the array, and can therefore be used for sound source localization (determining the location of the talker) and beam-forming (improving the sound quality of the talker by filtering only sound from a particular direction). The camera array is elevated from the table to provide a near frontal viewpoint of the meeting participants.

Abstract: A practical, real-time calibration of digital omnidirectional cameras in the areas of de-vignetting, brightness, contrast, and white balance control. Novel solutions for the color calibration of an omnidirectional camera rig, and an efficient method for devignetting images are presented. Additionally, a context-specific method of stitching images together into a panorama or a mosaic is provided.

Abstract: A system that captures both whiteboard content and audio signals of a meeting using a digital camera and a microphone. The system can be retrofit to any existing whiteboard. It computes the time stamps of pen strokes on the whiteboard by analyzing the sequence of captured snapshots. It also automatically produces a set of key frames representing all the written content on the whiteboard before each erasure. The whiteboard content serves as a visual index to efficiently browse the audio meeting. The system not only captures the whiteboard content, but also helps the users to view and manage the captured meeting content efficiently and securely.