Abstract: A recommendation system is disclosed that generates recommendations for one or more items based on user preferences and one or more environmental factors. The user's preferences are learned under various environmental conditions using an environmental data collection system. The observed environmental conditions may include, for example, location, characteristics of the location, weather or characteristics of the user's motion, such as a rate of movement. For each positive and negative behavioral example, a number of attributes of the selected item are classified in the user profile together with the prevailing environmental conditions. When recommending an item, the disclosed recommender retrieves the user preferences and evaluates the current environmental conditions. A recommendation score can be generated for each available item based on the user's demonstrated preferences under similar environmental conditions, such as in the same or a similar geographic area or under similar weather conditions.

Abstract: A system and apparatus are disclosed for modeling patterns of behavior of humans or other animate objects and detecting a violation of a repetitive pattern of behavior. The behavior of one or more persons is observed over time and features of the behavior are recorded in a multi-dimensional space. Over time, the multi-dimensional data provides an indication of patterns of human behavior. Activities that are repetitive in terms of time, location and activity, such as sleeping and eating, would appear as a Gaussian distribution or cluster in the multi-dimensional data. Probability distribution functions can be analyzed using known Gaussian or clustering techniques to identify repetitive patterns of behavior and characteristics thereof, such as a mean and variance. Deviations from repetitive patterns of behavior can be detected and an alarm can be triggered, if appropriate.

Abstract: A system and method for automatically controlling systems and devices in a local environment, such as a home. The system comprises a control unit that receives images associated with one or more regions of the local environment. The one or more regions are each serviced by one or more servicing components. The control unit processes the images to identify, from a group of known persons associated with the local environment, any one or more known persons located in the regions. For the regions in which one or more known person is identified, the control unit automatically generates a control signal for at least one of the servicing components associated with the region, the control signal reflecting a preference of at least one of the known persons located in the respective region.

Abstract: System and method for automatically programming a universal remote. In the system, the remote acquires identification data from a particular device at which it is pointed. The data is processed to determine command protocols associated with the particular device. Control commands for the particular device input to the remote by a user are formatted according to the command protocols associated with the particular device.

Abstract: To calibrate images from an overhead camera, two equal-length reference lines at different heights relative to the floor plane are imaged. By comparing the resultant image-width of each of these lines in the view provided by the camera, the effective focal point of the camera is determined. In a preferred embodiment, a doorframe is used to provide equal-length lines at different heights parallel to the floor. The threshold of the doorway at the floor plane is used as a reference plane, and a parallel calibration line at a known/measured height in the doorway is defined. The image produced by a vertically oriented camera of these two equal length lines will show a longer line at the elevated position, the different widths of the images of these lines being dependent upon the camera's effective focal point. From these two projections of the equal-length lines at different heights, the camera's focal point distance from the reference floor plane is determined.

Abstract: A video processing and analysis system is coupled to a building management and control system. The video processing system provides traffic, occupancy, and other information derived from video images of sections of the building and its environs to the building management and control systems.

Abstract: An image processing system detects a moving person or other object of interest. The system generates a thresholded difference image by processing a video signal or other type of image signal received from a camera. The difference image is then segmented into regions bounded by vertical lines passing through the image, and silhouette candidates are identified in one or more of the regions. Tensor voting is used to determine saliency values and corresponding tangents for each of the silhouette candidates, and the resulting values and tangents are used to detect the object of interest. In an embodiment in which the object of interest is a moving person, a neck position of the moving person may be detected by analyzing a sum of x-components of tangents along a corresponding silhouette. The detected neck position may then be utilized to determine a head position and a head size for the moving person.

Abstract: A video interface kernel with a defined application program interface includes each of a plurality of core functions for interfacing with video equipment and video processing subsystems. The core functions include such functions as video capture and video output, video recording and playback, and event notification. Optional embodiments include video analysis functions and/or interfaces to analysis subsystems, as well as interfaces to point-of-sale terminals, access control systems, and location tracking systems. An intelligent video management module facilitates managing the interactions among the other modules of the kernel, further simplifying the application level interface.

Abstract: An image processing system includes a combined display-camera having an array of interspersed display elements and camera elements, arranged substantially in a common plane of a flat panel or other display. Each of at least a subset of the camera elements has one or more imaging angles associated therewith, with the one or more imaging angles being selected to provide a desired imaging operation for the combined display-camera. The imaging angles of the camera elements can be selected to provide an imaging operation which approximates that of a lens-based single-camera system, a pin-hole camera system or other type of system. Each of the camera elements may include multiple image sensors, such that different imaging angles can be set for the different image sensors of a given camera element, and different perspectives of a scene can be generated in the image processing system.

Abstract: The present invention relates generally to the field of video-camera systems, such as a video conferencing systems, and more particularly to video camera targeting systems that locate and acquire targets using an input characterizing a target and a machine-classification system to assist in target acquisition responsively to that input. In some embodiments, the characterization and classification are employed together with one or more inputs of other modalities such as gesture-control. In one example of the system in operation, an operator is able to make pointing gestures toward an object and, simultaneously speak a sentence identifying the object to which the speaker is pointing. At least one term of the sentence, presumably, is associated with a machine-sensible characteristic by which the object can be identified. The system captures and processes the voice and gesture inputs and re-positions a PTZ video camera to focus on the object that best matches both the characteristics and the gesture.

Abstract: The shape of a three-dimensional object is determined from uncalibrated stereo images by first determining the projection of the point on a reference plane. The distance of each point from the reference plane is obtained by a technique that makes use of a calibration based on two known points visible in each camera's field of view. The method requires only one high precision measurement, which is the depth of the two calibration points from a reference plane. These may be supplied in a setup apparatus thereby avoiding the need for a user to make measurements. The rest of the required calibration data may be derived directly from the images of the cameras.

Abstract: An image processing system (250) and method (300) are disclosed for correcting a head pose in a video phone image, so that a frontal view is presented on a display. A disclosed head pose corrector (250) estimates the orientation of a head pose and adjusts the orientation of the head pose, if necessary, to present a frontal view. The orientation of the head pose is adjusted by generating a three dimensional model of the face surface and adjusting the orientation of the three dimensional face model to provide the desired frontal view. The head pose corrector (250) may be included in a video phone (100) to correct the head pose of transmitted or received images (or both) or may be included in a server on a network to automatically adjust the head shots of one or more participants to a video phone communication.

Abstract: A monocular input image is transformed to give it an enhanced three dimensional appearance by creating at least two output images. Foreground and background objects are segmented in the input image and transformed differently from each other, so that the foreground objects appear to stand out from the background. Given a sequence of input images, the foreground objects will appear to move differently from the background objects in the output images.

Abstract: A home control system for automatic detection and warning of abnormal behavior includes a unit for observing behavior in a predetermined area under surveillance, a unit for processing an output of observed behavior from the unit for observing, and a pattern recognition module for recognizing whether the observed behavior is associated with predefined normal behaviors. The detection of predetermined normal behavior in progress leads to a provision of an anticipatory action. Upon recognition that the observed behavior is abnormal, an alarm signal is triggered to remind the user.

Abstract: A video processing system tracks a moving person or other object of interest using a combined audio-video tracking system. The audio-video tracking system comprises an audio locator, a video locator, and a set of rules for determining the manner in which settings of a camera are adjusted based on outputs of the audio locator and video locator. The set of rules may be configured such that only the audio locator output is used to adjust the camera settings if the audio locator and video locator outputs are not sufficiently close and a confidence indicator generated by the audio locator is above a specified threshold. For example, in such a situation, the audio locator output alone may be used to direct the camera to a new speaker in a video conference. If the audio locator and video locator outputs are sufficiently close, the system determines if a confidence indicator generated by the video locator is above a specified level, and if so, the video locator output may be used to adjust the camera settings.

Abstract: System and method for automatically programming a universal remote. In the system, the remote acquires identification data from a particular device at which it is pointed. The data is processed to determine command protocols associated with the particular device. Control commands for the particular device input to the remote by a user are formatted according to the command protocols associated with the particular device.

Abstract: A virtual PTZ camera is described which forms a virtual image using multiple cameras whose fields of view overlap. Images from the cameras are merged by transforming to a common surface and property-blending overlapping regions to smooth transitions due to differences in image formation of common portions of a scene. To achieve high speed, the images may be merged to a common planar surface or set of surfaces so that transforms can be linear. Image information alone may be used to calculate the transforms from common feature points located in the images so that there is no need for three-dimensional geometric information about the cameras.

Abstract: A surveillance system allows a user to characterize the user's environment and/or the user's surveillance application, via a selection from among a variety of predefined environments and/or applications. Preferably, the selection is from among a variety of scene configurations, such as the expected number and type of targets in a typical scene, the lighting conditions of the scene, and so on. The selected environments and/or applications effect a determination of the parameters that are used in the various algorithms and processing modules within the surveillance system. Because the selection is preferably from a variety of common scene configurations, no technical skills are required to effect an optimization of the performance of the surveillance system for a particular environment.

Abstract: A system and method for automatically controlling systems and devices in a local environment, such as a home. The system comprises a control unit that receives images associated with one or more regions of the local environment. The one or more regions are each serviced by one or more servicing components. The control unit processes the images to identify, from a group of known persons associated with the local environment, any one or more known persons located in the regions.

Abstract: A home control system for automatic detection and warning of abnormal behavior includes a unit for observing behavior in a predetermined area under surveillance, a unit for processing an output of observed behavior from the unit for observing, and a pattern recognition module for recognizing whether the observed behavior is associated with predefined normal behaviors. The detection of predetermined normal behavior in progress leads to a provision of an anticipatory action. Upon recognition that the observed behavior is abnormal, an alarm signal is triggered to remind the user.