Abstract: The present invention is a method and system to provide correspondences between point-of-sale data registered at the store checkout and shopper behavior data observed at point-of-purchase through video analysis. The point-of-sale data include the list of shoppers and purchase items, and the shopper behavior data include the purchase events along with observed purchase items. The correspondence in the form of checkout shopper IDs matched to purchase event IDs is derived based on the algebraic constraint among the point-of-sale data and the purchase event data. Additional constraint based on shopper tracks and checkout/event times can also be incorporated to the correspondence problem. Uncertainties due to the video measurement can be systematically handled utilizing a Bayesian model.

Abstract: The present invention is a method and system for measuring viewership of people for a displayed object. The displayed object can be specific in-store marketing elements, such as static signage, POP displays, and other forms of digital media, including retail TV networks and kiosks. In the present invention, the viewership comprises impression level, impression count of the viewers, such as how many people actually viewed said displayed object, average length of impression, distribution of impressions by time of day, and rating of media effectiveness based on audience response. The viewership of people is performed automatically based on the 3-dimensional face pose estimation of the people, using a plurality of means for capturing images and a plurality of computer vision technologies on the captured visual information.

Abstract: Methods and systems for detecting expiration time of an article include receiving the article for a pre-planned event. The article is analyzed to identify one or more attributes. The article is matched to an event identifier using the article attributes. The event identifier is pre-defined and stored in an event database. The event identifier acts as an index to the event database for obtaining event characteristics of the pre-planned event. The article attributes are examined to identify event phase indicators that correlate to a specific event phase of the pre-planned event. The expiration time of the article is computed based on the specific event phase identified for the pre-planned event. The expiration time is used to rank the article and for considering assigning the article to a content pool of articles that is selected for publishing for the specific event phase of the pre-planned event.

Abstract: Methods and systems for detecting expiration time of an article include receiving the article for a pre-planned event. The article is analyzed to identify one or more attributes. The article is matched to an event identifier using the article attributes. The event identifier is pre-defined and stored in an event database. The event identifier acts as an index to the event database for obtaining event characteristics of the pre-planned event. The article attributes are examined to identify event phase indicators that correlate to a specific event phase of the pre-planned event. The expiration time of the article is computed based on the specific event phase identified for the pre-planned event. The expiration time is used to rank the article and for considering assigning the article to a content pool of articles that is selected for publishing for the specific event phase of the pre-planned event.

Abstract: The present invention is a method and system for forecasting the demographic characterization of customers to help customize programming contents on each means for playing output of each site of a plurality of sites in a media network through automatically measuring, characterizing, and estimating the demographic information of customers that appear in the vicinity of each means for playing output. The analysis of demographic information of customers is performed automatically based on the visual information of the customers, using a plurality of means for capturing images and a plurality of computer vision technologies on the visual information. The measurement of the demographic information is performed in each measured node, where the node is defined as means for playing output. Extrapolation of the measurement characterizes the demographic information per each node of a plurality of nodes in a site of a plurality of sites of a media network.

Abstract: The present invention is a method and system for optimizing the observation and annotation of complex human behavior from video sources by automatically detecting predefined events based on the behavior of people in a first video stream from a first means for capturing images in a physical space, accessing a synchronized second video stream from a second means for capturing images that are positioned to observe the people more closely using the timestamps associated with the detected events from the first video stream, and enabling an annotator to annotate each of the events with more labels using a tool. The present invention captures a plurality of input images of the persons by a plurality of means for capturing images and processes the plurality of input images in order to detect the predefined events based on the behavior in an exemplary embodiment. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the human behavior from the plurality of input images.

Abstract: The present invention is a method and system for producing a set of ratings for out-of-home media based on the measurement of behavior patterns and demographics of the people in a digital media network. The present invention captures a plurality of input images of the people in the vicinity of sampled out-of-home media in a digital media network by a plurality of means for capturing images, and tracks each person. The present invention processes the plurality of input images in order to analyze the behavior and demographics of the people. The present invention aggregates the measurements for the behavior patterns and demographics of the people, analyzes the data, and extracts characteristic information based on the estimated parameters from the aggregated measurements. Finally, the present invention calculates a set of ratings based on the characteristic information.

Abstract: The present invention is a system and method for determining the hierarchical purchase decision process of consumers in front of a product category. The decision path of consumers is obtained by combining behavior data with the category layout and transaction data based on observed actual in-store purchase behavior using a set of video cameras and software for extracting sequence and timing of each consumer's decision process. A hierarchical decision tree structure comprises nodes and edges, wherein a node represents the state-of-mind of the consumer, the number of nodes is predefined, and an edge represents the transition of the decision. The decisions for each product group are captured down to the product attribute level and analyzed by demographic group. The outcome provides relative importance of each product attribute in the purchase decision process, and helps retailers and manufacturers to evaluate the layout of the category and customize it for key segment.

Abstract: The present invention is a method and system for analyzing shopping behavior by associating RFID data, such as tracking data by the RFID tag identifications, with video-based behavior and segmentation data, such as behavior analysis and demographic composition analysis of the customers, utilizing a plurality of means for sensing and using RFID tags, a plurality of means for capturing images, and a plurality of computer vision technologies. In the present invention, the association can further comprise the association of the RFID with the transaction data or any time-based measurement in the retail space. The analyzed shopping behavior in the present invention helps people to better understand business elements in a retail space.

Abstract: The present invention is a method and system for segmenting a plurality of persons in a physical space based on automatic behavior analysis of the persons in a preferred embodiment. The behavior analysis can comprise a path analysis as one of the characterization methods. The present invention applies segmentation criteria to the output of the video-based behavior analysis and assigns segmentation label to each of the persons during a predefined window of time. In addition to the behavioral characteristics, the present invention can also utilize other types of visual characterization, such as demographic analysis, or additional input sources, such as sales data, to segment the plurality of persons in another exemplary embodiment. The present invention captures a plurality of input images of the persons in the physical space by a plurality of means for capturing images.

Abstract: The present invention is a method and system for automatically analyzing a category in a plurality of the categories in a physical space based on the visual characterization, such as behavior analysis or segmentation, of the persons with regard to the category. The present invention captures a plurality of input images of the persons in the category by a plurality of means for capturing images. The present invention processes the plurality of input images in order to understand the shopping behavior of the persons with the sub-categories of the category and analyzes the level of engagement and decision process at the sub-category level. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the visual characterization of the persons from the plurality of input images. The physical space may be a retail space, and the persons may be customers in the retail space.

Abstract: The present invention is a method and system for automatically determining the trip of people in a physical space, such as retail space, by capturing a plurality of input images of the people by a plurality of means for capturing images, processing the plurality of input images in order to track the people in each field of view of the plurality of means for capturing images, mapping the trip on to the coordinates of the physical space, joining the plurality of tracks across the multiple fields of view of the plurality of means for capturing images, and finding information for the trip of the people based on the processed results from the plurality of tracks. The trip information can comprise coordinates of the people's position and temporal attributes, such as trip time and trip length, for the plurality of tracks. The physical space may be a retail space, and the people may be customers in the retail space.

Abstract: The present invention is a method and system for automatically analyzing the behavior of a person and a plurality of persons in a physical space based on measurement of the trip of the person and the plurality of persons on input images. The present invention captures a plurality of input images of the person by a plurality of means for capturing images, such as cameras. The plurality of input images is processed in order to track the person in each field of view of the plurality of means for capturing images. The present invention measures the information for the trip of the person in the physical space based on the processed results from the plurality of tracks and analyzes the behavior of the person based on the trip information. The trip information can comprise coordinates of the person's position and temporal attributes, such as trip time and trip length, for the plurality of tracks. The physical space may be a retail space, and the person may be a customer in the retail space.

Abstract: The present invention is a method and system for characterizing physical space based on automatic demographics measurement, using a plurality of means for capturing images and a plurality of computer vision technologies. The present invention is called demographic-based retail space characterization (DBR). Although the disclosed method is described in the context of retail space, the present invention can be applied to any physical space that has a restricted boundary. In the present invention, the physical space characterization can comprise various types of characterization depending on the objective of the physical space, and it is one of the objectives of the present invention to provide the automatic demographic composition measurement to facilitate the physical space characterization.

Abstract: The present invention is a method and system for forecasting the behavioral characterization of customers to help customize programming contents on each means for playing output of each site of a plurality of sites in a media network through automatically measuring, characterizing, and forecasting the behavioral information of customers that appear in the vicinity of each means for playing output. The analysis of behavioral information of customers is performed automatically based on the visual information of the customers, using a plurality of means for capturing images and a plurality of computer vision technologies on the visual information. The measurement of the behavioral information is performed in each measured node, where the node is defined as means for playing output. Extrapolation of the measurement characterizes the behavioral information per each node of a plurality of nodes in a site of a plurality of sites of a media network.

Abstract: The present invention is a method and system for recognizing employees among the people in a physical space based on automatic behavior analysis of the people in a preferred embodiment. The present invention captures a plurality of input images of the people in the physical space by a plurality of means for capturing images. The present invention processes the plurality of input images in order to understand the behavioral characteristics of the people for the employee recognition purpose. The behavior analysis can comprise a path analysis as one of the characterization methods. The path analysis collects a plurality of trip information for each tracked person during a predefined window of time. The trip information can comprise spatial and temporal attributes, such as coordinates of the person's position, trip time, trip length, and average velocity for each of the plurality of trips.

Abstract: The present invention is a method and system for automatically changing the advertisement contents on a means for displaying the contents based on a customer's behavior or a group of customers' behavior in a retail store. The analysis of behavioral pattern is performed automatically and in real-time based on the visual information from the shopping and walkthrough history of a customer or a group of customers, using arrays of sensing devices, such as a plurality of means for capturing images, and a plurality of computer vision technologies on the visual information.

Abstract: The invention provides a face-based automatic demographics classification system that is robust to pose changes of the target faces and to accidental scene variables, by using a pose-independent facial image representation which comprises multiple pose-dependent facial appearance models. Given a sequence of people's faces in a scene, the two-dimensional variations are estimated and corrected using a novel machine learning based method. We estimate the three-dimensional pose of the people, using a machine learning based approach. The face tracking module keeps the identity of the person using geometric and appearance cues, where multiple appearance models are built based on the poses of the faces. Each separately built pose-dependent facial appearance model is fed to the demographics classifier, which is trained using only the faces having the corresponding pose.

Abstract: The present invention is a method and system for automatically analyzing a category in a plurality of the categories in a physical space based on the visual characterization, such as behavior analysis or segmentation, of the persons with regard to the category. The present invention captures a plurality of input images of the persons in the category by a plurality of means for capturing images. The present invention processes the plurality of input images in order to understand the shopping behavior of the persons with the sub-categories of the category and analyzes the level of engagement and decision process at the sub-category level. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the visual characterization of the persons from the plurality of input images. The physical space may be a retail space, and the persons may be customers in the retail space.

Abstract: The present invention includes a method and system for automatically extracting the multi-class age category information of a person from digital images. The system detects the face of the person(s) in an image, extracts features from the face(s), and then classifies into one of the multiple age categories. Using appearance information from the entire face gives better results as compared to currently known techniques. Moreover, the described technique can be used to extract age category information in more robust manner than currently known methods, in environments with a high degree of variability in illumination, pose and presence of occlusion. Besides use as an automated data collection system wherein given the necessary facial information as the data, the age category of the person is determined automatically, the method could also be used for targeting certain age-groups in advertisements, surveillance, human computer interaction, security enhancements and immersive computer games.