Abstract: Techniques are generally described for scene change detection. A first and second histogram representing frames of a video may be received. A Euclidean distance between the first and second histogram may be determined. A third histogram of a third frame may be received. Values of the third histogram may be compared to corresponding values of a background model of the environment. A fourth frame and a fifth frame of image data of the video may be received. A flow value between a first block of pixels of the fourth frame and a corresponding second block of pixels of the fifth frame may be determined. The flow value may be determined based on a motion vector and a sum of absolute differences between the first and second block. A portion of the video may be streamed to a remote computing device if the flow value exceeds a threshold value.

Abstract: A method and system for analyzing at least one food item on a food plate is disclosed. A plurality of images of the food plate is received by an image capturing device. A description of the at least one food item on the food plate is received by a recognition device. The description is at least one of a voice description and a text description. At least one processor extracts a list of food items from the description; classifies and segments the at least one food item from the list using color and texture features derived from the plurality of images; and estimates the volume of the classified and segmented at least one food item. The processor is also configured to estimate the caloric content of the at least one food item.

Abstract: A system may utilize sound localization techniques, such as time-difference-of-arrival techniques, to estimate an audio-based sound source position from which a sound originates. An optical image or depth map of an area containing the sound source location may then captured and analyzed to detect an object that is known or expected to have produced the sound. The position of the object may also be determined based on the analysis of the optical image or depth map. The position of the sound source may then be determined based at least in part on the position of the detected object or on a combination of the audio-based sound source position and the determined position of the object.

Abstract: In order to classify the presented pose of a human hand, a feature distribution map of the pose is compared to reference feature distribution maps. To generate a feature distribution map, each contour point of a hand is analyzed to determine a corresponding feature set. The feature set of a contour point includes a distance feature and an angle feature of the contour point in relation to one of its neighboring contour points. The feature sets generated from an observed pose are compared to feature sets of reference poses to determine which of the reference poses most closely matches the presented pose.

Abstract: Techniques are described for detecting the presence and positions of fingers using depth information. For each of multiple target points represented by a depth image, an evaluation is performed using a region of points surrounding the pixel. The region has an inner zone and side zones. Multiple iterations are performed for each target pixel, with different sizes and orientations of the region being used in the different iterations. An average depth and an average surface normal direction are determined for each zone. The average depth and average surface normal direction of the inner zone are compared to the average depths and the average surface normal directions of the side zones to determine whether the target pixel is part of a finger.

Abstract: Techniques are described for determining edge pixels from a depth map in order to identify edges of an object within an environment. The techniques may fit a plane to a group of pixels comprising a subject pixel and neighbors of the subject pixel. Each pixel of the group may then be classified as either an inlier pixel or an outlier pixel, and a ratio of inlier pixels to outlier pixels of the group may be calculated. The techniques then compare this ratio to a threshold ratio to designate the subject pixel as either an edge pixel or a non-edge pixel. Edge pixels designated in this manner may then be used to identify object edges and, hence, objects in the environment.

Abstract: Devices and techniques are described for automatically calibrating a device such as a camera, projector, or both using a video stream. The device undergoing calibration is coupled to a computing device and configured to acquire video comprising a plurality of images. These images, at least some of the time, include targets having known characteristics present in an environment. From these acquired images calibration data comprising parameters such as intrinsic parameters, extrinsic parameters, or both, may be determined. The calibration data may be embedded or otherwise associated with the video stream as calibration matrix metadata.

Abstract: Described herein are techniques and systems to determine movement of an imaging device (egomotion) using an analysis of images captured the by imaging device. The imaging device, while in a first position, may capture a first image of an environment. The image may be a depth map, a still photograph, or other type of image that enables identification of objects, reference features, and/or other characteristics of the environment. The imaging device may then capture a second image from a second position within the environment after the imaging devices moves from the first position to the second position. A comparison of corresponding reference features from the first image and second image may be used to determine translation and rotation of the imaging device.

Abstract: Techniques are described for detecting a hand gesture made by a user. Fingertips of a hand may be identified and tracked over time. When a user contracts the fingertips from an extended position, hand spread may be calculated based on the area of the hand and fingers. The hand spread over time may be compared to a Gaussian function to evaluate whether the observed motion represents a grasping motion.

Abstract: Devices and techniques are described for automatically calibrating a projector system. The projector system undergoing calibration is coupled to a computing device and an automated positioning platform coupled to a target structure. The computing device acquires images using a calibrated camera of one or more patterns projected by the projector onto the target structure when the target structure is in a plurality of repeatable poses. From these acquired images, intrinsic projector parameters may be determined. Once determined, the parameters may be used to correct images produced by the projector or acquired by the camera which include a projected pattern or portion thereof.

Abstract: A computer implemented method for matching video data to a database containing a plurality of video fingerprints of the type described above, comprising the steps of calculating at least one fingerprint representing at least one query frame from the video data; indexing into the database using the at least one calculated fingerprint to find a set of candidate fingerprints; applying a score to each of the candidate fingerprints; selecting a subset of candidate fingerprints as proposed frames by rank ordering the candidate fingerprints; and attempting to match at least one fingerprint of at least one proposed frame.

Abstract: Devices and techniques are described for automatically calibrating a camera system. The camera system undergoing calibration is coupled to a computing device and an automated positioning platform coupled to a target structure. The computing device acquires images from the camera of the target structure in a plurality of repeatable poses. From these acquired images, intrinsic camera parameters may be determined. Once determined, the parameters may be used to correct images acquired by the camera system.

Abstract: A computer implemented method for matching video data to a database containing a plurality of video fingerprints of the type described above, comprising the steps of calculating at least one fingerprint representing at least one query frame from the video data; indexing into the database using the at least one calculated fingerprint to find a set of candidate fingerprints; applying a score to each of the candidate fingerprints; selecting a subset of candidate fingerprints as proposed frames by rank ordering the candidate fingerprints; and attempting to match at least one fingerprint of at least one proposed frame.

Abstract: A method and system for analyzing at least one food item on a food plate is disclosed. A plurality of images of the food plate is received by an image capturing device. A description of the at least one food item on the food plate is received by a recognition device. The description is at least one of a voice description and a text description. At least one processor extracts a list of food items from the description; classifies and segments the at least one food item from the list using color and texture features derived from the plurality of images; and estimates the volume of the classified and segmented at least one food item. The processor is also configured to estimate the caloric content of the at least one food item.

Abstract: A computer implemented method computer implemented method for deriving a fingerprint from video data is disclosed, comprising the steps of receiving a plurality of frames from the video data; selecting at least one key frame from the plurality of frames, the at least one key frame being selected from two consecutive frames of the plurality of frames that exhibiting a maximal cumulative difference in at least one spatial feature of the two consecutive frames; detecting at least one 3D spatio-temporal feature within the at least one key frame; and encoding a spatio-temporal fingerprint based on mean luminance of the at least one 3D spatio-temporal feature. The least one spatial feature can be intensity. The at least one 3D spatio-temporal feature can be at least one Maximally Stable Volume (MSV).

Abstract: A method and system for analyzing at least one food item on a food plate is disclosed. A plurality of images of the food plate is received by an image capturing device. A description of the at least one food item on the food plate is received by a recognition device. The description is at least one of a voice description and a text description. At least one processor extracts a list of food items from the description; classifies and segments the at least one food item from the list using color and texture features derived from the plurality of images; and estimates the volume of the classified and segmented at least one food item. The processor is also configured to estimate the caloric content of the at least one food item.

Abstract: A method and system for analyzing at least one food item on a food plate is disclosed. A plurality of images of the food plate is received by an image capturing device. A description of the at least one food item on the food plate is received by a recognition device. The description is at least one of a voice description and a text description. At least one processor extracts a list of food items from the description; classifies and segments the at least one food item from the list using color and texture features derived from the plurality of images; and estimates the volume of the classified and segmented at least one food item. The processor is also configured to estimate the caloric content of the at least one food item.

Abstract: A computer implemented method computer implemented method for deriving a fingerprint from video data is disclosed, comprising the steps of receiving a plurality of frames from the video data; selecting at least one key frame from the plurality of frames, the at least one key frame being selected from two consecutive frames of the plurality of frames that exhibiting a maximal cumulative difference in at least one spatial feature of the two consecutive frames; detecting at least one 3D spatio-temporal feature within the at least one key frame; and encoding a spatio-temporal fingerprint based on mean luminance of the at least one 3D spatio-temporal feature. The least one spatial feature can be intensity. The at least one 3D spatio-temporal feature can be at least one Maximally Stable Volume (MSV).