Abstract: Various examples are directed to systems and methods for determining foreground regions in video frames. A computing device may select from a first frame of a video, a plurality of scene point locations and divide the first frame into a plurality of sections. For a first section of the first frame, the computing device may generate a first vector subspace, basis vectors of the first vector subspace are trajectories of scene point locations in the first section. The computing device may determine that a projection error for a first scene point location in the first section is greater than a projection error threshold and write an indication of the first pixel value to a listing of foreground pixel values.

Abstract: A system and method for selecting portions of video data from preview video data is provided. The system may extract image features from the preview video data and discard video frames associated with poor image quality based on the image features. The system may determine similarity scores between individual video frames and corresponding transition costs and may identify transition points in the preview video data based on the similarity scores and/or transition costs. The system may select portions of the video data for further processing based on the transition points and the image features. By selecting portions of the video data, the system may reduce a bandwidth consumption, processing burden and/or latency associated with uploading the video data or performing further processing.

Abstract: The location of a user's head, for purposes such as head tracking or motion input, can be determined. For example, a computing device (e.g., a tablet, mobile phone, etc.) can utilize one or more detectors or other image classifiers to detect the presence objects such as features of the face in images captured by one or more cameras of the computing device. Based on the detected feature(s), the subject technology provides embodiments for detecting a heart rate of the user of the computing device without requiring invasive procedures or addition sensors on the computing device by selecting a region of interest for analyzing color or other image data measurements over a period of time. Changes in color at the select region of interest may be further processed to extract a user's heart rate using techniques described further herein.

Abstract: Various embodiments enable a primary user to be identified and tracked using stereo association and multiple tracking algorithms. For example, a face detection algorithm can be run on each image captured by a respective camera independently. Stereo association can be performed to match faces between cameras. If the faces are matched and a primary user is determined, a face pair is created and used as the first data point in memory for initializing object tracking. Further, features of a user's face can be extracted and the change in position of these features between images can determine what tracking method will be used for that particular frame.

Abstract: A system and method for generating preview data from video data and using the preview data to select portions of the video data or determine an order with which to upload the video data. The system may sample video data to generate sampled video data and may identify portions of the sampled video data having complexity metrics exceeding a threshold. The system may upload a first portion of the video data corresponding to the identified portions while omitting a second portion of the video data. The system may determine an order with which to upload portions of the video data based on a complexity of the video data. Therefore, portions of the video data that may require additional processing after being uploaded may be prioritized and uploaded first. As a result, a latency between the video data being uploaded and a video summarization being received is reduced.

Abstract: A system and method for generating preview data from video data and using the preview data to select portions of the video data or determine an order with which to upload the video data. The system may sample video data to generate sampled video data and may identify portions of the sampled video data having complexity metrics exceeding a threshold. The system may upload a first portion of the video data corresponding to the identified portions while omitting a second portion of the video data. The system may determine an order with which to upload portions of the video data based on a complexity of the video data. Therefore, portions of the video data that may require additional processing after being uploaded may be prioritized and uploaded first. As a result, a latency between the video data being uploaded and a video summarization being received is reduced.

Abstract: According to one aspect of the teachings presented herein, a machine vision system includes one or more sensor units that are each advantageously configured to use different pairings among a set of spaced-apart image sensors, to provide redundant object detection for a primary monitoring zone, while simultaneously providing for the detection of objects that may shadow the primary monitoring zone. Further, a plurality of “mitigations” and enhancements provide safety-of-design and robust operation. Such mitigations and enhancements include, for example, bad-pixel detection and mapping, cluster-based pixel processing for improved object detection, test-image injection for fault detection, dual-channel, redundant processing for safety-critical object detection, temporal filtering to reduce false detections, and the use of high dynamic range (HDR) images, for improved operation over variable ambient lighting conditions.

Abstract: According to one aspect of the teachings presented herein, a projective volume monitoring apparatus is configured to detect objects intruding into a monitoring zone. The projective volume monitoring apparatus is configured to detect the intrusion of objects of a minimum object size relative to a protection boundary, based on an advantageous processing technique that represents range pixels obtained from stereo correlation processing in spherical coordinates and maps those range pixels to a two-dimensional histogram that is defined over the projective coordinate space associated with capturing the stereo images used in correlation processing. The histogram quantizes the horizontal and vertical solid angle ranges of the projective coordinate space into a grid of cells. The apparatus flags range pixels that are within the protection boundary and accumulates them into corresponding cells of the histogram, and then performs clustering on the histogram cells to detect object intrusions.

Abstract: A system determines if someone watching a live video feed looks or moves away from a display screen, and when their attention is back on the display, provides an accelerated recap of the content that they missed. The video component of the feed may be shown as a series of selected still images or clips from the original feed, while audio and/or text captioning is output at an accelerated rate. The rate may be adaptively adjusted to maintain a consistent speed, and superfluous content may be omitted. When the recap catches up to the live feed, output returns to regular speed.

Abstract: The subject technology provides embodiments for tracking a user's face/head (or another object) using one or more cameras provided by a computing device. Embodiments implement exposure sweeping based on an average intensity of a current scene to a target intensity for a given image. If a face is not detected, an exposure duration and/or gain may be adjusted and the face detection is performed again. Once the face is detected, an average intensity of a virtual bounding box surrounding the detected face is determined and exposure sweeping may be performed solely within the virtual bounding box to reach a target intensity. When the average intensity is within a predetermined threshold of the target intensity, the detected face may be at an optimal exposure. Embodiments also provide for switching to another camera(s) of the computing device when not detecting a face in the image upon performing a full exposure sweep.

Abstract: According to one aspect of the teachings presented herein, a projective volume monitoring apparatus is configured to protect against the intrusion of crawling and walking persons into a monitoring zone. The apparatus acquires stereo images of a first monitoring zone that begins at a defined height above a floor and of a second monitoring zone that lies below the first monitoring zone and extends downward to a floor or other surface on which persons may walk or crawl, and processes the stereo images to obtain range pixels. The apparatus detects object intrusions within the first monitoring zone by processing the corresponding range pixels using first object detection parameters that are tuned for the detection of walking or running persons, and detects object intrusions within the second monitoring zone by processing the corresponding range pixels using second object detection parameters that are tuned for the detection of crawling or prone persons.

Abstract: A computing device can acquire a set of images, each image including at least a portion of a user's face. The images can be acquired using one or more cameras and/or from an image library/database associated with the user. Based on the images including the user's face (or portions thereof), a virtual representation for the user's face can be generated. The device can subsequently receive or identify an image including a facial representation (e.g., face or portion thereof) to be adjusted. The device can analyze the image including the facial representation and determine that the facial representation sufficiently matches the virtual representation. Using the virtual representation, (at least a portion of) the face can be adjusted. For example, one or more variations or details associated with the user's face, which are provided via the virtual representation, can be used to replace, improve, or otherwise modify the face in the image.

Abstract: Various embodiments enable a primary user to be identified and tracked using stereo association and multiple tracking algorithms. For example, a face detection algorithm can be run on each image captured by a respective camera independently. Stereo association can be performed to match faces between cameras. If the faces are matched and a primary user is determined, a face pair is created and used as the first data point in memory for initializing object tracking. Further, features of a user's face can be extracted and the change in position of these features between images can determine what tracking method will be used for that particular frame.

Abstract: Object tracking, such as may involve face tracking, can utilize different detection templates that can be trained using different data. A computing device can determine state information, such as the orientation of the device, an active illumination, or an active camera to select an appropriate template for detecting an object, such as a face, in a captured image. Information about the object, such as the age range or gender of a person, can also be used, if available, to select an appropriate template. In some embodiments instances of templates can be used to process various orientations, while in other embodiments specific orientations, such as upside down orientations, may not be processed for reasons such as rate of inaccuracies or infrequency of use for the corresponding additional resource overhead.

Abstract: According to one aspect of the teachings herein, a method and apparatus detect mechanical misalignments in a machine vision system during run-time operation of the machine vision system, and compensate image processing based on the detected misalignments, unless the detected misalignments are excessive. Excessive misalignments may be detected by determining a worst-case error based on them. If the worst-case error exceeds a defined limit, the machine vision system transitions to a fault state. The fault state may include disrupting operation of a hazardous machine or performing one or more other fault-state operations. Among the detected misalignments are internal misalignments within individual cameras used for imaging, and relative misalignments between cameras. The method and apparatus may further perform run-time verifications of focus and transition the machine vision system to a fault state responsive to detecting insufficient focal quality.

Abstract: The subject technology provides embodiments for tracking a user's face/head (or another object) using one or more cameras provided by a computing device. Embodiments implement exposure sweeping based on an average intensity of a current scene to a target intensity for a given image. If a face is not detected, an exposure duration and/or gain may be adjusted and the face detection is performed again. Once the face is detected, an average intensity of a virtual bounding box surrounding the detected face is determined and exposure sweeping may be performed solely within the virtual bounding box to reach a target intensity. When the average intensity is within a predetermined threshold of the target intensity, the detected face may be at an optimal exposure. Embodiments also provide for switching to another camera(s) of the computing device when not detecting a face in the image upon performing a full exposure sweep.

Abstract: In one aspect of the teachings herein, a 3D imaging apparatus uses a “texture tool” or facilitates the use of such a tool by an operator, to add artificial texture to a scene being imaged, for 3D imaging of surfaces or background regions in the scene that otherwise lack sufficient texture for determining sufficiently dense 3D range data. In at least one embodiment, the 3D imaging apparatus provides for iterative or interactive imaging, such as by dynamically indicating that one or more regions of the scene require artificial texturing via the texture tool for accurate 3D imaging, and then indicating whether or not the addition of artificial texture cured the texture deficiency and/or whether any other regions within the image require additional texture.

Abstract: In one aspect, this disclosure presents a method and apparatus for verifying that minimum object contrast requirements are met within a region representing a volume to be monitored by a machine vision system. In complementary fashion, the disclosure also presents a methodology for constraining the positions of the lighting sources to be used for illuminating the monitored volume at a minimum height above the floor, and for the use of a key light that provides asymmetrical lighting within the monitored volume relative to the camera(s) used for imaging the monitored volume. Correspondingly, the disclosure also presents a method and apparatus for monitoring for proper operation of the key light and responding to improper operation. The minimum contrast verification and key light monitoring operations can be implemented using standalone apparatuses, or can be incorporated into the machine vision system.

Abstract: According to one aspect of the teachings presented herein, a machine vision system includes one or more sensor units that are each advantageously configured to use different pairings among a set of spaced-apart image sensors, to provide redundant object detection for a primary monitoring zone, while simultaneously providing for the detection of objects that may shadow the primary monitoring zone. Further, a plurality of “mitigations” and enhancements provide safety-of-design and robust operation. Such mitigations and enhancements include, for example, bad-pixel detection and mapping, cluster-based pixel processing for improved object detection, test-image injection for fault detection, dual-channel, redundant processing for safety-critical object detection, temporal filtering to reduce false detections, and the use of high dynamic range (HDR) images, for improved operation over variable ambient lighting conditions.

Abstract: In one aspect of the teachings herein, a 3D imaging apparatus uses a “texture tool” or facilitates the use of such a tool by an operator, to add artificial texture to a scene being imaged, for 3D imaging of surfaces or background regions in the scene that otherwise lack sufficient texture for determining sufficiently dense 3D range data. In at least one embodiment, the 3D imaging apparatus provides for iterative or interactive imaging, such as by dynamically indicating that one or more regions of the scene require artificial texturing via the texture tool for accurate 3D imaging, and then indicating whether or not the addition of artificial texture cured the texture deficiency and/or whether any other regions within the image require additional texture.