Abstract: Systems and methods for reducing motion blur in images or video in ultra low light with array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method for synthesizing an image from multiple images captured using an array camera includes capturing image data using active cameras within an array camera, where the active cameras are configured to capture image data and the image data includes pixel brightness values that form alternate view images captured from different viewpoints, determining sets of corresponding pixels in the alternate view images where each pixel in a set of corresponding pixels is chosen from a different alternate view image, summing the pixel brightness values for corresponding pixels to create pixel brightness sums for pixel locations in an output image, and synthesizing an output image from the viewpoint of the output image using the pixel brightness sums.

Abstract: Synthesizing a neural network from a plurality of component neural networks is disclosed. The method comprises mapping each component network to a respective graph node where each node is first labelled in accordance with the structure of a corresponding layer of the component network and a distance of the node from one of a given input or output. The graphs for each component network are merged into a single merged graph by merging nodes from component network graphs having the same first structural label. Each node of the merged graph is second labelled in accordance with the structure of the corresponding layer of the component network and a distance of the node from the other of a given input or output. The merged graph is contracted by merging nodes of the merged graph having the same second structural label. The contracted-merged graph is mapped to a synthesized neural network.

Abstract: Systems and methods in accordance with embodiments of the invention are configured to render images using light field image files containing an image synthesized from light field image data and metadata describing the image that includes a depth map. One embodiment of the invention includes a processor and memory containing a rendering application and a light field image file including an encoded image, a set of low resolution images, and metadata describing the encoded image, where the metadata comprises a depth map that specifies depths from the reference viewpoint for pixels in the encoded image. In addition, the rendering application configures the processor to: locate the encoded image within the light field image file; decode the encoded image; locate the metadata within the light field image file; and post process the decoded image by modifying the pixels based on the depths indicated within the depth map and the set of low resolution images to create a rendered image.

Abstract: Systems and methods for synthesizing high resolution images using image deconvolution and depth information in accordance embodiments of the invention are disclosed. In one embodiment, an array camera includes a processor and a memory, wherein an image deconvolution application configures the processor to obtain light field image data, determine motion data based on metadata contained in the light field image data, generate a depth-dependent point spread function based on the synthesized high resolution image, the depth map, and the motion data, measure the quality of the synthesized high resolution image based on the generated depth-dependent point spread function, and when the measured quality of the synthesized high resolution image is within a quality threshold, incorporate the synthesized high resolution image into the light field image data.

Abstract: A method of tracking an object across a stream of images comprises determining a region of interest (ROI) bounding the object in an initial frame of an image stream. A HOG map is provided for the ROI by: dividing the ROI into an array of M×N cells, each cell comprising a plurality of image pixels; and determining a HOG for each of the cells. The HOG map is stored as indicative of the features of the object. Subsequent frames are acquired from the stream of images. The frames are scanned ROI by ROI to identify a candidate ROI having a HOG map best matching the stored HOG map features. If the match meets a threshold, the stored HOG map indicative of the features of the object is updated according to the HOG map for the best matching candidate ROI.

Abstract: Systems and methods for automatically correcting apparent distortions in close range photographs that are captured using an imaging system capable of capturing images and depth maps are disclosed. In many embodiments, faces are automatically detected and segmented from images using a depth-assisted alpha matting. The detected faces can then be re-rendered from a more distant viewpoint and composited with the background to create a new image in which apparent perspective distortion is reduced.

Abstract: A method is disclosed for processing at least a portion of an input digital image comprising rows of pixels extending in two mutually perpendicular directions over a 2D field. The method comprises defining a kernel for processing an image, the kernel comprising at least one row of contiguous elements of the same non-zero value (such rows being referred to herein as equal-valued kernel regions), the equal-valued kernel regions, if more than one, extending parallel to one another. For each pixel in at least selected parallel rows of pixels within the image portion, the cumulative sum of the pixel is calculated by adding a value of the pixel to the sum of all preceding pixel values in the same row of the image portion. The kernel is convolved with the image portion at successive kernel positions relative to the image portion such that each pixel in each selected row is a target pixel for a respective kernel position.

Abstract: A dynamically reconfigurable heterogeneous systolic array is configured to process a first image frame, and to generate image processing primitives from the image frame, and to store the primitives and the corresponding image frame in a memory store. A characteristic of the image frame is determined. Based on the characteristic, the array is reconfigured to process a following image frame.

Abstract: A portrait lens configuration for meeting handheld device form factor constraints. First and second meniscus lenses each have a reflective surface to provide internal reflections for transmitting light toward a focal plane. A third lens is positioned between the meniscus lenses and the focal plane. The first lens includes an anterior concave surface having a reflective material extending over a portion thereof. Light received by the first meniscus lens can be transmitted therethrough. The reflective material is positioned along the anterior concave surface to receive light transmitted therethrough and reflected back from the second lens. In an associated method the first meniscus lens is positioned to receive light through a first of two opposing refractive surfaces. After each lens provides an internal reflection, reflected light is transmitted through the second of the two opposing surfaces and then through a bore positioned within the second lens to the third lens.

Abstract: A method of correcting an image obtained by an image acquisition device includes obtaining successive measurements, Gn, of device movement during exposure of each row of an image. An integration range, idx, is selected in proportion to an exposure time, te, for each row of the image. Accumulated measurements, Cn, of device movement for each row of an image are averaged across the integration range to provide successive filtered measurements, G, of device movement during exposure of each row of an image. The image is corrected for device movement using the filtered measurements G.

Abstract: A method of responding to a criterion-based request for information collected from users meeting the criterion while complying with a user-requested privacy requirement. In one embodiment a request is received for data comprising facial or audio expressions for users who meet the criterion. A program monitors activities indicative of user attention or user reaction based on face tracking, face detection, face feature detection, eye gaze determination, eye tracking, audio expression determination, or determination of an emotional state. When a user requests a high level of privacy, the timestream data collected for the user is aggregated with timestream data collected for other users into a statistical dataset by processing the timestreams to ensure the high level of privacy in the statistical dataset which is provided to a content provider without providing data collected for the user who has requested the high level of privacy.

Abstract: Systems and methods for training a conditional generator model are described. Methods receive a sample, and determine a discriminator loss for the received sample. The discriminator loss is based on an ability to determine whether the sample is generated by the conditional generator model or is a ground truth sample. The method determines a secondary loss for the generated sample and updates the conditional generator model based on an aggregate of the discriminator loss and the secondary loss.

Abstract: An image acquisition method operates in a hand held image acquisition device with a camera. A first image of a scene is obtained with the camera at a nominal exposure level. A number of relatively bright pixels and a number of relatively dark pixels within the first image are determined. Based on the number of relatively bright pixels, a negative exposure adjustment is determined and based on the number of relatively dark pixels, a positive exposure adjustment is determined. Respective images are acquired at the nominal exposure level; with the negative exposure adjustment; and with the positive exposure adjustment as component images for high dynamic range (HDR) image of the scene.

Abstract: A neural network image processing apparatus arranged to acquire images from an image sensor and to: identify a ROI containing a face region in an image; determine at plurality of facial landmarks in the face region; use the facial landmarks to transform the face region within the ROI into a face region having a given pose; and use transformed landmarks within the transformed face region to identify a pair of eye regions within the transformed face region. Each identified eye region is fed to a respective first and second convolutional neural network, each network configured to produce a respective feature vector. Each feature vector is fed to respective eyelid opening level neural networks to obtain respective measures of eyelid opening for each eye region. The feature vectors are combined and to a gaze angle neural network to generate gaze yaw and pitch values substantially simultaneously with the eyelid opening values.

Abstract: A convolutional neural network (CNN) for an image processing system comprises an image cache responsive to a request to read a block of N×M pixels extending from a specified location within an input map to provide a block of N×M pixels at an output port. A convolution engine reads blocks of pixels from the output port, combines blocks of pixels with a corresponding set of weights to provide a product, and subjects the product to an activation function to provide an output pixel value. The image cache comprises a plurality of interleaved memories capable of simultaneously providing the N×M pixels at the output port in a single clock cycle. A controller provides a set of weights to the convolution engine before processing an input map, causes the convolution engine to scan across the input map by incrementing a specified location for successive blocks of pixels and generates an output map within the image cache by writing output pixel values to successive locations within the image cache.

Abstract: An image processing system comprises at least one image sensor comprising a plurality of sub-pixels, and configured to provide a first image plane from a group of first sub-pixels selectively sensitive to a first NIR light band and a second image plane from a group of second sub-pixels selectively sensitive to a second NIR light band. An NIR light source is capable of separately emitting first NIR light corresponding to the first NIR light band and second NIR light corresponding to the second NIR light band. The system can be configured to operate according to at least a first working mode where a face detector is configured to detect at least a first face in the first image plane and a second face in the second image plane at a spatially non-coincident location to the first face.

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Abstract: A portable system providing augmented vision of surroundings. In one embodiment the system includes a helmet, a plurality of camera units and circuitry to generate a composite field of view from channels of video data. The helmet permits a user to receive a first field of view in the surroundings based on optical information received directly from the surroundings with the user's natural vision. The camera units are mounted about the helmet to generate the multiple channels of video data. Each camera channel captures a different field of view of a scene in a region surrounding the helmet.

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

Abstract: A method of tracking an object across a stream of images comprises determining a region of interest (ROI) bounding the object in an initial frame of an image stream. A HOG map is provided for the ROI by: dividing the ROI into an array of M×N cells, each cell comprising a plurality of image pixels; and determining a HOG for each of the cells. The HOG map is stored as indicative of the features of the object. Subsequent frames are acquired from the stream of images. The frames are scanned ROI by ROI to identify a candidate ROI having a HOG map best matching the stored HOG map features. If the match meets a threshold, the stored HOG map indicative of the features of the object is updated according to the HOG map for the best matching candidate ROI.