Abstract: A camera has two distinct image sensor arrays. The image sensor arrays are at least one of different types and different resolutions. The image sensor arrays have different dynamic ranges such that one of the image sensor arrays is a high dynamic range image sensor array configured to capture a wider range of light intensity than the other of the image sensor arrays. The camera has an imaging mechanism configured to combine information collected by each of the image sensor arrays to produce a high dynamic range image. The image sensor arrays include one low resolution image sensor array having a high dynamic range, and one high resolution image sensor array having a normal dynamic range smaller than the high dynamic range wherein luminance values within the normal dynamic range represent a subset of luminance values within the high dynamic range.

Abstract: Apparatuses and methods are disclosed that create a synthetic fovea in order to identify and highlight interesting portions of an image for further processing and rapid response. Synthetic foveal imaging implements a parallel processing architecture that uses reprogrammable logic to implement embedded, distributed, real-time foveal image processing from different sensor types while simultaneously allowing for lossless storage and retrieval of raw image data. Real-time, distributed, adaptive processing of multi-tap image sensors with coordinated processing hardware used for each output tap is enabled. In mosaic focal planes, a parallel-processing network can be implemented that treats the mosaic focal plane as a single ensemble rather than a set of isolated sensors. Various applications are enabled for imaging and robotic vision where processing and responding to enormous amounts of data quickly and efficiently is important.

Abstract: Electronic devices may include camera modules. A camera module may include an array camera having an array of lenses and an array of corresponding image sensors. Parallax correction and depth mapping methods may be provided for array cameras. A parallax correction method may include a global and a local parallax correction. A global parallax correction may be determined based on one-dimensional horizontal and vertical projections of edge images. Local parallax corrections may be determined using a block matching procedure. Further improvements to local parallax corrections may be generated using a relative block color saturation test, a smoothing of parallax correction vectors and, if desired, using a cross-check between parallax correction vectors determined for multiple image sensors. Three dimensional depth maps may be generated based on parallax correction vectors.

Abstract: The present invention provides an imaging apparatus capable of performing a live view display showing a multiple image combination result image generated by combining a through-the-lens image with an already captured image at a more suitable combination ratio according to a user's intended operation during multiple exposure shooting. The imaging apparatus switches a multiple image combination result image displayed as a live view from a multiple image combination result image combined at a first combination ratio to a multiple image combination result image combined at a second combination ratio allowing a user to preliminarily check a combination ratio for generating a multiple image combination result image to be recorded into a recording medium, according to a user's operation.

Abstract: A system and method for video navigation is disclosed. The system and method may include one or more video cameras configured to capture views external to the vehicle, a control mechanism for adjusting the sight-line of the one or more video cameras, and a display panel positioned within the vehicle for displaying the views captured within the sight-line of the one or more video cameras.

Abstract: An imaging system may include a multi-channel image sensor. In a multi-channel image sensor, multiple image streams from a common pixel array may be provided to a common interface transport for output on multiple logic channels. Image streams may include high-definition (HD) video and snapshot image data. Video image data may be resized using resize circuitry on the image sensors. Snapshot image data may be processed by the image sensor. Full-resolution portions of snapshot images may be output concurrently with resized HD video image data on multiple logic channels to an image signal processor. In an alternative embodiment, stack memory on the image sensor may be used to store full-resolution snapshot image data. Full-resolution snapshot image data may then be output concurrently with resized HD video image data at a slower data rate than the video image data.

Abstract: An image capture device (2) includes a sensor (4) having an active area comprising a plurality of pixels (6) and a shutter array (8) for controlling the exposure of individual pixels. The pixels are grouped in a plurality of pixel subsets and are arranged to capture a plurality of time-separated lo-res images, which can be viewed sequentially as a movie or data from all the pixels can be used to form a hi-res still image.

Abstract: A method for imaging includes capturing images of an object using a matrix of detector elements and performing a processing operation in the detector elements under control of a global clock. Results of the processing operation are read out under control of a rolling clock, which is unsynchronized with the global clock.

Abstract: An image acquisition system comprises: a single set of focussing optics focussing an incident beam from a scene to be imaged on a first sensor and a second sensor exhibiting identical spectral response curves; followed by separation means separating the irradiance of the beam coming from the focussing optics into first and second predetermined fractions, the separation means orienting the first fraction towards the first sensor and the second fraction towards the second sensor; control means for controlling a first acquisition time of the scene to be imaged by the first sensor to obtain a first image and/or a second acquisition time of the scene to be imaged by the second sensor to obtain a second image; and merging means that merge the first image coming from the first sensor and the second image coming from the second sensor to generate a final image.

Abstract: A method is performed to refocus a digital photographic image comprising a plurality of pixels. In the method, a set of images is computed corresponding to the digital photographic image and focused at different depths. Refocus depths for at least a subset of the pixels are identified and stored in a look-up table. At least a portion of the digital photographic image is refocused at a desired refocus depth determined from the look-up table.

Abstract: New systems and methods are hereby provided that inherently and naturally resolve the challenges of synthesizing coordinated inputs from multiple cameras. For example, a multi-sensor mediator may collect the input data from multiple sensors, and generate a composite signal that encodes the combined data from the different sensors. The multi-sensor mediator may then relay the composite signal to a sensor controller, as if the signal were coming from a single sensor. A computing device that receives the input from the sensor controller may then generate an output based on the composite signal, which may include processing the composite signal to combine the separate signals from the different sensors, such as to provide a stereo image output, for example. The multi-sensor mediator makes such an output possible by ensuring coordinated input and processing of the input from the multiple sensors, for example.

Abstract: A variety of optical arrangements and methods of modifying or enhancing the optical characteristics and functionality of these optical arrangements are provided. The optical arrangements being specifically designed to operate with camera arrays that incorporate an imaging device that is formed of a plurality of imagers that each include a plurality of pixels. The plurality of imagers include a first imager having a first imaging characteristics and a second imager having a second imaging characteristics. The images generated by the plurality of imagers are processed to obtain an enhanced image compared to images captured by the imagers. In many optical arrangements the MTF characteristics of the optics allow for contrast at spatial frequencies that are at least as great as the desired resolution of the high resolution images synthesized by the array camera, and significantly greater than the Nyquist frequency of the pixel pitch of the pixels on the focal plane, which in some cases may be 1.

Abstract: A method and apparatus are disclosed for forming an image signal by receiving a flux of photons at a convex photodetector such as a hemispherical photodetector. The convex photodetector includes a plurality of photosensors. Each photosensor has a different orientation with respect to a propagation vector of the flux of photons. The photosensors generate a respective plurality of intensity signals. Each of the intensity signals is related to the respective orientation of the photosensor that generates it. The intensity signals are received by a signal processor, such as a digital signal processor, which uses the intensity signals to compute an image signal related to the intensity signals and thereby produce a focused output image.

Abstract: An image processing apparatus that generates image data having a high dynamic range by combining a plurality of image data having different exposure amounts includes: an input unit configured to input a plurality of M-bit image data; a determining unit configured to determine reference image data serving as an exposure reference from among the plurality of input M-bit image data; a composition unit configured to combine the plurality of M-bit image data, and generate one N-bit composite image data (N>M); a characteristic parameter setting unit configured to set a characteristic parameter of tone compression on the N-bit composite image data such that a predetermined reference luminance in the reference image data does not change; and a generating unit configured to generate M-bit composite image data by compressing a luminance tone of an image component based on the set characteristic parameter.

Abstract: A digital image capturing system and method are disclosed. A system incorporating teachings of the present disclosure may include a processing engine that performs digital image processing functions like noise reduction, gain control, white balance and others. In one embodiment, the image capturing system may have a first image sensor operable to capture a first view of a scene and a second image sensor operable to capture a second and different view of the scene. A selector may be employed to selectively route either the first view information or the second view information onto the processing engine. In preferred embodiments, the sensors may be CMOS sensors and, as such, relatively inexpensive. By allowing two inexpensive sensors to share image processing electronics, an image capturing system incorporating teachings of the present disclosure may be able to offer compelling functional characteristics at a reasonable price.

Abstract: High dynamic range imaging includes creating a high dynamic range image file for a subject image captured from a first angle. The high dynamic range image file is created by an image capturing device and is stored on the image capturing device. High dynamic range imaging also includes receiving, at the image capturing device over a communications network, a second high dynamic range image file for the subject image captured from a second angle. The second high dynamic range image file is subject to an authorization requirement. High dynamic range imaging further includes creating, by the image capturing device, a composite high dynamic range image file by combining elements of both of the high dynamic range image files, and generating, by the first image capturing device, a three-dimensional high dynamic range image from the composite high dynamic range image file.

Abstract: A large format digital camera has a primary camera system configured for collecting panchromatic image data and a secondary camera system configured for collecting color image data. The secondary camera system has an optical system that has a longer focal length than the optical system of the primary camera system. The resolution of the secondary camera system is greater than the resolution of the primary camera system. The footprint of images produced by the primary camera system is larger in size than the footprint of images produced by the secondary camera system. Images produced by the primary camera system offer information for performing image-based georeferencing by means of photogrammetric triangulation. Images produced by the secondary camera system offer a high-resolution narrow angle color image suitable for use in ortho image production.

Abstract: In a typical two-dimensional rectilinear image, all pixel widths are constant across the image. If the two-dimensional rectilinear image is mapped to a cylindrical surface, the pixels at the edge of the image are viewed at an angle, making the effective width of the pixels at the edge smaller than the effective width of the pixels in the interior of the image. This problem may be solved by applying a tuned distortion curve to the two-dimensional rectilinear image, which makes the pixels near the edge of the image wider than pixels in the interior of the image. When pixels that have been distorted in this manner are mapped to a cylindrical surface, the effective pixel width is uniform. A lens with a tuned distortion curve built into its optical profile captures at least a portion of the surrounding imagery such that the resultant image has uniform resolution when projected onto a virtual cylinder centered around a viewer.

Abstract: A camera system (100) comprises a plurality of imaging units (190), wherein each imaging unit (190) includes a lens unit (120) and an aperture unit (110), and wherein the apertures (115) of the aperture units (110) have different size during an exposure period. A plurality of images of the same scene are captured at different aperture sizes during an exposure period, wherein the images are shifted versions of the scene and feature different depth-of-field and exposure value ranges. A processing unit (200) applies a high-dynamic resolution and/or a super-resolution method on the basis of the obtained images.

Abstract: A dual sensor camera that uses two aligned sensors each having a separate lens of different focal length but the same f-number. The wider FOV image from one sensor is combined with the narrower FOV image from the other sensor to form a combined image. Up-sampling of the wide FOV image and down-sampling of the narrow FOV image is performed. The longer focal length lens may have certain aberrations introduced so that Extended Depth of Field (EDoF) processing can be used to give the narrow FOV image approximately the same depth of field as the wide FOV image so that a noticeable difference in depth of field is not see in the combined image.

Abstract: Techniques for reconstructing an image of an object are provided. In some embodiments, an image reconstructing system includes one or more cameras, one or more light sources and an image processor. The cameras are configured to capture images of an object and the light sources are configured to irradiate the object. The image processor is configured to process the images to generate a first representation of the object and a second representation of the object, and to generate a reconstructed image of the object based on the first representation and the second representation.

Abstract: An electronic device and method for controlling a plurality of cameras includes determining a selected camera though a control system, displaying a route map of the selected camera, and calculating motion parameters of a touch motion on a touch panel of the electronic device. The motion parameters are sent to the control system to control the selected camera according to the motion parameters. The touch panel displays acquired video images from the selected camera.

Abstract: A digital camera system including a first video capture unit for capturing a first digital video sequence of a scene and a second video capture unit that simultaneously captures a second digital video sequence that includes the photographer. A data processor automatically analyzes first digital video sequence to determine a low-interest spatial image region. A facial video sequence including the photographer's face is extracted from the second digital video sequence, and inserted into the low-interest spatial image region in the first digital video sequence to form the composite video sequence.

Abstract: A system and method for creating special effects comprising capturing an image of an object or objects in motion from each of a plurality of cameras aligned in an array, assembling the images from each camera into a series of images in a manner providing a simulation of motion from a perspective along the array with respect to the object or objects within the images when a series of images is displayed in sequence, wherein certain object or objects appear substantially stationary in time during the simulation of motion.

Abstract: A method for forming a composite video sequence, from a first digital video sequence captured of a scene by a photographer, and a second digital video sequence captured simultaneously with the first digital video sequence that includes the photographer. The first digital video sequence is analyzed to determine a low-interest spatial image region. A facial video sequence including the photographer's face is extracted from the second digital video sequence, and inserted into the low-interest spatial image region in the first digital video sequence to form the composite video sequence.

Abstract: An imaging apparatus including a capturing section which captures an image at a first viewing angle, a capturing control section which performs a plurality of image capturing operations by the capturing section, a generation section which generates a composite image reproducing an image captured at a second viewing angle that is wider than the first viewing angle by combining a plurality of images acquired by the plurality of image capturing operations by the capturing control section, a judgment section which judges whether or not an uncaptured area that has not been captured by the capturing control section exists within the composite image with the second viewing angle generated by the generation section, and a notification section which notifies a user of a judgment result, when the judgment section judges that an uncaptured area exists.

Abstract: In one embodiment, a light sensor includes four cell arrays, one for each color of the Bayer pattern, and four lenses each focusing the light coming from the scene to be captured on a respective cell array. The lenses are oriented such that at least a second green image, commonly provided by the fourth cell array, is both horizontally and vertically shifted (spaced) apart by half a pixel pitch from a first (reference) green image. In a second embodiment, the four lenses are oriented such that the red and blue images are respectively shifted (spaced) apart by half a pixel pitch from the first or reference green image, one horizontally and the other vertically, and the second green image is shifted (spaced) apart by half a pixel pitch from the reference green image both horizontally and vertically.

Abstract: An image capture device (2) includes a sensor (4) having an active area comprising a plurality of pixels (6) and a shutter array (8) for controlling the exposure of individual pixels. The pixels are grouped in a plurality of pixel subsets and are arranged to capture a plurality of time-separated lo-res images, which can be viewed sequentially as a movie or combined to form a hi-res still image.

Abstract: This disclosure is directed to camera systems including a camera array, data buses shared by the cameras, and one or more processors that process images captured by the camera array. In one aspect, a camera system (450) includes a camera array (452), a controller (454), a number of camera buses (458-465), and a computing device (456) connected to the controller. Each camera is connected to one or more adjacent cameras and includes one or more processors. Each camera bus connects a subset of cameras of the camera array to the controller. And each camera can be operated to send to the controller only the image data used by the computing device to generate virtual images and receive instructions from the controller over a camera bus.

Abstract: An image registration method involves computing horizontal and vertical integral projection vectors for first and second distorted or partially distorted images. The images are registered by applying a scale factor estimation between the first and second images on the horizontal and vertical integral projection vectors.

Abstract: An image capture apparatus including: an image capturing unit; a light emitting unit; a first image capture controller configured to control the image capturing unit to capture a plurality of images at a first timing; a second image capture controller configured to control the image capturing unit to capture an image under a capturing condition illuminated by the light emitting unit at a second timing that is either directly before or directly after the first timing; an addition combination section configured to positionally align the images captured under control by the first image capture controller and to perform an addition combination on the aligned images to generate a combined image; and a combination section configured to combine the combined image generated by the addition combination section with the image captured under control by the second image capture controller.

Abstract: An image registration method involves computing horizontal and vertical integral projection vectors for first and second distorted or partially distorted images or distortion-corrected images, or both. The images are registered by applying a translation, rotation and/or scale factor estimation between the first and second images on the horizontal and vertical integral projection vectors.

Abstract: A precision motion platform carrying an imaging device under a large-field-coverage lens enables capture of high resolution imagery over the full field in an instantaneous telephoto mode and wide-angle coverage through temporal integration. The device permits automated tracking and scanning without movement of a camera body or lens. Coupled use of two or more devices enables automated range computation without the need for subsequent epipolar rectification. The imager motion enables sample integration for resolution enhancement. The control methods for imager positioning enable decreasing the blur caused by both the motion of the moving imager or the motion of an object's image that the imager is intended to capture.

Abstract: The present invention relates to a method to create and reproduce a panoramic sound image. Initially, there is a plurality of images associated with sound sequences, these images are for example taken using a camera, this camera having recorded the ambient sound at the time of taking the photograph. The ambient sound is then filtered to keep only the stationary component. Then, the images are concatenated to form a unique image called “panoramic”, each sound sequence being associated with a part of the panoramic image containing the image associated with this sequence. Finally, the commands are introduced to display the parts of the panoramic image. The command also enables to reproduce continually the stationary component of at least a sound content associated with at least a displayed part of the panoramic image. According to an improvement, the sound sequence is reproduced continually until the introduction of a new command.

Abstract: The method includes obtaining identification information representing a multi-view video data stream, obtaining initialization information of a reference picture list for a random access slice based on the identification information, initializing the reference picture list using the initialization information, obtaining modification information for the initialized reference picture list, determining an assignment modification value for modifying an inter-view reference index in the initialized reference picture list according to the modification information, and modifying the initialized reference picture list for inter-view prediction using the determined assignment modification value.

Abstract: Among pickup image data items of multiple frames picked up at different time instants, an image of a certain frame is regarded as a synthesizing base image. The synthesizing base image and an image of another frame are used to perform synthesizing processing. Specifically, an object-of-synthesis area is designated in the frame image regarded as the synthesizing base image, and processing of replacing pixel data in the object-of-synthesis area with pixel data in another frame is carried out. For example, if a person having his/her eyes closed appears in the synthesizing base image that is a group-photograph image, the part of the person's face is regarded as the object-of-synthesis area. As another frame, an image in which the person has his/her eyes left open is searched. Pixel data in the facial image part showing the open eyes is fetched and synthesized with the synthesizing base image. Thus, a group photograph in which all people have their eyes left open is produced.

Abstract: An electronic camera for generating an image having a wide dynamic range by synthesizing two image pickup signals of different exposure amount generated by using an electronic shutter function and unit for shutting off light, shading of image pickup signals resulting from difference in charge accumulating time among the pixels of image pickup device due to operation from opened state to closed state of the unit for shutting off light being corrected by a shading correction unit to form a synthesized image without an occurrence of false color due to shading.

Abstract: An approach for enabling users to generate an interpolated view of content based on footage captured by multiple cameras is described. An interpolation platform receives a plurality of images from a plurality of video cameras providing overlapping fields of view. A camera angle that is different from angles provided by the plurality of cameras is selected. The interpolation platform then generates an interpolated image corresponding to the selected camera angle using a portion or all of the plurality of images from the plurality of cameras.

Abstract: The generation of a false color at the periphery is prevented in an imaging device including a compound eye optical system in which parallax occurs for each color information.

Abstract: Array cameras and imager arrays configured to capture high dynamic range light field image data and methods of capturing high dynamic range light field image data in accordance with embodiments of the invention are disclosed. Imager arrays in accordance with many embodiments of the invention include multiple focal planes with associated read out and sampling circuitry. The sampling circuitry controls the conversion of the analog image information into digital image data. In certain embodiments, the sampling circuitry includes an Analog Front End (AFE) and an Analog to Digital Converter (ADC). In several embodiments, the AFE is used to apply different amplification gains to analog image information read out from pixels in a given focal plane to provide increased dynamic range to digital image data generated by digitizing the amplified analog image information. The different amplifications gains can be applied in a predetermined manner or on a pixel by pixel basis.

Abstract: An image processing system providing a monitor image in which an image of a characteristic region is easily viewed. The image processing system includes an image obtaining section that obtains a moving image; a characteristic region information obtaining section that obtains information indicating positions of characteristic regions in a plurality of moving image constituting images included in the moving image; and an image generating section that generates display images having substantially the same size as each other, by reducing or enlarging each of images of the characteristic regions included in the plurality of moving image constituting images, based on the positions indicated by the information obtained by the characteristic region information obtaining section.

Abstract: An apparatus and method for acquiring image data from a scanned, multi-bank time-delay and integrate (TDI) focal plane array (FPA) detector. Specifically a method and apparatus for warping and combining sequentially-acquired image data of a scene portion from multiple TDI detector banks into a single image having improved image quality, thereby providing improved FPA sensitivity. Also, a method and apparatus for enabling sensitivity and areal rate trade-offs in a multi-bank, scanning TDI FPA based on the number of TDI banks being used for sequential imaging of the same scene portion.

Abstract: A coverage area picture imaging a maximum area is displayed on a coverage area picture display. A camera is moved within a predetermined range and a plurality of frames obtained as a result form a picture. The pixels of the picture is decimated in vertical and horizontal directions and form a thumbnail as the coverage area picture. The coverage area picture display presents a display indicating a direction in which the camera is currently directed for picture photographing. In accordance with the display, a photographing direction is controlled. A plurality of frames are photographed with respect to a designated position, then stored, and displayed. A whole picture display presents a whole panorama picture. A selective picture display presents a frame at the position designated within the whole panorama picture as a selected picture.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, are described for rendering a mosaic from digital images using information about location and orientation of an image capturing device, and further about optics settings for the image capturing device when the digital images were captured. In one aspect, methods include generating respective virtual image sheets for frames captured from different camera locations and different camera orientations. Generating the virtual image sheets includes projecting texture maps of the captured frames over wire frames corresponding to optics settings of the camera. The methods further include positioning the generated virtual image sheets at locations and orientations within a viewing space that correspond to the different camera locations and the different orientations. The methods also include rendering the positioned virtual image sheets into a mosaic viewed from a reference point of the viewing space.

Abstract: Multiview videos are acquired of a scene with corresponding cameras arranged at poses, such that there is view overlap between any pair of cameras. V-frames are generated from the multiview videos. The V-frames are encoded using only spatial prediction. Then, the V-frames are inserted periodically in an encoded bit stream to provide random temporal access to the multiview videos. Additional view dependency information enables the decoding of a reduced number of frames prior to accessing randomly a target frame for a specified view and time, and decoding the target frame. The method also decodes multiview videos by maintaining a reference picture list for a current frame of a plurality of multiview videos, and predicting each current frame of the plurality of multiview videos according to reference pictures indexed by the associated reference picture list.

Abstract: The present invention provides a method and system for generating a surrounding seamless bird-view image with distance assistance interface, which process a plurality of surrounding images associated with a carrier captured by different image sensing devices. After that, an assistant marking is determined according to the relationship between acquired image space and the real space and thereby is formed on the surrounding seamless bird-view image so as to assist the driver in the carrier for avoiding collision or events affecting the safety being occurred.

Abstract: Various embodiments utilize multiple built-in cameras on a mobile device, e.g., a phone, to capture images, individuals of which include a portion from each of the cameras. The ratio and layout of the portions from different cameras can be adjusted before the image is captured and stored. In at least some embodiments, individual cameras face different directions, and images captured by one of the cameras can be incorporated into images captured by another of the cameras. For example, in at least some embodiments, particularly those in which a user's image is captured, the user's image can be extracted from the view of a first camera, such as a front-facing camera on a mobile device, and displayed to a user in the foreground of the image captured by a second camera, such as a landscape image captured by a back-facing camera on the mobile device.

Abstract: A control apparatus which controls a capturing device so as to obtain a plurality of captured images including an entire region of an object by capturing the object by a plurality of times while changing a capturing direction of the capturing device, the apparatus comprises: a derivation unit configured to derive a corrected capturing range when at least one of lens aberration correction and perspective correction which are based on a position and the capturing direction of the capturing device with respect to the object is applied to a capturing range obtained upon capturing the object by the capturing device; and a determination unit configured to determine a plurality of capturing ranges which cover the entire region of the object so that two corrected capturing ranges corresponding to two adjacent capturing ranges, respectively, overlap each other.

Abstract: In one embodiment, a light sensor includes four cell arrays, one for each color of the Bayer pattern, and four lenses each focusing the light coming from the scene to be captured on a respective cell array. The lenses are oriented such that at least a second green image, commonly provided by the fourth cell array, is both horizontally and vertically shifted (spaced) apart by half a pixel pitch from a first (reference) green image. In a second embodiment, the four lenses are oriented such that the red and blue images are respectively shifted (spaced) apart by half a pixel pitch from the first or reference green image, one horizontally and the other vertically, and the second green image is shifted (spaced) apart by half a pixel pitch from the reference green image both horizontally and vertically.

Abstract: A method for compressing videos obtains videos from a storage device, selects a main frame from the videos at each time layer and each angle, and further selects a new main frame after each time interval and angle interval. Then, the method obtains frames between two adjacent main frames, and stores different pixels between each of the obtained frames and a previous main frame of the two adjacent main frames, so as to obtain composite images of the videos.