Framing of Images in an Image Capture Device
Disclosed are various embodiments of framing and/or reframing an image captured by an image capture device. Framing characteristics associated with an image are identified. A determination is made regarding whether the framing characteristics comply with one or more framing guidelines. The image can be reframed if the framing characteristics do not comply with framing guidelines.
Latest BROADCOM CORPORATION Patents:
This application claims priority to co-pending U.S. provisional application entitled, “Image Capture Device Systems and Methods,” having Ser. No. 61/509,747, filed Jul. 20, 2011, which is entirely incorporated herein by reference.
BACKGROUNDUsers of image capture devices (e.g., still cameras, video cameras, etc.) may sometimes improperly frame an image that is captured by the device. In other words, a user may capture an image of a subject without framing the subject ideally. In some cases, the subject of an image may not be centered in the frame, the subject may occupy too little or too much of the frame relative to any background and/or foreground image elements, have insufficient or excessive lighting, or possess other imperfections or inadequacies related to the user's framing of the image.
Many aspects of the invention can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present invention. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.
Embodiments of the present disclosure relate to systems and methods that can be executed in an image capture device. More specifically, embodiments of the disclosure relate to systems and methods for framing and/or reframing of images captured by an image capture device to improve the framing characteristics and/or appearance. In the context of this disclosure, an image capture device can include a camera, video camera, a mobile device with an integrated image capture device, or other devices suitable to capturing imagery and/or video as can be appreciated. In some embodiments, an image capture device according to an embodiment of the disclosure can include a device such as a smartphone, tablet computing system, laptop computer, desktop computer, or any other computing device that has the capability to receive and/or capture imagery via image capture hardware.
Accordingly, image capture device hardware can include components such as lenses, image sensors (e.g., charge coupled devices, CMOS image sensor, etc.), processor(s), image signal processor(s), a main processor, memory, mass storage, or any other hardware or software components that can facilitate capture of imagery and/or video. In some embodiments, an image signal processor can be incorporated as a part of a main processor in an image capture device module that is in turn incorporated into a device having its own processor, memory and other components.
An image capture device according to an embodiment of the disclosure can provide a user interface via a display that is integrated into the image capture device. The display can be integrated with a mobile device, such as a smartphone and/or tablet computing device, and can include a touchscreen input device (e.g., a capacitive touchscreen, etc.) with which a user may interact with the user interface that is presented thereon. The image capture device hardware can also include one or more buttons, dials, toggles, switches, or other input devices with which the user can interact with software executed in the image capture device.
Referring now to the drawings,
The mobile device 102 may be configured to execute various applications, such as a camera application that can interact with an image capture module that includes various hardware and/or software components that facilitate capture and/or storage of images and/or video. In one embodiment, the camera application can interact with application programming interfaces (API's) and/or other software libraries and/or drivers that are provided for the purpose interacting with image capture hardware, such as the lens system and other image capture hardware. The camera application can be a special purpose application, a plug-in or executable library, one or more API's, image control algorithms, image capture device firmware, or other software that can facilitate communication with image capture hardware in communication with the mobile device 102.
The image capture device 104 includes a lens system 200 that conveys images of viewed scenes to an image sensor 202. By way of example, the image sensor 202 comprises a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) sensor that is driven by one or more sensor drivers 204. The analog image signals captured by the sensor 202 are provided to an analog-to-digital (ND) converter 206 for conversion into binary code that can be processed by a processor 208. The processor can also execute an image framing application 151 that can facilitate framing of images captured by a user as well as generating recommendations to the user regarding adjustments to image framing that can be made to produce higher quality images with the image capture device 104. In some embodiments, the image framing application 151 can take the form of API's, firmware, or other software accessible to the image capture device 104 and/or a mobile device 102 or other system in which the image capture device 104 is integrated.
Operation of the sensor driver(s) 204 is controlled through a camera controller 210 that is in bi-directional communication with the processor 208. In some embodiments, the controller 210 can control one or more motors 212 that are used to drive the lens system 200 (e.g., to adjust focus, zoom, and/or aperture settings). The controller 210 can also communicate with a flash system, user input devices (e.g., buttons, dials, toggles, etc.) or other components associated with the image capture device 104. Operation of the camera controller 210 may be adjusted through manipulation of a user interface. A user interface comprises the various components used to enter selections and commands into the image capture device 104 and therefore can include various buttons as well as a menu system that, for example, is displayed to the user in, for example, a camera application executed on a mobile device 102 and/or on a back panel associated with a standalone digital camera.
The digital image signals are processed in accordance with instructions from an image signal processor 218 that can be implemented as a standalone processor within the image capture device as well as being a part of the processor 208. Processed (e.g., compressed) images may then be stored in storage memory, such as that contained within a removable solid-state memory card (e.g., Flash memory card). The embodiment shown in
An image capture device (e.g., camera, mobile device with integrated camera, etc.) and/or processing system can be configured with automatic framing and/or reframing capabilities that are based at least upon an identification and characterization of various image elements. An image capture device 104 as described herein can identify various framing characteristics associated with an image captured by the device and automatically reframe the image and/or suggest adjustments to framing conditions that a user may take to comply with framing guidelines that can be accessible to the image capture device 104. For example, framing guidelines can specify various ranges of parameters regarding various types of image subjects (e.g., people, foreground elements, background elements, other objects, etc.). Additionally, the framing guidelines can also specify ranges of parameters that are related to various other image properties, such as, but not limited to, lighting sources, such as a device flash and/or natural or artificial light sources within the image, brightness, sharpness, tone, color intensity, contrast, gamma, etc., or other aspects of an image. As described herein, the image framing application 151 can determine whether the framing characteristics of an image captured by the image capture device comply with ranges of various parameters that are specified by at least one framing guideline that is accessible to the image framing application 151.
The analysis of imagery as well as determinations regarding whether framing characteristics of an image comply with framing guidelines can be accomplished via software executed by the processor 208, the ISP 218 as well as a processor associated with a device in communication with the image capture device 104. It should be appreciated that the specific implementation and/or embodiments disclosed herein are merely examples.
Accordingly, reference is now made to
According to one embodiment of the disclosure, the image framing application 151 executed by the image capture device 104 can analyze the image 303 to identify various framing characteristics of the image. To perform such an analysis, the image framing application 151 can identify the various elements in an image. In other words, the image framing application 151 can identify objects that are depicted in an image 303 captured by the image capture device 104. The image framing application 151 can also identify a subject of the image. For example, a subject of the image can be one or more people or any other object that is the focus of an image. The image framing application 151 can characterize the objects and/or elements within an image 303, which can be used to determine the framing characteristics of the image.
The various regions, objects, elements, etc., within an image 303 can be characterized based upon their content. Example, people depicted in an image 303 can be identified as such, background elements (e.g., sky, sun, etc.), foreground elements, and other elements can be characterized. Subsequently, the image framing application 151 can identify the framing characteristics of the image 303. In other words, the image framing application 151 can calculate a measure of how well-framed the captured image is as well as whether the framing of the image can be improved upon. The framing characteristics can then be compared with various framing guidelines, which can specify ranges of various parameters that represent best practices, or a well-framed image. Accordingly, in some embodiments, the image framing application 151 can automatically reframe the image 303 based upon the captured image data, which can result in a more aesthetically pleasing image.
Framing characteristics associated with the image 303 can include, as one example, a percentage of the image in which a subject appears. As another example, a framing characteristic can comprise a percentage of the subject that appears in the image. Framing characteristics can also include coordinates that describe a horizontal and/or vertical position of the subject within the image 303. As additional examples, framing characteristics can include: a position of lighting sources, such as a device flash and/or natural or artificial light sources relative to the subject, clarity of the subject, brightness, tone, color intensity, contrast, gamma, or other characteristics associated with the subject in the image 303.
In the depicted example, the image framing application 151 can identify the subject 305 of the image and identify its various framing characteristics. By way of illustration, the image framing application 151 can determine a fraction and/or percentage of the image that the subject 305 occupies as well as the percentage of the subject 305 appearing in the image. The image framing application 151 can estimate such a percentage by identifying the subject 305 as a human body and estimating a percentage of the image that does not appear in the image 303. The percentage of the image that the subject 305 occupies as well as coordinates describing the position of the subject 305 within the image can also be identified. The image framing application 151 can also determine a percentage of the image that the background (e.g., sky, landscape, etc.), foreground, and/or other image elements consume.
Accordingly, upon identifying the various framing characteristics of the image 303, the image framing application 151 can compare one or more of the framing characteristics against one or more framing guidelines. Framing guidelines can represent ideal or best practices as it relates to the framing and/or composition of an image. For example, a framing guideline can specify one or more percentage range that a subject of an image should consume. As another example, a framing guideline can specify one or more percentage range of a subject that should appear in an image. The framing guidelines can also specify these parameters as they relate to the other image elements that can be identified in the image 305. As a non-limiting example, the image framing application 151 can specify that a subject of an image, if it is a person or human body, should ideally comprise 15-25% or 65-75% of an image.
Continuing this illustrative example, the framing guidelines also specify that a human subject, if represented in the image 305, should appear such that the head of the subject is located within a certain range of the vertical and/or horizontal center position of the image. As yet another illustrative example, the framing guidelines can also specify that if a human body is the subject of an image that the body should not be cut off at the knees and/or legs. In other words, the framing guidelines can specify that a certain percentage range of the subject should be represented in the image 305.
As another example, the image framing application 151 can detect the lighting conditions of the image 303. For example, the intensity and/or position of light sources within the image 303 can be detected. As another example, the distance of the subject 305 from the image capture device 104, which can be derived from data regarding focusing from the lens system of the image capture device, can also be determined. Additionally, the image framing application 151 can determine an optimum distance from the image capture device 104 based at least upon the characteristics of a flash device incorporated into the image capture device 104.
To determine whether framing of an image captured by the image capture device 104 can be improved, the image framing application 151 can calculate a framing score that expresses the extent to which the framing characteristics of an image comply with the various framing guidelines. In one embodiment, such a framing score can be based at least upon how closely the identified framing characteristics comply with framing guidelines. Continuing the above example of a hypothetical framing guideline that specifies various percentage ranges of an image that a subject should consume, the framing score can include a measure of how closely the identified framing characteristics of an image comply with one or more of the percentage ranges.
Reference is now made to
In the depicted example, the image framing application 151 can crop the image 303 of
Reference is now made to
In some embodiments, the image framing application 151 can identify a distance of the subject 805 from the image capture device 104 and generate a recommendation that the user position the image capture device 104 and/or the subject 805 closer or further from one another depending on an optimum range associated with a flash device associated with the image capture device 105. The image capture device can also analyze color intensity, image quality, or other parameters associated with the subject 805 and generate similar recommendations that are related to framing of the image that can result in a higher quality result.
Referring next to
First, in box 901, the image framing application 151 can initiate capture of one or more images and/or video by the image capture device 104. In one embodiment, image capture can be initiated by the user and/or any software application executed by the image capture device 104 or any device in which the image capture device 104 is integrated. In box 903, the image framing application 151 can identify framing characteristics of the image. In box 905, the image framing application 151 can generate a framing score associated with the identified framing characteristics. In other words, the image framing application 151 can determine whether the framing characteristics comply with framing guidelines or whether the image characteristics can be adjusted to more closely comply with framing guidelines. In other words, the image framing application 151 can reframe an image when the framing characteristics do not comply with framing guidelines.
In box 907, the image framing application 151 can determine whether improvement of the framing score is possible. In other words, the image framing application 151 can determine whether the image can be reframed (e.g., a region of the image identified and/or cropped from the image) and/or adjust other image characteristics or parameters associated with the image to improve the framing score. If so, in box 909, the image framing application 151 can reframe the image such that the framing characteristics more closely comply with one or more framing guidelines.
It should be appreciated that in some embodiments, images may be adjusted and/or reframed without initiating image capture as described in box 901, and that the example illustrated in the flowchart of
Embodiments of the present disclosure can be implemented in various devices, for example, having a processor, memory as well as image capture hardware that can be coupled to a local interface. The logic described herein can be executable by one or more processors integrated with a device. In one embodiment, an application executed in a computing device, such as a mobile device, can invoke one or more API's that provide the logic described herein as well as facilitate interaction with image capture hardware. Where any component discussed herein is implemented in the form of software, any one of a number of programming languages may be employed such as, for example, processor specific assembler languages, C, C++, C#, Objective C, Java, Javascript, Perl, PHP, Visual Basic, Python, Ruby, Delphi, Flash, or other programming languages.
As such, these software components can be executable by one or more processors in various devices. In this respect, the term “executable” means a program file that is in a form that can ultimately be run by a processor. Examples of executable programs may be, for example, a compiled program that can be translated into machine code in a format that can be loaded into a random access portion of memory and run by a processor, source code that may be expressed in proper format such as object code that is capable of being loaded into a random access portion of the memory and executed by the processor, or source code that may be interpreted by another executable program to generate instructions in a random access portion of the memory to be executed by the processor, etc. An executable program may be stored in any portion or component of the memory including, for example, random access memory (RAM), read-only memory (ROM), hard drive, solid-state drive, USB flash drive, memory card, optical disc such as compact disc (CD) or digital versatile disc (DVD), floppy disk, magnetic tape, or other memory components.
Although various logic described herein may be embodied in software or code executed by general purpose hardware as discussed above, as an alternative the same may also be embodied in dedicated hardware or a combination of software/general purpose hardware and dedicated hardware. If embodied in dedicated hardware, each can be implemented as a circuit or state machine that employs any one of or a combination of a number of technologies. These technologies may include, but are not limited to, discrete logic circuits having logic gates for implementing various logic functions upon an application of one or more data signals, application specific integrated circuits having appropriate logic gates, or other components, etc. Such technologies are generally well known by those skilled in the art and, consequently, are not described in detail herein.
The flowchart of
Although the flowchart of
Also, any logic or application described herein that comprises software or code can be embodied in any non-transitory computer-readable medium for use by or in connection with an instruction execution system such as, for example, a processor in a computer device or other system. In this sense, the logic may comprise, for example, statements including instructions and declarations that can be fetched from the computer-readable medium and executed by the instruction execution system. In the context of the present disclosure, a “computer-readable medium” can be any medium that can contain, store, or maintain the logic or application described herein for use by or in connection with the instruction execution system. The computer-readable medium can comprise any one of many physical media such as, for example, magnetic, optical, or semiconductor media. More specific examples of a suitable computer-readable medium would include, but are not limited to, magnetic tapes, magnetic floppy diskettes, magnetic hard drives, memory cards, solid-state drives, USB flash drives, or optical discs. Also, the computer-readable medium may be a random access memory (RAM) including, for example, static random access memory (SRAM) and dynamic random access memory (DRAM), or magnetic random access memory (MRAM). In addition, the computer-readable medium may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or other type of memory device.
It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations set forth for a clear understanding of the principles of the disclosure. Many variations and modifications may be made to the above-described embodiment(s) without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.
Claims
1. An image capture device, comprising:
- at least one image sensor; and
- an image framing application executed in the image capture device, comprising: logic that initiates capture of an image via an image sensor associated with the image capture device; logic that identifies at least one framing characteristic associated with the image; logic that determines whether the at least one framing characteristic complies with at least one framing guideline stored in a memory accessible to the image capture device; and logic that reframes the image based at least upon the at least one framing characteristic and the at least one of framing guideline.
2. The image capture device of claim 1, wherein the logic that determines whether the at least one framing characteristic complies with at least one framing guideline further comprises:
- logic that calculates a framing score associated with the image, the framing score representing a framing quality of the image, the framing score based at least upon a deviation of the at least one framing characteristic from the at least one framing guideline.
3. The image capture device of claim 1, wherein the logic that identifies at least one framing characteristic associated with the image further comprises:
- logic identifies a subject in the image; and
- logic that calculates a percentage of the image in which the subject appears.
4. The image capture device of claim 3, wherein the subject is an object corresponding to at least one person and logic that identifies at least one frame characteristic associated with the image further comprises:
- logic that identifies a face associated with the at least one person; and
- logic that determines at least one coordinate corresponding to a position of the face in the image.
5. The image capture device of claim 4, wherein the logic that determines whether the at least one framing characteristic complies with at least one framing guideline stored in a memory accessible to the image capture device further comprises:
- logic that identifies at least one guideline coordinate specified by the at least one framing guideline, the at least one guideline coordinate identifying a reference coordinate range in the image associated with a face of a subject; and
- logic that determines whether the face is within the reference coordinate range.
6. The image capture device of claim 4, wherein the logic that reframes the image when the at least one framing characteristic does not comply with at least one of the framing guidelines further comprises logic that crops a region of the image surrounding at least a portion of the subject, the subject placed within the region to comply with the at least one framing guideline.
7. The image capture device of claim 1, wherein the logic that identifies at least one framing characteristic associated with the image further comprises:
- logic that identifies a subject in the image; and
- logic that calculates a percentage of the subject that appears in the image.
8. The image capture device of claim 7, wherein the at least one framing guideline specifies a percentage range of the subject that should appear in the image, and the logic that reframes the image when the at least one framing characteristic does not comply with at least one of the framing guidelines further comprises logic that crops a region of the image surrounding at least a portion of the subject, a size of the region selected so that the percentage of the subject that appears in the region is within the percentage range.
9. The image capture device of claim 1, wherein the at least one framing guideline further comprises at least one of: a distance of an object in the image from the image capture device, a percentage of the image comprising a background, a percentage of the image comprising a foreground, a percentage of the object visible in the image, lighting sufficiency of the image, and whether a light source appears in the image.
10. The image capture device of claim 9, wherein the logic that determines whether the at least one framing characteristic complies with at least one framing guideline further comprises logic that generates a framing score associated with the image, the framing score comprising a measure that expresses an extent to which the image complies with each of the framing guidelines.
11. A method, executed in an image capture device, comprising the steps of:
- initiating capture of an image via an image sensor associated with the image capture device;
- identifying at least one framing characteristic associated with the image;
- determining whether the at least one framing characteristic complies with at least one framing guideline stored in a memory accessible to the image capture device; and
- reframing the image when the at least one framing characteristic does not comply with at least one of the framing guidelines.
12. The method of claim 11, wherein the step of determining whether the at least one framing characteristic compiles with at least one framing guideline further comprises calculating a framing score associated with the image, the framing score representing a framing quality of the image, the framing score based at least upon a deviation of the at least one framing characteristic from the at least one framing guideline.
13. The method of claim 11, wherein the step of identifying at least one framing characteristic associated with the image further comprises the steps of:
- identifying a subject in the image; and
- calculating a percentage of the image in which the subject appears.
14. The method of claim 13, wherein the subject is an object corresponding to at least one person and step of identifying at least one frame characteristic associated with the image further comprises the steps of:
- identifying a face associated with the at least one person; and
- determining at least one coordinate corresponding to a position of the face in the image.
15. The method of claim 14, wherein the step of determining whether the at least one framing characteristic complies with at least one framing guideline stored in a memory accessible to the image capture device further comprises the steps of:
- identifying at least one guideline coordinate specified by the at least one framing guideline, the at least one guideline coordinate identifying a reference coordinate range in the image associated with a face of a subject; and
- determining whether the face is within the reference coordinate range.
16. The method of claim 14, wherein the step of reframing the image when the at least one framing characteristic does not comply with at least one of the framing guidelines further comprises the step of cropping a region of the image surrounding at least a portion of the subject, the subject placed within the region to comply with the at least one framing guideline.
17. The method of claim 11, wherein the step of identifying at least one framing characteristic associated with the image further comprises the steps of:
- identifying a subject in the image; and
- calculating a percentage of the subject that appears in the image.
18. The method of claim 17, wherein the at least one framing guideline specifies a percentage range of the subject that should appear in the image, and the step of reframing the image when the at least one framing characteristic does not comply with at least one of the framing guidelines further comprises the step of cropping a region of the image surrounding at least a portion of the subject, a size of the region selected so that the percentage of the subject that appears in the region is within the percentage range.
19. The method of claim 11, wherein the at least one framing guideline further comprises at least one of: a distance of an object in the image from the image capture device, a percentage of the image comprising a background, a percentage of the image comprising a foreground, a percentage of the object visible in the image, lighting sufficiency of the image, and whether a light source appears in the image.
20. A system, comprising:
- means for initiating capture of an image via an image sensor associated with the image capture device;
- means for identifying at least one framing characteristic associated with the image;
- means for determining whether the at least one framing characteristic complies with at least one framing guideline stored in a memory accessible to the image capture device; and
- means for reframing the image when the at least one framing characteristic does not comply with at least one of the framing guidelines.
Type: Application
Filed: Sep 14, 2011
Publication Date: Jan 24, 2013
Applicant: BROADCOM CORPORATION (Irvine, CA)
Inventors: Naushir Patuck (Cambridge), Benjamin Sewell (Truro)
Application Number: 13/232,052
International Classification: H04N 5/225 (20060101);