IMAGE PROCESSING APPARATUS AND DOCUMENT SCANNING SYSTEM HAVING THE SAME
Virtual straight lines are parallelly shifted at regular intervals from starting points, which are four vertexes of a rectangular-shaped captured image, toward the center of the captured image, the virtual straight lines being tilted at an angle of a predetermined degree with respect to coordinate axes extending along two mutually orthogonally intersecting edges of the captured image. While doing so, intersection points between the virtual straight lines and outline elements are obtained. Based on the intersection points, angular points of a contour of a document is obtained. Based on the angular points, a mesh model is generated.
Latest Panasonic Patents:
- NEGATIVE ELECTRODE SLURRY FOR LITHIUM ION SECONDARY BATTERY, METHOD FOR MANUFACTURING NEGATIVE ELECTRODE FOR LITHIUM ION SECONDARY BATTERY, METHOD FOR MANUFACTURING LITHIUM ION SECONDARY BATTERY, NEGATIVE ELECTRODE FOR LITHIUM ION SECONDARY BATTERY, AND LITHIUM ION SECONDARY BATTERY
- IMAGING APPARATUS
- NEGATIVE ELECTRODE ACTIVE MATERIAL FOR SECONDARY BATTERY, AND SECONDARY BATTERY USING SAME
- CONTROL METHOD, NON-TRANSITORY COMPUTER READABLE STORAGE MEDIUM, AND SERVER DEVICE
- COMPOSITE MEMBER
The present application claims priority under 35 U.S.C. §119 of Japanese Application No. 2011-078367, filed on Mar. 31, 2011, the disclosure of which is expressly incorporated by reference herein in its entirety.
BACKGROUND OF THE INVENTION1. Field of the Invention
The present invention is related to an image processing apparatus that processes an image scanned from a document such as a book, and a document scanning system having the image processing apparatus.
2. Description of Related Art
A book scanner (document camera) is widely used, which can capture, from above, a page of a book in a naturally opened state and scan an image of the page. Such a book scanner enables a user to sequentially scan images of pages while turning the pages, thereby making it possible to efficiently digitize books.
In capturing an image of a document using such a book scanner, it is critical to accurately detect, from the captured image, a contour of the document in order to appropriately obtain the image of the document. In relation to such a need, a technology is known in which an identification marking such as a diagonal line is provided to a surface of a platform on which the document is placed. Using the diagonal line, an area outside the document in the captured image is determined. Thereby, a document area is obtained (Japanese Patent Laid-open Publication No. 2006-211046).
The conventional technology, however, requires a platform specially provided with an identification marking. Further, in a case where there is dust attached to a surface of a desk on which a document is placed, there is a possibility that the dust causes an erroneous detection. Thus, it is necessary to carefully clean the desk surface so that no dust is attached thereto. Furthermore, erroneous detection may be caused by a design or the like on a surface of a desk. Therefore, the conventional technology is not necessarily convenient for a user.
SUMMARY OF THE INVENTIONThe present invention has been devised to address the circumstances of the above-described conventional technology. A main advantage of the present invention is to provide an image processing apparatus and a document scanning system having the same, the document processing apparatus is configured to accurately detect a contour of a document causing less troubles to the user.
An image processing apparatus of the present invention includes a captured image obtainer obtaining a captured image that is an image of a paper surface of a document captured along with a placement surface and an image processor processing the captured image. The image processor includes: an outline element extractor extracting an outline element of the document from the captured image; an intersection point detector obtaining an intersection point between a virtual straight line and the outline element obtained by the outline element extractor, while parallelly shifting the virtual straight line at a predetermined shift interval, the virtual straight line being tilted at a predetermined angle with respect to coordinate axes extending along mutually orthogonally intersecting two edges of the rectangular-shaped captured image; an outline intersection point obtainer obtaining, as an outline intersection point, an intersection point located on an outermost side among the intersection points obtained by the intersection point detector; and an angular point obtainer obtaining an angular point of a contour of the document based on the outline intersection point obtained by the outline intersection point obtainer. The contour of the document is obtained based on the angular point obtained by the angular point obtainer.
A document scanning system of the present invention includes the above-described image processing apparatus and an image inputting device having a camera. The image inputting device captures an image of a document and transmits the captured image to the image processing apparatus.
According to the present invention, by shifting the virtual straight line tilting at a predetermined angle, a row of the intersection points appear so as to be substantially perpendicularly divided into two directions when the virtual straight line reaches at an angular point of the contour of the document. The angular point of the contour of the document can be accurately detected based on a state of the appearance of the intersection points. Further, it is also possible to accurately detect the contour of the document based on the angular point. In addition, the present invention requires no special platform. The present invention is also less affected by noise such as dust, design, or the like on a placement surface. Therefore, it is possible to enhance usability for a user causing less troubles to the user.
The present invention is further described in the detailed description which follows, in reference to the noted plurality of drawings by way of non-limiting examples of exemplary embodiments of the present invention, in which like reference numerals represent similar parts throughout the several views of the drawings, and wherein:
The particulars shown herein are by way of example and for purposes of illustrative discussion of the embodiments of the present invention only and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the present invention. In this regard, no attempt is made to show structural details of the present invention in more detail than is necessary for the fundamental understanding of the present invention, the description is taken with the drawings making apparent to those skilled in the art how the forms of the present invention may be embodied in practice.
First EmbodimentHereinafter, an embodiment of the present invention is described with reference to the drawings.
The book scanner 1 includes a camera 3 capturing an image of a paper surface of the book B and a stand 4 holding the camera 3. The stand 4 is mounted on a placing surface 5 such as a desk. The book B is placed on the placing surface 5 provided directly below the camera 3. The camera 3 captures an image of a paper surface of the book B. An image of a paper surface is captured in a state where the book B is naturally opened, and the captured image of a two-page spread of the book B is thus obtained. Further, planarization is performed on an image of a distorted paper surface in order to obtain a planarized image of the paper surface of the two-page spread.
In the PC 2, operation conditions such as resolution, a frame rate, and the like of an image to be captured by the book scanner 1 are input with an operation through the inputter 27 configured with a keyboard or the like. These operation conditions are transmitted from the operation system controller 24 to the book scanner 1. In the book scanner 1, the image capturing processor 11 performs a predetermined operation according to an instruction from the operation instructor 12 based on the operation conditions transmitted from the PC 2.
The image data inputter 22 of the PC 2 stores, in a memory, image data transmitted from the book scanner 1, and outputs the image data to the image processor 23 as needed. The image data inputter 22 obtains a captured image of a document (paper surface of a book, in this example) B captured along with the placing surface 5.
The image processor 23 includes an angular point detector 31, an outline obtainer 32, a mesh model generator 33, and an image transformer 34. The angular point detector 31 includes a document edge area obtainer 41, an outline element extractor 42, an intersection point detector 43, an outline intersection point obtainer 44, and an angular point obtainer 45.
First, a user sets a book (document) below the camera 3 of the book scanner 1 (ST101). Then, after the book scanner 1 and a predetermined application in the PC 2 are started, image input, that is, image capturing, is started by the book scanner 1, and the captured image data is transmitted to the PC 2 (ST102). The image data inputter 22 of the PC 2 obtains the captured image of a paper surface of the book B captured along with the placing surface 5.
In the PC 2, as shown in
Thereafter, as shown in
The planarized image obtained above is displayed on the display 26 (ST 107). Thereby, the user can confirm whether or not the image has been properly cut out. The planarized image data is stored in the data storage 28 (ST108).
[Angular Point Detection]Next, the angular point detection (ST103) shown in
As shown in
First, the straight line element detection (ST201) shown in
Herein, straight line elements (linearly arranged pixels) within the captured image are detected. This straight line element detection may be performed by the Hough transformation. Thereafter, the detected straight line elements are extended to edges of the captured image to obtain extended straight line segments. In this embodiment, only the straight line elements having a length longer than a predetermined length are selected to obtain the extended straight line segments.
(Document Edge Area Obtainment)Next, the document edge area obtainment (ST202) shown in
The processing here obtains, from the captured image, edges of the document, specifically, document edge areas in which the contour of the book, edges of a page, and a binding of pages are estimated to exist. Although document edge areas in a vertical direction are first detected and document edge areas in a horizontal direction are then detected in the example shown in
Detection of the document edge areas extending in the vertical direction is first explained. First, of the extended straight line segments obtained in the straight line element detection (ST201) shown in
Then, as shown in
Thereafter, as shown in
The approximate straight lines obtained here become centerlines of the document edge areas. Using the centerlines as references, document edge areas SL, SC, and SR extending in the vertical direction are set (ST304). The document edge areas SL, SC, and SR each have two sides (upper side and lower side), which are an upper edge and a lower edge of the captured image. The document edge areas SL, SC, and SR form parallelograms each having predetermined widths SLX, SCX, and SRX, respectively, in the X direction around the approximate straight lines as centerlines. A left document edge area SL and a right document edge area SR are areas in which left and right edges of a cover page or page of a book exist. A central document edge area SC is an area in which a binding of pages exists.
The widths SLX, SCX, and SRX in the X direction of the document edge areas SL, SC, and SR are set based on a distance LX in the X direction between the left peak points LC1 to LC5 and the right peak points RC1 to RC5. In this embodiment, the widths SLX, SCX, and SRX have values obtained by multiplying the distance LX by an experimentally determined predetermined ratio. For example, the widths SLX and SRX in the X direction of the document edge areas SL and SR are 1/8 of the distance LX, and the width SCX in the X direction of the central document area SC is 1/10 of the distance LX.
In addition, the widths SLX and SRX in the X direction of the left and right document edge areas SL and SR vary depending on the number of pages of a book. Thus, they may be calculated based on a measurement result. Furthermore, by comparing the value calculated based on the measurement result with the value calculated with the ratio with respect to the distance LX, a greater value between the two may be selected.
In this case, the widths SLX and SRX in the X direction of the left and right document edge areas SL and SR are set to be predetermined multiples (two times, for example) of the width in the X direction of an area having the counted number more than or equal to a predetermined threshold value at positions before or after the peak points LC1 to LC5 and RC1 to RC5, for example. In this embodiment, the threshold value may be a value obtained by multiplying the counted number (greatest value) at the peak points LC1 to LC5 and RC1 to RC5 by a predetermined ratio (50%, for example).
Furthermore, in a case of a small book, an image of the book becomes small in the captured image. Accordingly, there may be a case where there are few significant peak points, and it thus becomes difficult to properly extract the document edge areas SL, SC, and SR. In such a case, a calculation is performed again with modified Y direction positions y1 to y5. For example, the Y coordinates of the Y direction positions y1 to y5 each are set as 300, 800, 1,000, 1,200, and 1,700. In this example, a central portion of the Y direction has smaller spacing.
Further, in this embodiment, five Y direction positions (positions of the detection lines) are set in order to detect the peak points. However, the number of detection positions is not limited to this, and may be two, for example. Further, the detection position may be one of the centers in the Y direction (Y=1,000, for example). Furthermore, in a preliminary detection for showing a user a state of processing, a smaller number of detection positions (two, for example) may be set. Then, in a real detection for generating an image to be eventually stored, a larger number of detection positions may be set so that accurate detection is ensured.
Next, detection of the document edge area extending in the horizontal direction is described. First, of the extended straight line segments obtained by the straight line element detection (ST201) shown in
Thereafter, as shown in
Thereafter, as shown in
The approximate straight lines obtained here become centerlines of the document edge areas. Using these centerlines as references, document edge areas SU and SD extending in the horizontal direction are set (ST308). The document edge areas SU and SD each have two sides (left side and right side), which are a left edge and a right edge of the captured image. The document edge areas SU and SD form parallelograms each having predetermined widths SUY and SDY, respectively, around the approximate straight lines as centerlines. The document edge areas SU and SD are areas in which upper and lower edges of a cover page or page of a book exist.
The widths SUY and SDY in the Y direction of the document edge areas SU and SD are set based on a distance LY in the Y direction between the upper peak points UC1 to UC5 and the lower peak points DC1 to DC5. In this example, the widths SUY and SDY are set to values obtained by multiplying the distance LY by an experimentally determined predetermined ratio, for example, 1/7 of the distance LY.
In addition, the widths SUY and SDY in the Y direction of the document edge areas SU and SD are affected by an amount of distortion of an opened paper and also vary depending on a book size or the number of pages. Thus, they may be calculated based on a measurement result. Furthermore, by comparing the value calculated based on the measurement result with the value calculated with the ratio with respect to the distance LY, a greater number between the two may be selected.
In this case, the widths SUY and SDY in the Y direction of the document edge areas SU and SD may be predetermined multiples (three times, for example) of the width in the Y direction of the area having the counted number more than or equal to a predetermined threshold value at a position before or after the peak points UC1 to UC5 and DC1 to DC5, for example. In this embodiment, the threshold value may be a value obtained by multiplying the counted number (greatest value) at the peak points UC1 to UC5 and DC1 to DC5 by a predetermined ratio (66%, for example).
Furthermore, in a case of a small book, an image of the book becomes small in the captured image. Accordingly, there may be a case where there are few significant peak points, and thus it becomes difficult to properly extract the document edge areas SU and SD. In such a case, a calculation is performed again with modified X direction positions x1 to x5. For example, the X coordinates of the X direction positions x1 to x5 are set as 500, 950, 1,300, 1,750, and 2,000. In this example, a central portion of the X direction has smaller spacing.
Further, in this embodiment, five X direction positions (positions of the detection lines) are set in order to detect the peak points. However, the number of detection positions is not limited to this, and may be two, for example. Furthermore, in a preliminary detection for showing a user a state of processing, a smaller number of detection points (two, for example) may be set. Then, in a real detection for generating an image to be eventually stored, a larger number of detection positions may be provided so that accurate detection is ensured.
Descriptions have been provided to the case of a two-page spread, using an example where an image of a book is captured while the book being opened to left and right directions. There is also a case where an image of a book is captured while the book being opened to up and down directions. In such a case, the vertical direction and the horizontal direction are reversed from the above-described example. Accordingly, two document edge areas extending in the vertical direction shown in
Further, an optimum value for the width of the document edge area varies depending on a book size or the number of pages. The value shown in this embodiment is an empirical value assuming general users. When types of books (documents) are not common, there is a possibility that proper processing is not performed. In such a case, the optimum value may be changed by a user as needed, or may be changed according to statistics based on an assessment function (that minimizes the variance at the angular point positions, for example) using values obtained by processing a plurality of captured images.
Furthermore, in the present embodiment, the document edge areas are set based on the peak points that have the greatest counted numbers of the extended straight line segments on the detection lines. Alternatively, a threshold value may be set for the counted number of the extended straight line segments, and the document edge area may be set with the number of areas exceeding the threshold value. When the number of the document edge areas does not have an appropriate value (three, in the example shown in
In addition, when there is a table on a paper surface of a book, lines of the table are detected as straight line elements. Thus, a great number of peak points appear in the middle area. In this case, first, the left and right document edge areas are set based on the peak points appearing on the both edges. Then, the width (pixel numbers) of the counting sections for counting the number of the extended straight line segments on the detection lines are modified. With the modified number, counting of the extended straight line segments on the detection lines is performed. At this time, for example, with the width of the counting sections being reduced (five pixels, for example), the counted number for the area having only one extended straight line segment may be excluded from the processing. In addition, with the width of the counting sections being increased (30 pixels, for example), the predetermined number (two in the example shown in
Next, the document edge area adequacy determination (ST203) shown in
Herein, adequacy of the document edge areas obtained by the document edge area obtainment (ST202) shown in
More specifically, when a peak point CC, which is the center indicating a binding, does not exist, it is determined as not adequate. Accordingly, the process returns to the straight line element detection (ST201) shown in
Further, in comparison between a distance between a left peak point LC and the central peak point CC and a distance between the central peak point CC and a right peak point RC, when a difference between these two is greater than a predetermined value, it is determined as not adequate. Accordingly, the process returns to the document edge area obtainment (ST202) shown in
Next, the edge detection (ST204) shown in
Herein, edge detection is performed with respect to the captured image in order to extract an edge element. This edge detection may be performed using the Canny method. Herein, when a threshold value is small, noise is increased. By contrast, when a threshold value is great, fragmented edge elements are increased. Thus, the threshold value is modified based on a detection result to optimize the performance of the edge detection. In this case, a total length and the number of continuous edge elements having a predetermined length or more, and a total length of continuous edge elements having a predetermined width or less are used as assessment references to learn a threshold value having the greatest assessment reference value.
(Outline Element Extraction)Next, the outline element extraction (ST205) shown in
Herein, outline elements (pixels configuring the outline of the document) within the edge image is extracted, the edge image having been obtained by the edge detection (ST204) shown in
Next, the intersection point detection (ST206) shown in
Herein, virtual straight lines L1 to L4 are tilted at a predetermined angle with respect to the coordinate axes (X axis and Y axis) extending along mutually orthogonally intersecting two sides of the rectangular-shaped captured image. The virtual straight lines L1 to L4 are parallelly shifted from starting points, which are four vertexes of the captured image, toward the center of the captured image at predetermined shift intervals gradually. Thereby, intersection points between the outline elements (pixels configuring the contour of the book) and the virtual straight lines L1 to L4 are collected, the outline elements having been extracted by the outline element extraction (ST205) shown in
The virtual straight lines L1 to L4 each form an angle of ±45 degrees with respect to both the X axis and the Y axis. More specifically, the first virtual straight line L1 starting from an upper right vertex (2560,1920) is tilted at an angle of −45 degrees with respect to the X axis. The second virtual straight line L2 starting from a lower right vertex (2560,0) is tilted at an angle of +45 degrees with respect to the X axis. The third virtual straight line L3 starting from an upper left vertex (0,1920) is tilted at an angle of +45 degrees with respect to the X axis. The fourth virtual straight line L4 starting from a lower left vertex (0,0) is tilted at an angle of −45 degrees with respect to the X axis
The example here shows a case where there are 2,560 pixels in the X direction and 1,920 pixels in the Y direction. In this case, the virtual straight lines L1 to L4 are each expressed by the following formulas, respectively.
L1: y=−x+4,480−Step×(N−1)
L2: y=+x−2,560−Step×(N−1)
L3: y=+x+1,920−Step×(N−1)
L4: y=−x+Step×(N−1)
Herein, Step is a shift interval of the virtual straight lines, and N is a line number. The line number is 1 at the starting point and increases by one as the virtual straight line is parallelly shifted at the shift interval Step.
Especially, in this embodiment, the shift intervals Step of the virtual straight lines L1 to L4 are changed before and after detection of an intersection point. Specifically, from the starting point until the detection of the intersection points, the virtual straight lines L1 to L4 are greatly shifted at a first shift interval Step 1. After the detection of the intersection point, the virtual straight lines are minutely shifted at a second shift interval Step 2 from the position immediately before the detected intersection point. This shifting at the second shift interval Step 2 is performed for the predetermined number of times. Thereafter, the virtual straight lines are shifted at a third shift interval Step 3 having a length between the first shift interval Step 1 and the second shift interval Step 2.
The shift interval Step of the virtual straight line is changed in three steps as described above, and the relationship among the first shift interval Step 1, the second shift interval Step 2, and the third shift interval Step 3 is expressed by the following formula.
Step 1>Step 3>Step 2
More specifically, the first shift interval Step 1 is equal to 50, and the virtual straight line in this case parallelly shifts in the X direction by 50 pixels. The second shift interval Step 2 is equal to 4, and the virtual straight line in this case parallelly shifts in the X direction by 4 pixels. The third shift interval Step 3 is equal to 10, and the virtual straight line in this case parallelly shifts in the X direction by 10 pixels. Shifting of the virtual straight line at the second interval Step 2 is equivalent to shifting by 50 virtual straight lines.
As described above, while parallelly shifting the tilted virtual straight line at predetermined shift intervals gradually, the intersection points between the outline elements and the virtual straight line are collected. When the virtual straight line reaches the angular point of the contour of the document, a row of intersection points appears so as to be substantially orthogonally divided into two directions. Based on a state of the appearance of the intersection points, the angular point of the contour of the document is accurately detected. In particular, by minutely shifting the virtual straight lines L1 to L4 at the second shift interval Step 2 after the intersection points are detected, it is possible to improve the accuracy.
(Outline Intersection Point Obtainment)Next, the outline intersection point obtainment (ST207) shown in
Herein, of the intersection points obtained by the intersection point detection (ST206) shown in
First, as shown in
Next, from the intersection points within the document edge areas SL, SR, SU, and SD, intersection points configuring the outline of the book are extracted. Herein, the virtual straight line in the Y direction is parallelly shifted from one end (position where the X coordinate has the greatest value of 2,560) of the captured image toward the other end (position where the X coordinate is 0). At this time, one or two intersection points that overlap with the virtual straight line are extracted. In a case where a plurality of intersection points is detected at one position on the virtual straight line, two points each of which has the greatest or the smallest Y coordinate, respectively, are extracted as the outline intersection points. Thereby, as shown in
Thereafter, outline intersection points that are determined as noise are removed from the extracted outline intersection points. Herein, as shown in
In addition, there is a case where the shift amount ratio Sp at a corner is greater than that at a side because of characteristics that polarity (+,−) of the shift amount ratio Sp changes. Thus, a set value (three, for example) different from the above-described calculated average may be used as a threshold value.
In the example shown in
Next, the angular point obtainment (ST208) shown in
Herein, based the outline intersection points having been obtained by the outline intersection point obtainment (ST207) shown in
First, as shown in
Thereafter, temporary angular areas where the angular points are expected to exist are set. In a case where an angular point Ca0 is calculated, an angular area SLSD where the document edge area SL and the document edge area SD are overlapped is set. Alternatively, an area that is the enlarged or reduced angular area SLSD may be set. Herein, first, based on the polarity of the shift amount ratio Sp, the outline intersection points p(1) and p(n) are selected. Then, a smallest value and a greatest value for each of the X coordinate and the Y coordinate are obtained from a total of ten outline intersection points, five intersection points being selected from each of the vertical edge and the horizontal edge including the above-described two intersection points. Thereafter, a rectangular-shaped base area defined by the smallest value and the greatest value for each of the X coordinate and the Y coordinate is set. Thereafter, by enlarging the base area with a predetermined magnification (1.5 times, for example) to the side where the angular points exist, a rectangular-shaped angular area is set. In the example shown in
Thereafter, as shown in
Thereafter, as shown in
Thereafter, an intersection point between the virtual straight line and the outline element is obtained within the angular area, the virtual straight line being tilted at a predetermined angle (45 degrees, in this example) with respect to both the X axis and the Y axis. At this time, as shown in
In addition, in a case where the lower left angular point shown in
Herein, as shown as the example in
On the other hand, as shown as the example in
More specifically, as shown in
Cv=(V3×k3+V2×k2+V1×k1)/100
Herein, k1 to k3 are weight coefficients. The weight coefficient becomes greater as it is closer to an angular point. For example, k1=10, k2=6, and k3=3.
Furthermore, for convenience of description,
Furthermore, as shown in
(45°/kk)<θ<(135°×kk)
Herein, kk is an intersection point spacing coefficient. For example, when the spacing is four pixels, kk=1. When the spacing is 10 pixels, kk=0.9.
(Angular Point Adequacy Determination)Next, the angular point adequacy determination (ST209) shown in
Herein, positions of the four angular points Ca0, Cb0, Cc0, and Cd0 obtained by the angular point obtainment (ST208) shown in
First, as shown in
When the central point Cu on the upper side or the central point Cd on the lower side is not located within the central document edge area SC, it is determined that the obtained four angular points Ca0, Cb0, Cc0, and Cd0 are not adequate. In such a case, each type of processing from the straight line element detection (ST201) to the outline intersection point extraction (ST207) shown in
Herein, in a case where the central point Cu on the upper side is displaced to the left side to the document edge area SC, for example, there is a possibility that the upper right angular point Cc0 has been erroneously detected due to a reason such as lack of the outline information on the upper right corner. Thus, detection of the upper right angular point Cc0 is performed again by the angular point obtainment (ST208) shown in
Next, an explanation is provided in a case of a cover page of a book or a sheet document as shown in
Specifically, when a ratio of lengths between mutually opposing two sides of a quadrangle whose vertexes are the four angular points Ca0, Cb0, Cc0, and Cd0 is smaller than a predetermined threshold value, it is determined that the four angular points Ca0, Cb0, Cc0, and Cd0 are adequate. More specifically, when both a ratio DS1/DS2 between the length of the right side DS1 and the length of the left side DS2 and a ratio DS3/DS4 between the length of the upper side DS3 and the length of the lower side DS4 are within a range of 0.7 to 1.3, it is determined as adequate. When the ratio DS1/DS2 or the ratio DS3/DS4 is less than or equal to 0.7 or more than or equal to 1.3, it is determined as not adequate. In this case, each type of processing from the straight line element detection (ST201) to the outline intersection point extraction (ST207) shown in
Next, the outline obtainment (ST104) shown in
Herein, outline points (outline pixels) configuring the contour of the document, as shown in
As shown in
Further, a histogram of pixel values of the pixels within the outline area SS may be obtained. Based on a result of an analysis on the histogram, a threshold value may be modified. With the modified threshold value, outline elements may be extracted again by performing the edge detection (ST204) and the outline element extraction (ST205) shown in
Next, of the outline elements existing in the outline area SS having been extracted by the outline element extraction (ST402) shown in
Next, as shown in
This processing to search for the outline points configuring the lower edge of the document outline is started at the outline point closest to the lower left angular point Ca0 and is repeated until reaching the position of the X coordinate of the lower right angular point Cb0. In this processing, an outline point that is separated from an adjacent outline point by a predetermined distance or more, that is, an outline point located outside the detection area is removed as noise.
Herein, in a case where no outline point is found in a vicinity of the focal outline point, the next outline point is searched for at a position displaced from the focal outline point by one pixel in the X direction, setting a temporary outline point, which is a pixel located at the Y direction position same as that of the focal outline point.
After the above-described processing to search for the outline points configuring the lower edge of the document outline has been ended, next, processing to search for outline points configuring the right edge of the document outline is performed. In this processing with respect to the right edge, the processing that sequentially searches for outline points in a vicinity is repeatedly performed from the last outline point found in the processing with respect to the lower edge. At this time, at a position shifted from the focal outline point by one pixel in the Y direction, an outline point located within the detection area is searched centering around the Y direction position same as that of a focal pixel, the detection area being an area extended in both sides of the X direction by the predetermined number of pixels (five pixels, for example).
Herein, in a case where no outline point is found in the vicinity of the focal outline point, the next outline point is searched for at a position displaced from the focal outline point by one pixel in the Y direction, using a pixel in the X direction position same as that of the focal point as a temporary outline point.
This processing to search for the outline points configuring the right edge of the document outline is repeated until reaching the position of the Y coordinate of the upper right angular point Cc0. Then, processing to search for outline points configuring the upper and right edges of the document outline is sequentially performed. This processing with respect to the upper and right edges may be performed with a similar procedure. Thereby, outline points configuring a periphery of the document outline are searched, and outline points that do not match the periphery is removed as noise.
In the above-described noise point removal (ST404), there is a gap at a portion processed as the temporary outline point. Thus, next, interpolation processing is performed to fill the gap due to the temporary outline point (ST405). Herein, as shown in
Further, although
Furthermore, in the present embodiment, an outline is generated to have a line width of one pixel. Performing processing such as smoothing, thickening of lines, and thinning of lines using this data may improve a performance of mesh model generation.
[Mesh Model Generation]Next, the mesh model generation (ST105) shown in
First, as shown in
Thereafter, distortion elements are extracted, the distortion elements indicating a distorted outline of a page of a book (ST501). The outlines of the upper and lower edges of the page are displayed in a distorted state where the outlines externally project. The distorted outline elements (pixels configuring the outline) of these upper and lower edges of the page are extracted form the edge image obtained by the edge detection (ST204) shown in
Thereafter, a computation is performed to convert the distortion elements into height elements (ST504). The outlines of the upper and lower edges of the page are shown in a distorted state in the captured image because the paper surface is three-dimensionally distorted so as to project upward. Thus, the height elements, which indicate three-dimensionally distorted state of the upper and lower edges of the page, can be obtained based on the distortion elements indicating the outlines of the upper and lower edges of the page. More specifically, the heights (coordinates of a Z axis) of the upper and lower edges of the page with respect to the placing surface 5 are obtained at regular intervals. Using the height elements, actual lengths of the upper and lower edges of the page can be estimated.
Thereafter, as shown in
After the mesh model is generated as described above, image transformation is performed by the image transformer 34 to planarize the image based on the mesh model. At this time, by cutting out and planarizing only the image of the document area, an undistorted image of only the document area can be obtained. In addition, processing such as skew correction correcting a tilt of an image and the like is performed as needed.
Further, in the present embodiment, as shown in
Furthermore, in the present embodiment, as shown in
In addition, in the present embodiment, an example is shown in which an image planarization is performed based on a generated mesh model. In a case of a cover page of a book or a sheet document, however, the image planarization is not required. In such a case, omitting the image planarization, processing to cut out only an image of a document area based on a contour of the document is performed, along with a skew correction as needed.
It is noted that the foregoing examples have been provided merely for the purpose of explanation and are in no way to be construed as limiting of the present invention. While the present invention has been described with reference to exemplary embodiments, it is understood that the words which have been used herein are words of description and illustration, rather than words of limitation. Changes may be made, within the purview of the appended claims, as presently stated and as amended, without departing from the scope and spirit of the present invention in its aspects. Although the present invention has been described herein with reference to particular structures, materials and embodiments, the present invention is not intended to be limited to the particulars disclosed herein; rather, the present invention extends to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims.
The present invention is not limited to the above described embodiments, and various variations and modifications may be possible without departing from the scope of the present invention.
Claims
1. An image processing apparatus comprising:
- a captured image obtainer obtaining a captured image of a paper surface of a document;
- an outline element extractor extracting an outline element of the document from the captured image;
- an intersection point detector obtaining an intersection point between a virtual straight line and the outline element obtained by the outline element extractor, while parallelly shifting the virtual straight line at a predetermined shift interval, the virtual straight line being tilted at a predetermined angle with respect to coordinate axes extending along mutually orthogonally intersecting two edges of the rectangular-shaped captured image;
- an outline intersection point obtainer obtaining, as an outline intersection point, an intersection point located on an outermost side among the intersection points obtained by the intersection point detector; and
- an angular point obtainer obtaining an angular point of a contour of the document based on the outline intersection point obtained by the outline intersection point obtainer, wherein
- the contour of the document is obtained based on the angular point.
2. The image processing apparatus according to claim 1, wherein a mesh model is generated based on the obtained contour of the document, and the captured image is planarized based on the mash model.
3. The image processing apparatus according to claim 1, wherein the intersection point detector greatly shifts the virtual straight line at a first shift interval from a starting point until detection of an intersection point, and minutely shifts the virtual straight line at a second shift interval from a position immediately before the detected intersection point after the detection of the intersection point.
4. The image processing apparatus according to claim 1, further comprising:
- a document edge area obtainer obtaining, from the captured image, a document edge area in which an edge of the document is estimated to exist, wherein
- of the intersection points obtained by the intersection point detector, an intersection point located outside the document edge area is removed as noise, the document edge area having been obtained by the document edge area obtainer.
5. The image processing apparatus according to claim 4, wherein the document edge area obtainer counts the number of a straight line element detected from the captured image, and obtains the document edge area based on the counted number of the straight line element.
6. A document scanning system, comprising:
- the image processing apparatus according to claim 1; and
- an image inputting device having a camera, wherein
- the image inputting device captures an image of a document and transmits the captured image to the image processing apparatus.
7. An image processing method, comprising:
- obtaining a captured image of a paper surface of a document;
- extracting an outline element of the document from the captured image;
- obtaining an intersection point between a virtual straight line and the outline element, while parallelly shifting the virtual straight line at a predetermined shift interval, the virtual straight line being tilted at a predetermined angle with respect to coordinate axes extending along mutually orthogonally intersecting two edges of the rectangular-shape captured image;
- obtaining, as an outline intersection point, an intersection point located on an outermost side among the obtained intersection points;
- obtaining an angular point of a contour of the document based on the obtained outline intersection point; and
- obtaining the contour of the document based on the angular point.
Type: Application
Filed: Mar 12, 2012
Publication Date: Oct 4, 2012
Applicant: PANASONIC CORPORATION (Osaka)
Inventors: Toshiyuki OGAWA (Fukuoka), Naomi SHIRAI (Fukuoka), Taichi YAMADA (Fukuoka), Minoru ARIMURA (Fukuoka)
Application Number: 13/417,668
International Classification: H04N 1/04 (20060101);