System and method for reducing visibility of registration errors in an image to be printed using a digital color printer by convolution with a laplacian kernel
A system and method for trapping in electrophotographic color printing and related technologies for printing or display in which the final image is an overlay of multiple components subject to alignment errors. Trapping is based on the cyan (C), magenta (M), and black (K) planes. There are four steps as follows: detect object edges on each of the four color planes; detect coincident and opposing edge transitions on each pair of planes (CM, CK, and KM); determine which plane to trap, i.e., to extend object across edge; and generate trap on that plane using a simple trap generation rule and a single trap generation rule.
Latest Kyocera Mita Corporation Patents:
- Display control apparatus, image forming apparatus, and computer-readable recording medium storing display control program
- Developing device, image forming unit and image forming apparatus provided with the same
- Operating device and image forming apparatus with display format receiver for receiving instructions from a user for selecting a display format
- Image display device and computer-readable storage medium storing a display control program
- Image forming apparatus
This application claims priority from U.S. Provisional Application Ser. No. 60/795,901, filed on Apr. 28, 2006, which is incorporated herein by reference in its entirety.
FIELDThe present invention applies to color printing by electrophotographic technology in which the final print consists of an overlay of multiple impressions. For example, using impressions in the colors cyan, magenta, yellow and black respectively, a visual equivalent of a full color view can be printed. In principle, it also applies to any electronic digital method for printing or display in which the final print or image consists of more than one print plane component or image plane component where the components overlaid are not perfectly in register and the visibility of the errors can be reduced by a technique known as trapping.
BACKGROUNDIn the terminology of traditional printing the term trapping (more particularly wet trapping and dry trapping) is believed to have been applied first to the press technique of overprinting on a previously printed page. Such a technique has been widely used to produce color prints by printing different color impressions in succession on the same sheet. Print technologies that layer multiple colorants in this way are sensitive to mechanical positioning errors. The different color layers must be almost perfectly aligned, or registered, over the whole sheet for good results. A relative lateral misalignment even as small as a fraction of a pixel in a digital system, or its equivalent in traditional printing, will usually be visible as undesired gaps or overlaps at the edges of printed shapes where colors change and involve different layers. Gaps that reveal a white or light-colored substrate are the most visible and objectionable. Overlaps are also visible, but not nearly as objectionable. Because sufficiently precise alignment is seldom practical, various pre-press techniques have been developed to close the gaps by creating at the pre-press stage small overlaps between abutting colors to mask registration problems in the printing process itself. The essentials of the pre-press technique are known as spreading and choking and collectively called trapping. In view of the potential ambiguity, the subject of this invention is trapping in the pre-press context in order to reduce the visibility of misregistration artifacts.
The best visual forms for changes to the image to be printed have been learned over the years, mostly in the traditional printing industry, and codified into a set of rules. In traditional printing, implementation of the rules required knowledge and craftsmanship, but because they are systematic and thorough they have proved suitable for automatic implementation in contemporary digital printing systems. However, they are not simple, they do not completely address all practical cases and their interaction with other digital image path processes presents particular difficulties further compounded by the challenges of inexpensive and efficient implementation.
The invention describes a system for trapping in electrophotographic color printing and related technologies for printing or display in which the final image is an overlay of multiple components subject to alignment errors. Trapping is a technique, well-known in principle, for reducing the visibility of alignment errors, also known as registration errors, by selectively adjusting the boundaries of image objects to overlap slightly and thereby conceal the errors. In the present invention the new methods apply to digital systems, especially to electrophotographic color printers in desktop publishing systems. The new methods offer improvements in simplicity of implementation, performance, cost, customization, compatibility with color printer controllers, automatic operation and visually acceptable results.
The present invention provides effective methods for implementation in digital systems by means of software and hardware described below. In order to understand the present invention, and the improvements it offers relative to previous methods, we begin with the underlying rules, which are addressed in the present invention and which will be recognized by most practitioners. The following trapping rules determine whether to create a trap at a color boundary and, if so, what the characteristics of the trap should be.
When and How to Trap
Whenever two colored objects abut each other or an object is overlaid on top of another, an edge is formed where the two meet. This edge is where trapping may occur. Given that the color of an object is comprised of one or more primary colorants from a respective impression or impressions, each constituent colorant is considered individually as a candidate for trapping. Thus the neutral density of each of the colorants and not the specific color of the object is used to determine whether to create a trap for that colorant. If the absolute difference in neutral density between objects, for the same colorant, exceeds a certain threshold (usually about 5%) then the colorant at that edge becomes a candidate for trapping. Another test, known as a relative mode threshold, is also sometimes applied. In relative mode, if the ratio of the difference in neutral densities to that of the colorant with lower value exceeds a threshold (say 50%), then the colorant at that edge becomes a candidate for trapping. However, because this mode takes effect at low color density levels, it rarely provides perceptible quality improvement and it may be omitted where performance and cost take priority.
The shapes of the objects as well as properties of the colorants of the objects that produce the color edge are factored into the decision to trap. As discussed below, photographic images and color gradients (also called blends, smooth shading or vignettes) have their own rules for trapping. Generally the trap is produced by adjusting the boundary of the lighter colorant into the darker colorant. Thus if there is a lighter colored object on top of a darker background, then the object is ‘spread’ into the background. Conversely, in the case of a darker object against a lighter background, the background ‘chokes’ into the object. The trap property of spreading or choking is called the trap direction. In the case of two colored objects that are very close to each but do not meet it may be necessary to establish by experiment with a particular system whether they should be subjected to trapping. Two objects separated by a pixel or two will produce a more-or-less visible white line along the edge and it is a circumstantial matter whether the line will be perceived as intentional or as an artifact. Establishing the width and the exact color of the trap are also critical for producing traps with low visible artifacts. The thinner the trap the less visible it will be; its width should therefore be no more than the maximum misregistration to be addressed, which is a characteristic of the printing device. Ordinarily the color of the trap is the same as that of the lighter colored object, but as discussed below its intensity may be reduced to minimize the visibility of the trap.
When Not to Trap
When color objects are separated by significant white space or if one of the colorants has a low neutral density, then trapping is not necessary. Also, since in many CMYK (for cyan, magenta, yellow, black) printing processes yellow has a low neutral density, it need not then be taken into consideration for trapping decisions. In another case, when the two adjacent colors share a significant amount of the same primary color component, it is not necessary to trap. In all other cases, the color edge needs to be trapped.
Colorant Type
The type of the colorant used is important for trapping decisions. For example, translucent colorants use normal trapping rules; opaque colorants may never be spread into other colorants when trapping; transparent colorants may require they not be subject to trapping, etc. For the typical color electrophotographic printer, the color process is CMYK; the cyan and magenta toners are considered to be translucent; yellow is not taken into account in deciding where to trap and black is considered to be opaque.
Images
Trapping within photographic images can produce unwanted artifacts, so photographic images should normally not be trapped. However sometimes it may become necessary to do so when the photographic image contains saturated colors such as scanned logos and images generated from non-photographic sources. The manner in which color edges adjacent to images are trapped also affects visual quality. Using normal trapping rules can cause abrupt transitions in the trap direction. Generally the choices are to trap by extending the adjacent color into the image, or the image colors into the adjacent color, or to center a trap along the edge of the image and the color being trapped, depending on image and print system characteristics.
Color Gradients
Also known as smooth shading, blends or vignettes, gradients cause particular problems. First, the trapping is not appropriate inside a color gradient since that has the potential to highlight color changes within the gradient which is counter to the intent of a smooth transition from one color to another. The present invention avoids such missteps through application of the edge detection threshold which removes edges from consideration for trapping when the color changes gradually as is the case inside color gradients.
Black
Color edges adjacent to black areas are always trapped by spreading the colorant into the black region and never by spreading the black object regardless of the intensity of the black. In addition black rules can also apply to some colorants that have a certain minimum neutral density.
Pure Black, Process Black and Rich or Super Black
A pure black object is printed with only black colorant. The dark color produced by overprinting only the other colorants at maximum density is usually known as process black. In offset printing, rich or super black consisting of an overprint of pure black and process black with all colorants at full density is quite common, but it is not used in electrophotographic printing with CMYK colorants because the printing process imposes a much lower limit on the sum total of colorant densities. Thus, in electrographic printing different concepts apply in that objects specified in a page description language (PDL) as process black or super black may, as a result of digital color processing, be printed using all four toners but with the K colorant much lower in value than full density because of the total colorant limit. Therefore in the present invention, both process black and super black will be encompassed by the term super black and the one trapping rule is applied to both.
If there is misalignment of color planes, super black objects will have a cyan or magenta halo around them, especially if the background is light colored or white, or vice-versa (light colored object on super black background).
Yellow Trapping Rule
If the value of the yellow colorant on the side of the edge that generates the trap is high, then the trap will be visible as a light colored halo around the edge. To avoid this effect, whenever a trap is calculated, the yellow colorant is also copied along with the other colorants. This prevents the halo from showing up. An example is a black object on a green background comprised of 100% cyan and 100% yellow—in this case the cyan background will choke into the black object. If there is a misalignment, then the area of the cyan trap that is uncovered by the black object will show up as a blue halo to the naked eye. This is avoided if the yellow in the green color is also choked into the black object. In this case, the color of the trap is green and not cyan.
Simple and Single Trap Generation Rules
Under the basic trap generation rule, described in detail below and known in the present invention as simple trap generation, any two colorants other than yellow changing across the color edge in opposite directions are trapped, which can result in traps from one or two of the three colorants. Thus it can generate more than one trap per edge, either on the same side of the edge or on both sides. A second rule, described in detail below and known in the present invention as single trap generation, is used when application of the simple rule indicates that an edge will generate two traps on both sides of a color edge. In this case, the lighter of the two traps is discarded and the darker trap is used to generate the final trap.
Spot Colors
Prepress software for conventional printing generally provides support for trapping to a number of spot colorants, but spot color handling is seldom applicable to digital printers using the CMYK process and it is not addressed in the present invention.
Corners, Trap Ends, Trap Intersections, Thin Sections and Small Elements
Under some conditions and for relatively large misregistrations and traps, image elements with sharp corners, thin sections and small features, also trap ends and intersections, all may require special treatment to reduce objectionable visual artifacts. In the present invention and for traps up to two pixels wide no such artifacts are visible. Therefore the merits of high performance and low cost are preserved by not providing special treatment in these cases.
Implementation of the Trapping Rules
In their application to color printing, the trapping rules require selective modification of the color separations. Contemporary color printing workflows are generally digital and they vary in detail, but the key well-known and common steps useful for understanding the present invention are document composition and raster image processing. Trapping can be accomplished in either step.
Composition is performed with a publishing application and trapping accomplished at this stage is known as application-level trapping. In application-level trapping, the traps are created by the application itself such as Adobe Illustrator, while the page to be printed is in the native application format, or by an application written specifically to generate traps from the representation of the page in a well-known page description language (PDL) such as PDF or PostScript. This kind of trapping is semi-automatic in the sense that the user has explicitly to include a trapping step in the workflow, but the application can then analyze the page and generate the traps. Application level trapping has the advantages that the user can apply any special trapping required by the printing or display process, and some feedback in the form of a preview and trapping progress indicator can be provided. However it has the disadvantage that it can produce unintended results because information of the physical properties of the printing device and colorants may not be available when the traps are being generated. Editability of trapped pages is reduced because the user has to manage the trapping marks if a change is made to the page. The present invention does not perform application-level trapping.
Raster image processing is performed with a Raster Image Processor (RIP) which accepts the representation of the page in PDL format and from it generates image rasters acceptable to the printing or display device. Trapping accomplished at this stage is known as in-RIP trapping. It has the advantage of creating the most accurate traps because all the pertinent information about the device and the colorants is available. Since trapping is the last stage of modification in the workflow, there is minimal interference with any other step.
As is well known, in the PDL datastream supplied to the RIP the digital image format is a vector or object representation; in the output to the device it is a raster format. In-RIP trapping, in turn, can be accomplished at either of these levels or in a hybrid combination. The traps can be calculated as the page description lays down each object on the raster or alternatively the entire page can be rasterized before any trapping is performed. The former case, known as object-based trapping, has the advantages that object type can be taken into account to handle complex trapping rules and performance is favored because object information such as the path and color of the entire outline are available and do not have to be calculated. Object-based trapping has the disadvantages that it is complicated in cases where several objects are overlaid on top of each other and transparent objects need to be ‘flattened’ (a process in which the final color of transparent objects is determined by applying the transparency value to the color of the transparent object and to the color of the background) before any trapping can be performed on them. Also, it can conveniently be implemented only in software, unfavorably impacting RIP performance.
The case in which the page is rasterized completely before trapping begins is known as raster-based trapping. In this case, each pixel is analyzed along with its neighbors in its plane and in the other color planes to calculate the traps. This has the advantage of decoupling the trapping process from the rasterizing process and it allows incorporation into a program library or even into an ASIC for a hardware solution. The disadvantages are that object information can be lost and a performance penalty may be incurred through having to address each pixel at least twice in the process, first when the pixel is rasterized and second when it is trapped. Performance problems can be mitigated by incorporating trapping into an ASIC or by integrating it into other steps in the page pipeline process such as black generation, tag bit generation, etc.
In addition, a hybrid approach with features and methods of both cases can be designed that retains after rasterization some object information which can be used to facilitate raster based trapping.
Implementation and Operation
Trapping is based on the cyan (C), magenta (M), and black (K) planes. There are four steps:
-
- 1. Detect object edges
- 2. Detect coincident and opposing edge transitions on each pair of planes (CM, CK, and KM)
- 3. Determine which plane to trap (to extend object across edge)
- 4. Generate trap on that plane
Edges are detected by measuring spatial rates of change of color value and comparing the color values across the edges against several thresholds. Once the edges in a page raster have been identified, the information in the edge data and in the original color plane is used to perform the actual trap calculations. The output of the trap calculations identifies the color plane or planes that should be trapped and the direction in which the trap should be generated. Traps are generated by using input from the trap calculations and performing a morphological operation as simple as copying the value of the pixel from one side of the color edge to the neighboring pixel on the other side of the color edge, or two pixels across and perpendicular to the color edge or a more complex operation such as a gray level dilation operation.
In its application to printing with color electrographic printers, the invention requires as input raster data for the color planes either for each plane separately or in packed form with data for all colors for each pixel. Thus, for example in the case of four colors CMYK, in planar form the data would be organized into four arrays of pixels and in packed form it would be as a single array with each element representing all four color values for one pixel. In one possible realization the invention is implemented in ISO C++. It is a layered architecture that allows pluggable and interchangeable components in the form of templatized classes that can be combined by means of the C++ template mechanism to generate the code for a complete trapper. Trapping is a computationally intensive process, and since the characteristics of electrographic printers vary greatly, it must be able to handle a wide variety of real world situations. This goal conflicts directly with those of trapping in real time with high performance and low cost. Therefore a hybrid approach has been utilized that capitalizes on the expressivity of C++ templates and operator overloading as well as the terse and fast assembly code generated by C code, resulting in a flexible architecture that does not compromise on speed and cost.
The present invention can be implemented as a raster-based trapping system and as a hybrid system. It operates on multi-bit per pixel raster data. Other information about the content of the page raster or the source of graphic elements that comprise the page is not essential, i.e. whether graphic elements are derived from photographic images, are color gradients, text glyphs or geometrical shapes. However, if in the image data available to the invention image objects that are photographs are identified as such, the information can be used to adjust trap thresholds.
The present invention can be implemented as an application-level trapper or as an in-RIP trapper in the printer. When implemented in a printer it does not conveniently provide a preview but as application-level software it can do so.
Object Edge Detection
Object edge detection is performed on each color plane independent of other color planes. The well-known method of convolution with a so-called Laplacian kernel is used for each pixel in the color plane. The result can be thought of as, in effect, a new plane of convolved values whose dimensions correspond pixel-for-pixel with the original plane. For efficiency and resource reasons in a practical software implementation for trap widths up to one pixel only three scanlines are convolved at a time. This three scanline window starts at the top of the page raster image and moves downwards to the bottom of the page one scanline at a time as each scanline is trapped. For trap widths up to two pixels, five scanlines are used in the same way. In practical hardware, convolutions are calculated, but they are not saved to a “new plane.” Thus a vertical “swath” of five convolution values (for 1.5 and 2.0-pixel traps) is generated in a hardware pipeline.
The Laplacian kernel is an array of weights applied to a corresponding region of pixels for the purpose of performing a convolution calculation on an image array. The center element of the kernel is used to multiply the value of a pixel which is the current subject of the convolution calculation. The other elements are used to multiply the values of the corresponding pixels immediately adjacent to the current pixel. The sum of the products is the value of a new element in a new array created by the convolution process. Laplacian kernel (1-4) shown in
Convolution is the process of adding the weighted values of the pixel in question and its neighboring pixels. The weights are generally composed of a 3×3 matrix such as the well-known examples shown in
The direction of the color change is encoded in the signs of the convolved values. Thus if the color value increases from left to right on a vertical edge, then the convolved value will be positive to the left of the edge and negative to the right. Conversely if the color value decreases from left to right on a vertical edge, then the convolved value will be negative to the left of the edge and positive to the right of the edge. This information is used when applying the trapping rules.
The 1-4 Kernel is preferred to the 1-8 kernel because it has been found to be to be more sensitive to diagonal edges. Also, because some weights are 0 the corresponding pixels need not be taken into account, thus enabling higher performance. With the 1-4 kernel convolved values range from −1020 to +1020.
Since each color plane is convolved separately, an example of a portion of a single plane is shown in
In
Convolution by Partial Registers
Convolution processing applies to every pixel in a page raster and there is a very large number of pixels in a high-quality image. Therefore convolution is a CPU- and memory bus-intensive process and software optimization techniques are useful in practice. Consider the convolution of an image performed by the following Laplacian kernel:
R=A+C+D+E−4*B function #1
Where A through E are 8-bit unsigned pixel values and the result is denoted by R. Also consider the generally available execution units of a contemporary example superscalar processor namely a load/store unit and two arithmetic logic units (ALUs), and how the above function can be scheduled to execute on such a processor given that the execution units are capable of simultaneous parallel operation. Such work can be scheduled in 4 CPU cycles on the superscalar processor that is capable of executing a load or a store in 1 cycle but with a latency of 2 cycles. Given first and second ALUs, the actual calculation can be performed in 3 cycles with a one cycle stall in the second ALU. Both ALUs then stall for one cycle for lack of data. The values used in this calculation would have been loaded into scratch registers during the previous 4 cycles, and while calculating the R value for the data thus previously loaded the values for the next 4 cycles would be loaded. This ‘time shifting’ of calculations tends to eliminate processor stalls caused by data dependencies. While there are five inputs to the equation only three new values (e.g. C,D,E) need be loaded per four-cycle iteration because two values can be reused from the previous four-cycle iteration (e.g. A, B).
To further optimize the execution of function #1 certain observations are useful. The unsigned pixel values are constrained to between −1020 and 1020. As each register in the superscalar processor has a word width of 32 bits, many bits remain unused as long as R neither overflows nor goes negative. Overflow is impossible given the pixel value range constraint, and R can be prevented from going negative by modifying function #1 thus:
R+1020=1020+A+C+D+E−4*B function #2
Function #2 can always be performed in 11 bits (0-2040). Each register can then be logically divided into partial registers (bit fields) and more than one calculation can be performed simultaneously. To convolve by partial registers each data load becomes a word (32 bit) load where each input value (A, B, C, D, E) is 16-bit aligned instead of 8-bit aligned. 16-bit alignment is used rather than 11 bits since it is the next power of 2 up from 11. There are therefore two pixels represented in each word. These paired loads are then fed directly to the ALUs resulting in an R value which contains two separate convolution results. The user of R then need only subtract 1024 from each 16 bit field to obtain a signed value. Since R is often used for relative comparisons this subtraction may not even be necessary.
It will be understood that such parallel processing can be used with superscalar processors having numbers of ALUs and word widths that differ from the example superscalar processor.
After all the pixels in the input plane have been convolved, the convolved plane contains the values shown in
The convolved value is important also because its absolute value encodes the rate at which the color value changed. The previous simple example does not illustrate this point, but when color gradients and more complex color changes such as in photographic images are introduced, the convolved value can be thresholded to eliminate small differences in color from pixel to pixel. Thresholding is a point process and the result is +1 or −1.
Detect Coincident and Opposing Edge, Determine which Plane to Trap, Generate Trap
Now that the positions of the edges are known, the color values of the Cyan, Magenta & Black planes across both sides of the color edge are used as well as the direction of the edge to generate traps. Assume a vertical color edge, which would mean the color change in the pixels is horizontally across the edge; denote the pixel values of the Cyan Magenta & Black planes on the left side of the edge as CL, ML & KL; similarly call the color values on the right hand side of the edge for the three color planes CR, MR & KR respectively. Note that it is here that the Relative Toner Density Lookup Tables will be used.
Trapping Parameters:
The present invention takes the Toner density curves or engine transfer function into account when calculating and generating traps. It requires this data in the form of look-up tables (referred to as Relative Toner density LUT, Toner Density LUT or simply LUT) for each Toner (called the Referent Toners) with 256 entries each, with the value in each entry corresponding to the relative density of that Toner with respect to the Black Toner (which will be called the Reference Toner) at the same index in the table. The LUT for the Black Toner should contain values increasing monotonically from 0 to 255.
In case a colorant has higher neutral density than the Black Toner, as may be the case for some marking engines, then that Toner will be the Reference Toner and the LUTs for the Referent Toners should be generated relative to the Reference Toner and its LUT must contain entries from 0 to 255 as described for Black Toner above (Black Toner will be considered a Referent Toner in this case). Since the Toner transfer curves are non-linear, it may happen that the density of a Toner at a certain level may exceed that of the Black Toner (or whichever Toner is the Reference Toner) at that same level, in this case the relative Density in the Referent LUT must be clamped at 255.
The following pseudo-code may be used to generate the relative LUT for each Referent Toner, given the engine transfer curves for each toner and assuming the engine transfer curve has 256 entries. If that is not the case, then the engine transfer curves will have to be interpolated to obtain exactly 256 entries in the engine transfer curve.
Loop for index from 0 to 255 using index i
Entry in Referent Toner LUT at index i=(Density of Referent Toner at index i/Density of Reference Toner at index i)*255.
Repeat loop. It will be understood that integer overflow and underflow issues must be handled correctly.
Simple Case of a Vertical Color Edge
The following steps will be taken to generate traps:
Find edges in all three planes independently of each other as shown in the description of “Edge detection” given above. Call the convolved pixel values in the three planes across the color edge CLC, MLC & KLC on the left hand side of the edge and CRC, MRC & KRC on the right hand side of the edge respectively.
Considering the Cyan and Magenta planes first, the flowchart in
Another trap generation rule is the Super Black trapper. In this case, if the K level is above a given threshold, say 75% to 100% depending upon how dark the K toner is and the Super Black contains some amount of Cyan or Magenta, then none of the aforementioned rules are evaluated and Cyan and Magenta values outside the object containing the K toner are choked into the object, thus avoiding Cyan or Magenta fringes around Super Black objects, especially if they are on a white or light colored backgrounds as may be the case for black text. In the present invention the super black trapping rule states that if the value of the K colorant is above the super black threshold (which is a function of the particular electrographic process) and there is a significant difference between the cyan or magenta component (or both) in the black object and the same component or components in the background, then all other trapping rules are ignored and the cyan and magenta colorants in the background always choke into the super black object, or vice-versa in the case of a light colored object on super black background. Since by definition the super black threshold is high, this rule applies to very dark objects only and the choking will not result in a visible halo. The rule can also be generalized for the magenta and cyan colorants such that if the density values of the different colorants that comprise a colored object on a white (or very light) background are all very high, then the lighter colorant may be choked into the object to prevent color halos from appearing in the case of misalignment. Note that this rule runs counter to the usual one of generating a trap only if two colorants change values across the edge in opposite directions. This rule is invoked especially when the colorants change value in the same direction across the colored edge, i.e. the object contains large amounts of cyan, magenta and black whereas the background contains very little or none of the colorants, or vice-versa in the case of a white object on a very dark background.
Although the flowcharts in
Edge Detection Across Color Planes
After the convolve operation, pairs of these convolved planes are examined to find color edges that need to be trapped. Using the algorithm shown in
Generation of Traps of Various Widths
With reference to
Where:
- Ci=Value of Cyan pixel on the left side of the color edge
- Ci+1=Value of Cyan pixel on the right side of the color edge
- [ ]=the round up to nearest integer operator
A 1½ pixel wide trap is simulated by adjusting values of C1i and C1i−1 as shown at table 15.5c in
It will be understood that the same rules apply in the cases of trap generation for other directions and edge orientations and in other color planes.
Practical Consolidation of Simple Trap Generation and Single Trap Generation
The somewhat complex set of decisions and operations laid out in
Identify which of the two pixels on each side of an edge has the larger value in each color plane. There are eight possibilities.
Of the three larger values identify the color of the middle value. There are three possibilities, cyan, magenta or black.
It will be understood that there are then 8×3=24 possible outcomes of these two identification steps, outcomes established by the rules illustrated in
Note that the method illustrated in
Hardware
A trapper hardware implementation must satisfy the following general objectives:
Functional: Implement the trapping algorithm described above.
Design: Be compatible with high-performance imaging pipelines used in raster image processors. In the example described here, the hardware is envisaged as a combination of application-specific logic, referred to here as “chip” or “logic,” with a programmable general-purpose CPU and its associated system memory. Minimize use of memory and avoid using temporary buffers in system memory, instead working only from input buffers and writing once to output buffers. Minimize use of memory bandwidth by using burst accesses and minimum re-fetching of data.
Size: In the example described the trapper logic is economical in its use of gates, including flop-based register files and FIFOs, and on-chip memory.
This section highlights those portions of the algorithm that affect hardware implementation.
-
- 1. Detect Object Edges. The first step is to locate the colored edges of objects. The algorithm uses a convolution function and multiple levels of thresholding to detect edges on each plane—cyan (C), magenta (M), and black (K)—which are detected independently of one another. The yellow plane is not used in this step. Convolution is a function of a pixel and its neighbors. The present method uses a 3×3 convolution matrix. As will be seen later, the use of neighboring pixels places special requirements on the hardware implementation.
- 2. Detect Coincident and Opposing Edge Transitions between Planes. Once edges are found on each plane, the present method compares edges between all three planes C, M and K. If there is at least one pair of planes with coincident edges and if the edges transition from light to dark and dark to light in opposite directions (where light and dark correspond to low and high pixel values respectively), then the edge coincidence requires trapping. The hardware implementation for this step requires a number of comparators.
- 3. Determine Which Plane to Trap. Once it is determined to trap an edge coincidence, the algorithm next identifies on which plane the trap should be performed, and in what direction the trap should occur. This step is straightforward to implement in hardware.
- 4. Generate Trap on That Plane. Generation of a trap means copying a pixel value (or percentage of a pixel value) from one side of an edge to another. The method supports ½-pixel to 2-pixel traps (in ½ pixel increments), so the hardware must support moving of a value by up to two pixels from its origin.
It will be understood that the Trapping algorithm is a configurable algorithm that can be operated in different modes with different options. For this example of implementation in hardware, Tables 1, 2 & 3 show which modes, options and configurations are supported.
Hardware Glossary
Table 4 defines terms that are used in this description. It will be understood that where specific numeric values are given it is to maintain consistency within the example given in this description and that other consistent numeric values can be chosen by those skilled in the art.
The trapper hardware is a component in a pipeline of image processing stages in a raster processing unit (RPU) which is itself part of a raster image processor. As in customary practice in the field, images are in the form of arrays of pixel data C, M, Y and K representing cyan, magenta, yellow and black color separations respectively, plus a fifth array, labelled T for tag, to carry attribute information required for certain image processing steps. Examples of the attribute information include pixel-by-pixel selection of color conversion table and halftone table. As shown in the example of
Rotator Data Format Requirement
A customary design requirement for an image processing pipeline is that it use little memory, which is generally addressed by dividing plates and panes into bands, stripes, and subbands. Near the end of the pipeline, raster images must be able to be rotated, band by band. To accomplish this economically, stripes must be divided into subbands, which the rotator is designed to rotate, which in turn allows it to step vertically through a logical plate and, as each subband is rotated, to construct a new horizontal band of subbands. This horizontal band can then be consumed by a Print Engine Video Controller (PVC) for output to the video interface of the print engine.
Although the rotator operates on subbands, good logic design requires it to break rotation of subbands into cells 32 pixels wide by 32 scanlines high, for which it uses a 1 Kbyte block of memory per plane. To support this operation, the RPU and trapper provide subbands divided into cells that can be individually rotated by the Rotator. It should be noted that subbands are compressed as they exit the RPU, and they are decompressed after they enter the rotator module. However, this is not significant to this data format discussion nor to the requirements that data format places on these stages of the pipeline.
Hardware Implementation versus Software Implementation
It will be understood that implementations may differ in ways that do not conflict with the principles of the present invention. For example, hardware and software implementations are likely to show characteristic differences in how each processes pixels. Whereas software implementation may well use temporary buffers to hold convolution results for each plane it is a practical goal of hardware to not require temporary buffers in system memory, in order to minimize memory bandwidth and memory consumption. Another significant difference concerns data access. Input data to Trapper is randomly accessible. That is, any portion of a band can be read and re-read, or revisited, during processing, and in fact Trapper takes advantage of this to re-fetch pixels that neighbor the cell it is currently processing. Output pixels, however, cannot efficiently be re-written or revisited. As each cell of each subband is processed, it is compressed and written to memory. Once compressed, those output pixels may not be accessed again. However, with reference to
The case for software is shown in
The case for hardware is shown in
Given the ordering of processing of cells through a subband, this presents a difficulty at the edges of a cell for hardware implementation. If a horizontal edge coincides with the top of the band boundary, and if the trapping algorithm determines that a pixel value should be moved from the bottom side of the edge to the top, this would require moving the pixel across the band boundary into the last scanline of the previous band. However, that band no longer exists in an uncompressed form; instead, it is stored, compressed, in a page store in system memory.
Conceptually, for the hardware implementation, the following rule applies: the Trapper must process data by making all necessary updates to an output pixel and, in general, it may not make a second update to a pixel, especially to a pixel in the scanline above. This allows the Trapper to turn the pixel over to the next stage of the pipeline without needing to access it again. This is important for the imaging pipeline, since the next stage is the Subband Compressor, and once a pixel is compressed, it may be difficult to access it again. Thus, the bottom half of
Two-Pixel Trap Nine-Way Convolution
In the discussion of differences between software and hardware implementations, parallel processing of convolutions was introduced to deal with band boundary conditions. To perform this analysis, neighboring pixels within the central scanline are required, in addition to neighboring pixels in the previous and successive scanlines. In the case of 1.5- and 2.0-pixel traps, the Trapper requires seven scanlines of pixels to be available to it as can be seen in
A larger issue for parallelism, however, concerns one-dimensional Relative Toner Density (RTD) Lookup Tables (LUTs). It might seem that many LUTs would have to be duplicated to allow for parallel decision processing. However, only five duplicates of one-dimensional LUTs per plane are required, one for each string of pixels that passes horizontally through the five-scanline high convolution “+” pattern in
Extension for Greater Registration Errors
It will be understood that traps must be wider than two pixels if registration errors exceed two pixels, and that more convolutions and correspondingly more buffering will be required according to straightforward extension of the examples described here.
Imaging Pipeline Data Flow
As shown in
Trapper Module Data Format and Order
Depending on the implementation approach, the neighboring strings above and below a cell, as well as the neighboring pixels to the left and right of a cell, must be either buffered on-chip, which requires greater on-chip memory, or re-fetched from system memory, which requires greater bandwidth.
System Buffering Requirements
The convolution analysis also affects system buffering of input data. In order for Trapper to properly convolve data at the top and bottom of a stripe, it must have access to neighboring scanlines from neighboring stripes. This is depicted in
With reference to
21b are first scanlines from the next stripe—two scanlines for trap widths of 0.5 or 1 pixel, or three scanlines for trap widths of 1.5 or 2 pixels.
21c is the end of stripe address for the previous stripe.
21d is the stripe address for the current stripe.
21d is the stripe address for the next stripe.
Note that in
Data Buffering inside Trapper
In general, trapping causes pixel values to be copied from the dark side of an edge to the lighter side. The trapping algorithm defines that only original pixel values are used during the processing to determine how to modify the image. It is important for Trapper to isolate input and output pixels. Image analysis (e.g., convolution, direction, trapping thresholds) is always performed on input pixels, and modification to an image is always performed on output pixels. Thus, Trapper hardware requires output buffering to maintain output pixels separate from input pixels.
Tag Plane Processing
Through particular section of the imaging pipeline from Color Converter to Trapper to Subband Compressor data in memory is reformatted from uncompressed planar bands to compressed planar subbands. The tag plane must also follow this transformation, so that the Rotator can easily rotate the tag plane as well as the color data. It will be understood that other factors in a particular implementation will determine whether this occurs in the Trapper module or the Subband Compressor Module.
Edge Detection by Convolution
For the case of two-pixel traps, once seven strings of pixels are buffered on-chip (using block SRAMs, for example), a Trapper module must read pixel data from the block SRAMs or other generally-understood storage mechanism to have matrices of pixels available for the up to nine convolutions (L1, L2, R1, R2, T1, T2, B1, B2, C) that it must perform. In a typical implementation these matrices consist of simple flops to allow parallel access to the data by up to nine convolution modules. This exemplary hardware implementation takes advantage of the use of the 1-4 Laplacian kernel in order to minimize logic size. No multipliers are necessary, but instead for each convolution function on each plane, only one shifter, two adders and one subtractor are required. One comparator is then necessary to compare the convolved value against a convolution threshold. The result of this comparison determines whether there is an edge or not on the given plane. This result in turn is compared against results from other planes, and if the directions of the edges are opposite to one another, then this edge is a candidate for trapping.
Determine Direction
Once the comparisons in the previous step are complete, it is a simple matter to check the sign of one of the two convolved pixels on one of the plane-pairs. Its sign determines what “direction” (light-to-dark versus dark-to-light) that both plane-pairs are heading.
Apply Trap Threshold
Once two coincident and opposite edges are found, trapping is performed only if the color difference on both planes is greater than the trapping threshold. Logically, this is implemented in a well-known way with two subtractors and a comparator for each plane in the pair, and ANDing the result together. A true output indicates that a trap is to be performed.
Generate Trap
Once the decision to trap is made, which pixel on which plane is converted from one code to the other must be determined by generally-understood means using the results from the “determine direction” step plus a simple 8-bit comparator. The result of this causes a pixel value to write over its neighbor.
Design Optimization
It will be understood that Trapper hardware can be implemented in a variety of ways, such as that shown in the Trapper hardware block diagram,
The performance of a Trapper module can be assessed by two measures: processing throughput and I/O throughput. Processing throughput is the rate at which data can be processed through the Trapper pipeline, assuming no I/O limitations. I/O throughput is the rate at which data is read into and written out of Trapper's I/O ports, assuming no internal processor limitations. Also since the Trapper implementation re-fetches neighboring pixels, there is an inherent inefficiency, or overhead, which is analyzed below. However, note that this is a design tradeoff between the cost of bandwidth in the system and the cost of on-chip memory.
In order to process one composite pixel per clock, it will be understood that the Trapper design may require duplicate convolution and trapping logic blocks, as well as duplicate Relative Toner Density Lookup Tables and that pipelining may be required, all according to well-known methods. The convolution and trapping functions are composed of multi-level adder trees and followed by additional levels of comparators. Depending on the actual speed of the ASIC cell library, pipeline stages might have to be inserted.
Trapper Bandwidth Overhead
As discussed here, a Trapper performs its processing on neighborhoods of pixels. Also discussed is the fact that Trapper must sequence through rasters in a cellular order within a subband. These two requirements cause Trapper to require access to the same pixel more than once, as it processes one cell and then the next. Accessing a pixel more than once is called re-fetching, and the bandwidth requirements of the Trapper are affected by how much it has to re-fetch data, known as its bandwidth overhead factor, or KOH.
Trapper bandwidth overhead can be improved by adding on-chip memory. Instead of re-fetching data, Trapper can fetch the data once, and then maintain it on-chip for use as required. However, given a goal of limiting major blocks, for example to not more than 8 Kbytes of memory, and given that the size of the data structures for Trapper's processing pipeline are on the order of cells and subbands, re-fetching is the preferred trade-off. Nevertheless, there are two practical design approaches: one that uses a minimum amount of additional on-chip memory, and one that uses more on-chip memory to reduce bandwidth. Ideally, a Trapper reads each pixel only once in an efficient burst cycle. For one subband, in the current example, this would amount to 256 bursts per column. However, due to re-fetching, the Trapper must fetch additional bursts. The analysis that follows determines how many bursts each respective approach needs to make to handle all fetches and re-fetches, and then compares these burst counts against the ideal, to compute the bandwidth overhead factors, KOH. These KOH values are used in the subsequent section to determine total bandwidth requirements for the Trapper in this ASIC.
A first design approach requires minimal on-chip pixel processing memory, for example just 1,330 bytes (32 B/line×7 lines×5 planes) for all five planes, but it requires a large amount of bandwidth. For this design approach, a Trapper must read three additional strings above and three below each column of cells, as shown in
A second design approach for two-pixel traps requires more pixel processing memory, for example 12,730 bytes (67 B/line×38 lines×5 planes), to maintain two cells (plus neighboring pixels) on-chip, but in return cuts bandwidth by up to half. As shown in
The two design approaches may be compared for respective the bandwidth overhead factors of 3.07, the worst case value for the first design approach, and 1.48 respectively.
In an example implementation, each image plane contains 32 MB of data and the total for CMYT is 128 MB. For this example it is assumed that an average CMYKT compressed image contains 35 MB. With reference to
- Input raw CMYT image 128 MB/s
- Output compressed CMYK image 35 MB/s
- Total net bandwidth 163 MB/s
The actual bandwidth requirement is the net bandwidth multiplied by the bandwidth overhead factor. Therefore, the first design approach requires 163×3.07=500.4 MB/s and the second design approach requires 163×1.48=241.24 MB/s, the ratio being approximately 2:1 for the first design approach relative to the second design approach.
Note that in addition to the pixel processing memory, it will be understood that each of the design approaches requires additional memory for the Relative Toner Density look-up tables and, assuming the 32×32-pixel cell organization illustrated in
(3 CMK planes)×(256 entries)×(1 byte/entry)×(5 copies)=3,840 bytes
Therefore the additional memory is 160+3,840=4,000 bytes and the total memory requirements for the first and second design approaches respectively are: 1,330+4,000=5,330 bytes and 12,730+4,000=16,730 bytes, the ratio being approximately 1:3.14.
Either of the design approaches as well as others with various well-understood tradeoffs between memory bandwidth and memory consumption may be taken for implementing the present invention depending on application requirements.
Referring now to
This is but one example of implementing the trapping algorithm in hardware. Implementation can vary to satisfy different requirements. The implementation above provides high performance by using identical hardware processing blocks and running them in parallel with one another. This provides high throughput while avoiding a requirement for large scanline buffers on-chip. Other implementations could reduce the size of the hardware by taking advantage of relaxed performance requirements, or by taking advantage of large scanline buffers.
The hardware shown in
convolve 1×4=a01+a10+a12+a21−(a11<<2);
This Verilog statement can synthesize to four adders for each of the 15 convolve 1×4 functions that are referenced in trcore.
For each of the C, M, and Y planes, the Relative Toner Density Lookup Table Block (26b) is composed of five 256×8 look-up tables, for a total of fifteen. These look-up tables can be implemented with flip flops, register arrays, or block RAMs. For slower applications the look-up tables can be implemented with system memory.
For a maximum trap width of two pixels, the Rules Analysis Block (26c) analyzes eight edges in parallel surrounding a given “center” pixel. In one implementation the edge analysis is embodied in a Verilog module named “analyzeedge” that is instantiated eight times corresponding to the eight edges. Each “analyzeedge” module takes a pair of C, M, and K convolution values from the Convolution Block (26a), and a pair of one-dimensionally mapped C, M, and K pixel values from the Relative Toner Density Lookup Table Block (26b), plus trapping algorithm controls, such as trapping thresholds and options, and produces a single 3-bit bitmask for the edge. Up to two of the three bits in the bitmask can be set by “analyzeedge” to identify which of up to two of the C, M and K planes should trap from the far side of an edge to the near side. This is a complex module that requires comparators and absolute functions that would be synthesized as a series of adders, multiplexers, inverters, and random discrete logic, including 8- and 10-bit adders.
The Prioritization Block (26d) takes the bitmask outputs of all eight instances of the analyzeedge module and prioritizes them so that only one of the eight edges is selected to have its far-side component pixels trapped into the “center” composite pixel. In one implementation, this prioritization is embodied in a series of if-then-else Verilog statements, which would be synthesized into discrete logic. The output of this logic is a decode of which of the eight edges should be trapped, and on that edge, which planes should be trapped.
In one embodiment, the final block, the Trapping Block (26e), is composed of eight sets of Verilog function references “trappix”, one for each of the eight edges that may trap. Each set of “trappix” functions operates on each of the five planes in a composite pixel: C, M, Y, K, and A. Thus, there is a total of 8 edges×5 planes, or 40, “trappix” references. Each “trappix” function traps a single plane, and takes control inputs as well as 8-bit unsigned far and center pixels, and returns an 8-bit unsigned trapped pixel result, which is either the original far input pixel value, the original center input pixel value, or in the case of ½ and 1½ pixel trap widths, the average of the far and center pixel values. This averaging requires an 8-bit adder. Otherwise, “trappix” can be implemented with a Verilog case-statement, which can synthesize into multiplexers controlled by decode logic.
Claims
1. A system for reducing visibility of registration errors in an image to be printed using a digital color printer, the image having cyan (C), magenta (M), yellow (Y), and black (K) color planes, each color plane having a plurality of respective C, M, Y and K pixels, said pixels used to form objects on said C, M, Y and K color planes, said system comprising:
- a) means for detecting colored edges of colored objects on each of the C, M and K color planes;
- b) means for detecting coincident and opposing edge transitions on CM, CK and KM color plane pairs:
- c) means for performing processing on the detected coincident and opposing edge transitions to determine the color plane and direction to extend the objects, if necessary;
- d) means for spreading or choking the objects across the determined color plane using a simple trap generation rule and a single trap generation rule wherein said means for spreading or choking using said single trap generation rule comprises, with respect to first and second color planes, if there are extensions on both sides of the color edge, determining which two colorants will generate said extensions and, determining which of the two extensions is darker and generating the darker extension, wherein said means for detecting said edges of objects comprises:
- means for measuring spatial rates of change of color value and comparing the color values across the edges against at least two thresholds on each color plane independent of other color planes using a Laplacian kernel for each pixel in the color plane,
- wherein said means for performing processing to determine the color plane and direction to extend the objects comprises:
- means for comparing convolved pixel values for each edge of each color plane, wherein said convolved pixel values are obtained by performing a convolution calculation for each pixel in each of said color planes; and
- wherein said means for spreading or choking comprises nine convolution matrices, wherein the nine convolution matrices include up to nine convolution modules in a hardware implementation to allow parallel access to image data, wherein the hardware implementation uses a 1-4 Laplacian kernel, including one shifter, two adders, one subtractor and, one comparator for each convolution module,
- wherein the comparator compares a convolved pixel value against a convolution threshold, to determine whether there is an edge or not on for each color plane, and the result is compared with the results from other planes, and the edge is a candidate for trapping if the directions of the determined edges are opposite to one another, wherein the nine convolution matrices include up to nine convolution modules in a hardware implementation to allow parallel access to image data, wherein the hardware implementation uses a 1-4 Laplacian kernel, including one shifter, two adders, one subtractor and, one comparator for each convolution module,
- wherein the comparator compares a convolved pixel value against a convolution threshold, to determine whether there is an edge or not on for each color plane, and the result is compared with the results from other planes, and the edge is a candidate for trapping if the directions of the determined edges are opposite to one another.
2. The system defined by claim 1 wherein said means for detecting coincident and opposing edge transitions comprises:
- means for comparing look-up table values of opposite edges of each of said C, M and K color plane pairs with predetermined threshold, said look-up table values corresponding to C, M and K toner density, respectively.
3. The system defined by claim 2 herein said comparing means uses sequential testing to determine if a current pixel will generate a trap, and if no trap will be generated, skipping remaining tests.
4. The system defined by claim 1 wherein said means for spreading or choking using said simple trap generation rule comprises, with respect to first and second color planes,
- i) if a look-up table pixel value on a right side edge of the first color plane is greater than a look-up table pixel value on a left side edge of the second color plane, logic to extend a corresponding second color plane pixel from left to right,
- ii) if a look-up table pixel value on a right side edge of the first color plane is not greater than a look-up table pixel value on a left side edge of the second color plane, logic to extend a corresponding first color plane pixel from right to left,
- iii) if a look-up table pixel value on a left side edge of the first color plane is greater than a look-up table pixel value on a right side edge of the second color plane, logic to extend a corresponding second color plane pixel from right to left,
- iv) if a look-up table pixel value on a left side edge of the first color plane is not greater than a look-up table pixel value on a right side edge of the second color plane, logic to extend a corresponding first color plane pixel from left to right.
5. The system defined by claim 1, wherein said means for spreading or choking using said single trap generation rule ensures that traps are generated on only one side of a color edge.
6. The system defined by claim 1 wherein said means for spreading or choking operates to provide trap widths of 0.5, 1, 1.5 and 2 pixels.
7. A method for reducing visibility of registration errors in an image to be printed using digital color printer, the image having cyan (C), magenta (M), yellow (Y), and black (K) color planes, each color plane having a plurality of respective C, M, Y and K pixels, said pixels used to form objects on said C, M, Y and K color planes, said method comprising:
- a) detecting colored edges of objects on each of the C, M and K color planes;
- b) detecting coincident and opposing edge transitions on CM, CK and KM color plane pairs;
- c) performing processing on the detected coincident and opposing edge transitions to determine the color plane and direction to extend the objects;
- d) spreading or choking the objects across the determined color plane using a simple trap generation rule and a single trap generation rule, wherein said detecting said edges of objects comprises:
- measuring spatial rates of change of color value and comparing the color values across the edges against at least two thresholds on each color plane independent of other color planes using a Laplacian kernel for each pixel in the color plane,
- wherein said performing processing to determine the color plane and direction to extend the objects comprises:
- comparing convolved pixel values for each edge of each color plane, wherein said convolved pixel values are obtained by performing a convolution calculation for each pixel in each of said color planes using nine convolution matrices, wherein the nine convolution matrices include up to nine convolution modules allowing parallel access to image data, using a 1-4Laplacian kernel, including a comparator for each convolution module,
- wherein the comparator compares a convolved pixel value against a convolution threshold, to determine whether there is an edge or not on for each color plane, and the result is compared with the results from other planes, and the edge is a candidate for trapping if the directions of the determined edges are opposite to one another.
8. The method defined by claim 7 wherein said detecting coincident and opposing edge transitions comprises:
- comparing look-up table values of opposite edges of each of said C, M and K color plane pairs with a predetermined threshold, said look-up table values corresponding to C, M and K toner density, respectively.
9. The method defined by claim 7 wherein said spreading or choking using said simple trap generation rule comprises, with respect to first and second color planes,
- i) if a look-up table pixel value on a right side edge of the first color plane is greater than a look-up table pixel value on a left side edge of the second color plane, extending a corresponding second color plane pixel from left to right,
- ii) if a look-up table pixel value on a right side edge of the first color plane is not greater than a look-up table pixel value on a left side edge of the second color plane, extending a corresponding first color plane pixel from right to left,
- iii) if a look-up table pixel value on a left side edge of the first color plane is greater than a look-up table pixel value on a right side edge of the second color plane, extending a corresponding second color plane pixel from right to left,
- iv) if a look-up table pixel value on a left side edge of the first color plane is not greater than a look-up table pixel value on a right side edge of the second color plane, extending a corresponding first color plane pixel from left to right.
10. The method defined by claim 7, wherein said spreading or choking using said single trap generation rule ensures that traps are generated on only one side of a color edge.
| 5481379 | January 2, 1996 | Yosefi |
| 5581667 | December 3, 1996 | Bloomberg |
| 6549303 | April 15, 2003 | Trask |
| 6970271 | November 29, 2005 | Estrada et al. |
| 20010055130 | December 27, 2001 | Geurts et al. |
| 20020167526 | November 14, 2002 | Kinoshita |
| 20030044065 | March 6, 2003 | Harrington |
| 20030142314 | July 31, 2003 | Hubble, III et al. |
| 20030197877 | October 23, 2003 | Lee |
| 20040095590 | May 20, 2004 | Megawa et al. |
| 20040212628 | October 28, 2004 | Becker et al. |
| 20040263834 | December 30, 2004 | Alumot et al. |
| 20050168644 | August 4, 2005 | Demas et al. |
| 20050219631 | October 6, 2005 | Accad et al. |
| 0 996 282 | April 2000 | EP |
Type: Grant
Filed: Apr 24, 2007
Date of Patent: Jun 12, 2012
Patent Publication Number: 20080007752
Assignee: Kyocera Mita Corporation (Osaka)
Inventors: Lakhbir Gandhi (Auburn, WA), Robert Westervelt (Los Angeles, CA), Kenneth A. Schmidt (Redondo Beach, CA), Dongpei Su (Lomita, CA), Alfred Abkarian (La Crescenta, CA), Mark Raley (Torrance, CA)
Primary Examiner: Benny Q Tieu
Assistant Examiner: Qian Yang
Attorney: Blakely, Sokoloff, Taylor & Zafman
Application Number: 11/789,206
International Classification: G06F 15/00 (20060101);
